Configuration options for the Live API


Even with the basic implementation for the Live API, you can build engaging and powerful interactions for your users. You can optionally customize the experience even more by using the following configuration options:


Response voice and language

You can make the model respond in a specific voice and influence the model to respond in different languages.

Specify a response voice

Click your Gemini API provider to view provider-specific content and code on this page.

The Live API supports 30 different synthisized HD voices, each with distinct characteristics. You can view a list of response voice options and hear demos of each voice by expanding the section below.

If you don't specify a response voice, the default is Puck.

To specify a response voice, set the voice name within the speechConfig object as part of the model configuration.

Swift 


// ...

let liveModel = FirebaseAI.firebaseAI(backend: .googleAI()).liveModel(
  modelName: "GEMINI_LIVE_API_MODEL_NAME",
  // Configure the model to use a specific voice for its audio response.
  generationConfig: LiveGenerationConfig(
    responseModalities: [.audio],
    speech: SpeechConfig(voiceName: "VOICE_NAME")
  )
)

// ...

Kotlin 


// ...

val model = Firebase.ai(backend = GenerativeBackend.googleAI()).liveModel(
    modelName = "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to use a specific voice for its audio response.
    generationConfig = liveGenerationConfig {
        responseModality = ResponseModality.AUDIO
        speechConfig = SpeechConfig(voice = Voice("VOICE_NAME"))
    }
)

// ...

Java 


// ...

LiveGenerativeModel lm = FirebaseAI.getInstance(GenerativeBackend.googleAI()).liveModel(
    "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to use a specific voice for its audio response.
    new LiveGenerationConfig.Builder()
        .setResponseModality(ResponseModality.AUDIO)
        .setSpeechConfig(new SpeechConfig(new Voice("VOICE_NAME")))
        .build()
);

// ...

Web 


// ...

const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });

const liveModel = getLiveGenerativeModel(ai, {
  model: "GEMINI_LIVE_API_MODEL_NAME",
  // Configure the model to use a specific voice for its audio response.
  generationConfig: {
    responseModalities: [ResponseModality.AUDIO],
    speechConfig: {
      voiceConfig: {
        prebuiltVoiceConfig: { voiceName: "VOICE_NAME" },
      },
    },
  },
});

// ...

Dart 


// ...

final _liveModel = FirebaseAI.googleAI().liveGenerativeModel(
  model: 'GEMINI_LIVE_API_MODEL_NAME',
  // Configure the model to use a specific voice for its audio response.
  liveGenerationConfig: LiveGenerationConfig(
    responseModalities: [ResponseModalities.audio],
    speechConfig: SpeechConfig(voiceName: 'VOICE_NAME'),
  ),
);

// ...

Unity 


// ...

var liveModel = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI()).GetLiveModel(
    modelName: "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to use a specific voice for its audio response
    liveGenerationConfig: new LiveGenerationConfig(
        responseModalities: new[] { ResponseModality.Audio },
        speechConfig: SpeechConfig.UsePrebuiltVoice("VOICE_NAME")
    )
);

// ...

Influence the response language

The Live API models automatically choose the appropriate language for their responses.

If you want the model to respond in a non-English language or always in a specific language, you can use influence the model's responses by using system instructions like these examples:

  • Reinforce to the model that a non-English language may be appropriate

    Listen to the speaker carefully. If you detect a non-English language, respond
in the language you hear from the speaker. You must respond unmistakably in the
speaker's language.

  • Tell the model to always respond in a specific language

    RESPOND IN LANGUAGE. YOU MUST RESPOND UNMISTAKABLY IN LANGUAGE.


Transcriptions for audio input and output

Click your Gemini API provider to view provider-specific content and code on this page.

As part of the model's response, you can receive transcriptions of the audio input and the model's audio response. You set this configuration as part of the model configuration.

  • For transcription of the audio input, add inputAudioTranscription.

  • For transcription of the model's audio response, add outputAudioTranscription.

Note the following:

  • You can configure the model to return transcriptions of both input and output (as shown in the following example), or you can configure it to return only one or the other.

  • The transcripts are streamed along with the audio, so it's best to collect them like you do text parts with each turn.

  • The transcription language is inferred from the audio input and the model's audio response.

Swift 


// ...

let liveModel = FirebaseAI.firebaseAI(backend: .googleAI()).liveModel(
  modelName: "GEMINI_LIVE_API_MODEL_NAME",
  // Configure the model to return transcriptions of the audio input and output.
  generationConfig: LiveGenerationConfig(
    responseModalities: [.audio],
    inputAudioTranscription: AudioTranscriptionConfig(),
    outputAudioTranscription: AudioTranscriptionConfig()
  )
)

var inputTranscript: String = ""
var outputTranscript: String = ""

do {
  let session = try await liveModel.connect()
  for try await response in session.responses {
    if case let .content(content) = response.payload {
      if let inputText = content.inputAudioTranscription?.text {
        // Handle transcription text of the audio input.
        inputTranscript += inputText
      }

      if let outputText = content.outputAudioTranscription?.text {
        // Handle transcription text of the audio output.
        outputTranscript += outputText
      }

      if content.isTurnComplete {
        // Log the transcripts after the current turn is complete.
        print("Input audio: \(inputTranscript)")
        print("Output audio: \(outputTranscript)")

        // Reset the transcripts for the next turn.
        inputTranscript = ""
        outputTranscript = ""
      }
    }
  }


} catch {
  // Handle error
}

// ...

Kotlin 


// ...

val liveModel = Firebase.ai(backend = GenerativeBackend.googleAI()).liveModel(
    modelName = "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to return transcriptions of the audio input and output.
    generationConfig = liveGenerationConfig {
        responseModality = ResponseModality.AUDIO
        inputAudioTranscription = AudioTranscriptionConfig()
        outputAudioTranscription = AudioTranscriptionConfig()
   }
)

val liveSession = liveModel.connect()

fun handleTranscription(input: Transcription?, output: Transcription?) {
    input?.text?.let { text ->
        // Handle transcription text of the audio input.
        println("Input Transcription: $text")
    }
    output?.text?.let { text ->
        // Handle transcription text of the audio output.
        println("Output Transcription: $text")
    }
}

liveSession.startAudioConversation(null, ::handleTranscription)

// ...

Java 


// ...

ExecutorService executor = Executors.newFixedThreadPool(1);

LiveGenerativeModel lm = FirebaseAI.getInstance(GenerativeBackend.googleAI()).liveModel(
    "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to return transcriptions of the audio input and output.
    new LiveGenerationConfig.Builder()
            .setResponseModality(ResponseModality.AUDIO)
            .setInputAudioTranscription(new AudioTranscriptionConfig())
            .setOutputAudioTranscription(new AudioTranscriptionConfig())
            .build()
    );

LiveModelFutures liveModel = LiveModelFutures.from(lm);
ListenableFuture<LiveSessionFutures> sessionFuture = liveModel.connect();

Futures.addCallback(sessionFuture, new FutureCallback<LiveSessionFutures>() {
    @Override
    public void onSuccess(LiveSessionFutures ses) {
        LiveSessionFutures session = ses;
        session.startAudioConversation((Transcription input, Transcription output) -> {
            if (input != null) {
                // Handle transcription text of the audio input.
                System.out.println("Input Transcription: " + input.getText());
            }
            if (output != null) {
                // Handle transcription text of the audio output.
                System.out.println("Output Transcription: " + output.getText());
            }
            return null;
        });
    }

    @Override
    public void onFailure(Throwable t) {
        // Handle exceptions
        t.printStackTrace();
    }
}, executor);

// ...

Web 


// ...

const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });

const liveModel = getLiveGenerativeModel(ai, {
  model: 'GEMINI_LIVE_API_MODEL_NAME',
  // Configure the model to return transcriptions of the audio input and output.
  generationConfig: {
    responseModalities: [ResponseModality.AUDIO],
    inputAudioTranscription: {},
    outputAudioTranscription: {},
  },
});

const liveSession = await liveModel.connect();

liveSession.sendAudioRealtime({ data, mimeType: "audio/pcm" });

const messages = liveSession.receive();
for await (const message of messages) {
  switch (message.type) {
    case 'serverContent':
      if (message.inputTranscription) {
        // Handle transcription text of the audio input.
        console.log(`Input transcription: ${message.inputTranscription.text}`);
      }
      if (message.outputTranscription) {
        // Handle transcription text of the audio output.
        console.log(`Output transcription: ${message.outputTranscription.text}`);
      } else {
      	 // Handle other message types (modelTurn, turnComplete, interruption).
      }
    default:
      // Handle other message types (toolCall, toolCallCancellation).
  }
}

// ...

Dart 


// ...

final _liveModel = FirebaseAI.googleAI().liveGenerativeModel(
  model: 'GEMINI_LIVE_API_MODEL_NAME',
  // Configure the model to return transcriptions of the audio input and output.
  liveGenerationConfig: LiveGenerationConfig(
    responseModalities: [ResponseModalities.audio],
    inputAudioTranscription: AudioTranscriptionConfig(),
    outputAudioTranscription: AudioTranscriptionConfig(),
  ),
);

final LiveSession _session = await _liveModel.connect();

await for (final response in _session.receive()) {
  if (response.message case LiveServerContent message) {
    if (message.inputTranscription?.text case final inputText?) {
      // Handle transcription text of the audio input.
      print('Input: $inputText');
    }

    if (message.outputTranscription?.text case final outputText?) {
      // Handle transcription text of the audio output.
      print('Output: $outputText');
    }
  }
}

// ...

Unity 


// ...

var liveModel = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI()).GetLiveModel(
    modelName: "GEMINI_LIVE_API_MODEL_NAME",
    // Configure the model to return transcriptions of the audio input and output
    liveGenerationConfig: new LiveGenerationConfig(
        responseModalities: new[] { ResponseModality.Audio },
        inputAudioTranscription: new AudioTranscriptionConfig(),
        outputAudioTranscription: new AudioTranscriptionConfig()
    )
);

try
{
    var session = await liveModel.ConnectAsync();
    var stream = session.ReceiveAsync();
    await foreach (var response in stream) {
        if (response.Message is LiveSessionContent sessionContent) {
            if (!string.IsNullOrEmpty(sessionContent.InputTranscription?.Text)) {
              // handle transcription text of input audio
            }

            if (!string.IsNullOrEmpty(sessionContent.OutputTranscription?.Text)) {
              // handle transcription text of output audio
            }
        }
    }
}
catch (Exception e)
{
    // Handle error
}

// ...


Voice activity detection (VAD)

Voice activity detection (VAD) lets the model recognize when a person is speaking. This is essential for creating natural conversations, as it lets you interrupt the model at any time. VAD is enabled by default.

You can customize the behavior of VAD, including handling interruptions. You can alternatively disable automatic VAD to take complete manual control over conversational turns (such as for a push-to-talk interface).

Configure voice activity detection (VAD)

Click your Gemini API provider to view provider-specific content and code on this page.

The following example shows how you can customize the behavior of VAD by setting realtimeInputConfig as part of the model configuration. Learn about available settings for this configuration in the table later in this section.

Swift 


// ...

let liveModel = FirebaseAI.firebaseAI(backend: .googleAI()).liveModel(
  modelName: "GEMINI_LIVE_API_MODEL_NAME",
  generationConfig: LiveGenerationConfig(
    responseModalities: [.audio],
    // Configure voice activity detection (VAD), interruption handling, and turn coverage.
    // IMPORTANT: Example values shown here. Make sure to update for your use case.
    realtimeInputConfig: RealtimeInputConfig(
      automaticActivityDetection: ActivityDetectionConfig(
        startSensitivity: .low,
        endSensitivity: .low,
        prefixPadding: .milliseconds(20),
        silenceDuration: .milliseconds(500)
      ),
      activityHandling: .interrupt,
      turnCoverage: .onlyActivity
    )
  )
)

Kotlin 


// ...

val liveModel = Firebase.ai(backend = GenerativeBackend.googleAI()).liveModel(
    modelName = "GEMINI_LIVE_API_MODEL_NAME",
    generationConfig = liveGenerationConfig {
        responseModality = ResponseModality.AUDIO
        // Configure voice activity detection (VAD), interruption handling, and turn coverage.
        // IMPORTANT: Example values shown here. Make sure to update for your use case.
        realtimeInputConfig = realtimeInputConfig {
            automaticActivityDetection = ActivityDetectionConfig.builder()
                .setStartSensitivity(ActivityDetectionConfig.Sensitivity.LOW)
                .setEndSensitivity(ActivityDetectionConfig.Sensitivity.LOW)
                .setPrefixPaddingMs(20)
                .setSilenceDurationMs(500)
                .build()
            activityHandling = RealtimeInputConfig.ActivityHandling.INTERRUPT
            turnCoverage = RealtimeInputConfig.TurnCoverage.ONLY_ACTIVITY
        }
    }
)

Java 


// ...

LiveGenerativeModel lm = FirebaseAI.getInstance(GenerativeBackend.googleAI()).liveModel(
        "GEMINI_LIVE_API_MODEL_NAME",
        new LiveGenerationConfig.Builder()
                .setResponseModality(ResponseModality.AUDIO)
                // Configure voice activity detection (VAD), interruption handling, and turn coverage.
                // IMPORTANT: Example values shown here. Make sure to update for your use case.
                .setRealtimeInputConfig(RealtimeInputConfig.builder()
                        .setAutomaticActivityDetection(ActivityDetectionConfig.builder()
                                .setStartSensitivity(ActivityDetectionConfig.Sensitivity.LOW)
                                .setEndSensitivity(ActivityDetectionConfig.Sensitivity.LOW)
                                .setPrefixPaddingMs(20)
                                .setSilenceDurationMs(500)
                                .build())
                        .setActivityHandling(RealtimeInputConfig.ActivityHandling.INTERRUPT)
                        .setTurnCoverage(RealtimeInputConfig.TurnCoverage.ONLY_ACTIVITY)
                        .build())
                .build()
);

Web 

Not yet supported for Web apps - check back soon!

Dart 


// ...

final _liveModel = FirebaseAI.googleAI().liveGenerativeModel(
  model: 'GEMINI_LIVE_API_MODEL_NAME',
  liveGenerationConfig: LiveGenerationConfig(
    responseModalities: [ResponseModalities.audio],
    // Configure voice activity detection (VAD), interruption handling, and turn coverage.
    // IMPORTANT: Example values shown here. Make sure to update for your use case.
    realtimeInputConfig: RealtimeInputConfig(
      automaticActivityDetection: ActivityDetectionConfig(
        startSensitivity: Sensitivity.low,
        endSensitivity: Sensitivity.low,
        prefixPaddingMS: 20,
        silenceDurationMS: 500,
      ),
      activityHandling: ActivityHandling.interrupt,
      turnCoverage: TurnCoverage.onlyActivity,
    ),
  ),
);

Unity 


// ...

var liveModel = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI()).GetLiveModel(
    modelName: "GEMINI_LIVE_API_MODEL_NAME",
    liveGenerationConfig: new LiveGenerationConfig(
        responseModalities: new[] { ResponseModality.Audio },
        // Configure voice activity detection (VAD), interruption handling, and turn coverage.
        // IMPORTANT: Example values shown here. Make sure to update for your use case.
        realtimeInputConfig: new RealtimeInputConfig(
            automaticActivityDetection: new ActivityDetectionConfig(
                startSensitivity: ActivityDetectionConfig.Sensitivity.Low,
                endSensitivity: ActivityDetectionConfig.Sensitivity.Low,
                prefixPaddingMS: 20,
                silenceDurationMS: 500
            ),
            activityHandling: RealtimeInputConfig.ActivityHandling.Interrupt,
            turnCoverage: RealtimeInputConfig.TurnCoverage.OnlyActivity
        )
    )
);

Available VAD configuration settings

Setting Supported values Description
Automatic activity detection (automaticActivityDetection)
startSensitivity
  • LOW (default)
  • HIGH

Determines how readily the server detects when a user begins speaking:

  • LOW: Less sensitive to faint sounds. This helps to prevent ambient noise from inadvertently starting a speech turn.
  • HIGH: More sensitive. This helps to quickly detect quieter speech or soft-spoken words.
endSensitivity
  • LOW (default)
  • HIGH

Determines how readily the server detects when a user stops speaking:

  • LOW: Less sensitive to trailing pauses. The server accommodates natural hesitations or quieter words at the end of a sentence.
  • HIGH: More sensitive. The server concludes the speech turn promptly once voice levels drop.
prefixPadding Duration in milliseconds
(default: None (0 ms))

Defines the amount of audio to include before speech is detected.

This "look-back" buffer ensures the model captures the full onset of speech, including the first syllable which often starts before the detector triggers.

  • The default value of 0 can cause the beginning of words to be clipped.
  • Recommended: 20-50 milliseconds
silenceDuration Duration in milliseconds
(default: ~800 ms)

Defines how long the server waits through silence before ending a speech turn. This determines how tolerant the system is of natural mid-sentence pauses (such as thinking, breathing, or clause boundaries).

  • Recommended (500-800 ms): Provides a good balance—the model receives complete, contextually rich audio while keeping response latency reasonable.
  • Too low (below 200 ms): Ends speech turns during natural pauses, splitting utterances into small fragments that can degrade transcription and response quality.
  • Too high (above 2000 ms): Adds noticeable delay before the model begins responding.
Interruption behavior (activityHandling)
activityHandling
  • INTERRUPT (default)
  • NO_INTERRUPT

Defines how the server handles incoming user activity while generating a response:

  • INTERRUPT: Immediately cancels the model's current response when user activity is detected.
  • NO_INTERRUPT: Continues generating the response without interruption.
Turn context coverage (turnCoverage)
turnCoverage
  • ONLY_ACTIVITY (default)
  • ALL_INPUT
  • AUDIO_ACTIVITY_AND_ALL_VIDEO

Defines what streamed input is included in the conversational turn context:

  • ONLY_ACTIVITY: Excludes silence, retaining only segments where user activity was detected.
  • ALL_INPUT: Retains all streamed input, including audio silence.
  • AUDIO_ACTIVITY_AND_ALL_VIDEO: Retains all streamed video frames, but only audio segments where activity was detected.

Handle interruptions

If the server detects incoming user speech while the model is generating a response and activityHandling is set to INTERRUPT (the default), be aware of the following behaviors:

  • Generation is canceled: The model immediately halts its response and discards any pending tool calls.

  • Session history is truncated: Only the content already transmitted to your app before the interruption is preserved in the session context window.

  • An interruption signal is emitted: The server sends a response message indicating that generation was interrupted (such as wasInterrupted in Swift or interrupted in Kotlin, Java, Web, Flutter, and Unity).

Disable automatic voice activity detection (VAD)

To take complete manual control over conversational turns (such as for a push-to-talk interface), you can disable automatic voice activity detection (VAD) instead of configuring sensitivity parameters.

When automatic detection is disabled, the server won't automatically detect speech or trigger responses based on audio silence. Instead, your app must manually signal user activity boundaries during the active session.

1. Disable automatic VAD in the model configuration

Click your Gemini API provider to view provider-specific content and code on this page.

Swift 


// ...

let liveModel = FirebaseAI.firebaseAI(backend: .googleAI()).liveModel(
  modelName: "GEMINI_LIVE_API_MODEL_NAME",
  generationConfig: LiveGenerationConfig(
    responseModalities: [.audio],
    // Disable automatic voice activity detection (VAD).
    realtimeInputConfig: RealtimeInputConfig(
      automaticActivityDetection: .disabled()
    )
  )
)

Kotlin 


// ...

val liveModel = Firebase.ai(backend = GenerativeBackend.googleAI()).liveModel(
    modelName = "GEMINI_LIVE_API_MODEL_NAME",
    generationConfig = liveGenerationConfig {
        responseModality = ResponseModality.AUDIO
        // Disable automatic voice activity detection (VAD).
        realtimeInputConfig = realtimeInputConfig {
            automaticActivityDetection = ActivityDetectionConfig.disabled()
        }
    }
)

Java 


// ...

LiveGenerativeModel lm = FirebaseAI.getInstance(GenerativeBackend.googleAI()).liveModel(
        "GEMINI_LIVE_API_MODEL_NAME",
        new LiveGenerationConfig.Builder()
                .setResponseModality(ResponseModality.AUDIO)
                // Disable automatic voice activity detection (VAD).
                .setRealtimeInputConfig(RealtimeInputConfig.builder()
                        .setAutomaticActivityDetection(ActivityDetectionConfig.disabled())
                        .build())
                .build()
);

Web 

Not yet supported for Web apps - check back soon!

Dart 


// ...

final _liveModel = FirebaseAI.googleAI().liveGenerativeModel(
  model: 'GEMINI_LIVE_API_MODEL_NAME',
  liveGenerationConfig: LiveGenerationConfig(
    responseModalities: [ResponseModalities.audio],
    // Disable automatic voice activity detection (VAD).
    realtimeInputConfig: RealtimeInputConfig(
      automaticActivityDetection: ActivityDetectionConfig.disabled(),
    ),
  ),
);

Unity 


// ...

var liveModel = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI()).GetLiveModel(
    modelName: "GEMINI_LIVE_API_MODEL_NAME",
    liveGenerationConfig: new LiveGenerationConfig(
        responseModalities: new[] { ResponseModality.Audio },
        // Disable automatic voice activity detection (VAD).
        realtimeInputConfig: new RealtimeInputConfig(
            automaticActivityDetection: ActivityDetectionConfig.Disabled()
        )
    )
);

2. Manually signal user activity boundaries

If you disable VAD, your app must manually demarcate the boundaries of each user activity turn:

  1. Signal activity start: Call sendStartActivityRealtime before sending audio data (for example, when a user presses a push-to-talk button).

  2. Stream user input: Continuously stream audio (or video) data using sendAudioRealtime.

  3. Signal activity end: Call sendStopActivityRealtime when the user finishes speaking (for example, when the user releases the push-to-talk button). This signals the end of the user's turn and triggers the model's response.

Swift 

// If you disable automatic voice activity detection (VAD), you need to manually demarcate user activity.

// Signal the start of user activity (for example, on push-to-talk button press).
await session.sendStartActivityRealtime()

// Stream user audio data while the user speaks.
await session.sendAudioRealtime(audioData)

// Signal the end of user activity (for example, on button release) to trigger model response.
await session.sendStopActivityRealtime()

Kotlin 

// If you disable automatic voice activity detection (VAD), you need to manually demarcate user activity.

// Signal the start of user activity (for example, on push-to-talk button press).
session.sendStartActivityRealtime()

// Stream user audio data while the user speaks.
session.sendAudioRealtime(audioData)

// Signal the end of user activity (for example, on button release) to trigger model response.
session.sendStopActivityRealtime()

Java 

// If you disable automatic voice activity detection (VAD), you need to manually demarcate user activity.

// Signal the start of user activity (for example, on push-to-talk button press).
session.sendStartActivityRealtime();

// Stream user audio data while the user speaks.
session.sendAudioRealtime(audioData);

// Signal the end of user activity (for example, on button release) to trigger model response.
session.sendStopActivityRealtime();

Web 

Not yet supported for Web apps - check back soon!

Dart 

// If you disable automatic voice activity detection (VAD), you need to manually demarcate user activity.

// Signal the start of user activity (for example, on push-to-talk button press).
await session.sendStartActivityRealtime();

// Stream user audio data while the user speaks.
await session.sendAudioRealtime(audioData);

// Signal the end of user activity (for example, on button release) to trigger model response.
await session.sendStopActivityRealtime();

Unity 

// If you disable automatic voice activity detection (VAD), you need to manually demarcate user activity.

// Signal the start of user activity (for example, on push-to-talk button press).
await session.SendStartActivityRealtimeAsync();

// Stream user audio data while the user speaks.
await session.SendAudioRealtimeAsync(audio: audioData);

// Signal the end of user activity (for example, on button release) to trigger model response.
await session.SendStopActivityRealtimeAsync();


Session management

Learn about the following sessions-related topics: