您可以要求 Gemini TTS 模型根據文字提示詞生成語音 (音訊) 輸出內容。使用 Firebase AI Logic 時,您可以直接從應用程式提出這項要求。
文字轉語音 (TTS) 生成功能可控,也就是說,您要提供確切文字,才能合成語音。您也可以在提示中使用自然語言,引導音訊輸出的風格、口音、語速和語氣。您可以將 TTS 視為語音轉文字的反向操作。
這項功能可搭配任何 Gemini -tts 模型使用,這些模型經過最佳化,可生成高品質、低延遲的語音。
這項功能可讓您執行下列操作:
互動式故事:打造沉浸式有聲書或角色扮演遊戲,讓模型根據不同角色切換聲音,或調整語氣 (例如在懸疑情節中低語,或在笑話中大笑),配合敘事內容。
語言學習:建立發音指南,以特定地區口音或較慢的速度朗讀文字,協助學習者練習困難的發音。
情境感知內容朗讀器:使用符合內容的語音角色和情緒語氣,朗讀新聞報導、食譜或網誌文章 (例如以嚴肅語氣朗讀即時新聞,或以溫暖、耐心的語氣朗讀逐步烹飪指示)。
本指南說明如何根據文字輸入內容生成單一或多位說話者的語音,以及如何串流音訊回覆。
跳至單一說話者程式碼 跳至多位說話者程式碼 跳至串流回覆程式碼
TTS 與Live API的比較
文字轉語音 (TTS) 模型和 Live API 模型都是低延遲的語音生成模型,可設定不同的回覆語音和語言。不過,兩者的用途大不相同。
文字轉語音 (TTS) 生成是單向要求/回應互動 (輸入文字,輸出音訊)。這項功能適用於需要精確朗讀所提供文字,並精細控制風格和聲音的情境,例如 Podcast 旁白、有聲書或朗讀文章。
Live API 生成支援雙向串流,可進行即時語音對話 (語音輸入、語音輸出)。這項功能適合用於動態對話情境,由模型決定要傳回的語音。請注意,最新的 Live API 模型也支援影片和圖片輸入。
事前準備
|
按一下您的Gemini API供應商,即可在這個頁面查看供應商專屬內容和程式碼。 |
如果尚未完成入門指南,請先完成。該指南說明如何設定 Firebase 專案、將應用程式連結至 Firebase、新增 SDK、為所選 Gemini API 供應商初始化後端服務,以及建立 GenerativeModel 執行個體。
如要測試及反覆調整提示,建議使用 Google AI Studio。
支援這項功能的模型
gemini-3.1-flash-tts-preview
根據文字生成語音
您可以使用 Gemini TTS 模型,根據提供的文字生成語音。
生成單一說話者的語音
|
嘗試這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也需要點選所選Gemini API供應商的按鈕,這樣這個頁面才會顯示供應商專屬內容。 |
您可以設定模型,使用單一聲音輸出音訊。
在 GenerationConfig 中加入下列內容:
將
responseModalities設為包含AUDIO。使用下列項目設定
SpeechConfig:(必填) 回覆語音名稱 (例如
Kore)(選用) 語言代碼。
如未指定語言,Gemini TTS 模型會自動偵測提示中的語言。
使用文字提示詞呼叫 generateContent。模型會在回應部分傳回原始 PCM 音訊資料。
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: SpeechConfig(voiceName: "Kore", languageCode: "en-US")
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt.
let prompt = "Say cheerfully: Have a wonderful day!"
// Call `generateContent` to generate the speech output based on your text prompt.
let response = try await model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
for part in response.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
print("Received audio data with MIME type: \(mimeType)")
// To play back raw PCM audio bytes, you'll need to write your own `playRawPcm` function.
playRawPcm(data: data)
}
Kotlin
在 Kotlin 中,這個 SDK 的方法是暫停函式,必須從協同程式範圍呼叫。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = SpeechConfig(
voice = Voice("Kore"),
languageCode = "en-US"
)
}
// Initialize the Gemini Developer API backend service.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt.
val prompt = "Say cheerfully: Have a wonderful day!"
// Call `generateContent` to generate the speech output based on your text prompt.
val response = model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
val part = response.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmData = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData)
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 類型。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(new SpeechConfig(new Voice("Kore"), "en-US"))
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
// Use the GenerativeModelFutures Java compatibility layer.
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt.
String prompt = "Say cheerfully: Have a wonderful day!";
Content content = new Content.Builder().addText(prompt).build();
Executor executor = Executors.newSingleThreadExecutor();
// Call `generateContent` to generate the speech output based on your text prompt.
// Extract the audio data and handle it for downstream use.
ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
@Override
public void onSuccess(GenerateContentResponse result) {
Part part = result.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmData = ((InlineDataPart) part).getInlineData();
String mimeType = ((InlineDataPart) part).getMimeType();
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
@Override
public void onFailure(Throwable t) {
t.printStackTrace();
}
}, executor);
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } },
languageCode: "en-US"
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt.
const prompt = "Say cheerfully: Have a wonderful day!";
// Call `generateContent` to generate the speech output based on your text prompt.
const result = await model.generateContent(prompt);
const inlineDataParts = result.response.inlineDataParts();
// Extract the audio data and handle it for downstream use. For example:
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data;
// Decode base64 to ArrayBuffer
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// To play back a PCM buffer, you'll need to write your own `playAudio` function.
playAudio(pcmBuffer);
}
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
final config = GenerationConfig(
responseModalities: [ResponseModalities.audio],
speechConfig: SpeechConfig(voiceName: 'Kore', languageCode: 'en-US'),
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
generationConfig: config,
);
// Provide a text prompt.
final prompt = 'Say cheerfully: Have a wonderful day!';
// Call `generateContent` to generate the speech output based on your text prompt.
final response = await model.generateContent([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
final part = response.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmData = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
await playAudio(pcmData);
}
Unity
using Firebase.AI;
// Set `responseModalities` to include `Audio`.
// Configure a `SpeechConfig` with your chosen voice name and language code.
var config = new GenerationConfig(
responseModalities: new System.Collections.Generic.List<ResponseModality> { ResponseModality.Audio },
speechConfig: SpeechConfig.UsePrebuiltVoice("Kore", "en-US")
);
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt.
var prompt = "Say cheerfully: Have a wonderful day!";
// Call `GenerateContentAsync` to generate the speech output based on your text prompt.
var response = await model.GenerateContentAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
if (response.Candidates.Count > 0) {
foreach (var part in response.Candidates[0].Content.Parts) {
if (part is ModelContent.InlineDataPart inlineData) {
byte[] pcmData = inlineData.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
}
生成多人語音
|
嘗試這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也需要點選所選Gemini API供應商的按鈕,這樣這個頁面才會顯示供應商專屬內容。 |
您可以設定模型,為文字中的不同說話者使用不同語音。這項功能適合用於生成對話或交談的音訊。
建立
MultiSpeakerVoiceConfig,將說話者名稱 (用於提示) 對應至特定回覆語音名稱 (例如Kore)。多位說話者設定僅支援 2 位說話者。
在
GenerationConfig中加入下列內容:將
responseModalities設為包含AUDIO。使用下列項目設定
SpeechConfig:(必要) 傳入
MultiSpeakerVoiceConfig。(選用) 語言代碼。
如未指定語言,Gemini TTS 模型會自動偵測提示中的語言。
在提示中,使用講者名稱做為前置字元,指出發言者 (例如
Joe: Hello. Jane: Hi.)。
使用文字提示詞呼叫 generateContent。模型會在回應部分傳回原始 PCM 音訊資料。
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
let multiSpeechConfig = SpeechConfig(
multiSpeakerVoiceConfig: MultiSpeakerVoiceConfig(
speakerVoiceConfigs: [
SpeakerVoiceConfig(speaker: "Joe", voiceName: "Puck"),
SpeakerVoiceConfig(speaker: "Jane", voiceName: "Kore")
]
),
languageCode: "en-US"
)
// Set `responseModalities` to include `audio`.
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: multiSpeechConfig
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt that includes the names of the speakers.
let prompt = """
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
"""
// Call `generateContent` to generate the speech output based on your text prompt.
let response = try await model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
for part in response.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
print("Received audio data with MIME type: \(mimeType)")
// To play back raw PCM audio bytes, you'll need to write your own `playRawPcm` function.
playRawPcm(data: data)
}
Kotlin
在 Kotlin 中,這個 SDK 的方法是暫停函式,必須從協同程式範圍呼叫。
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
val multiSpeechConfig = SpeechConfig(
multiSpeakerVoiceConfig = MultiSpeakerVoiceConfig(
speakerVoiceConfigs = listOf(
SpeakerVoiceConfig(speaker = "Joe", voice = Voice("Puck")),
SpeakerVoiceConfig(speaker = "Jane", voice = Voice("Kore"))
)
),
languageCode = "en-US"
)
// Set `responseModalities` to include `AUDIO`.
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = multiSpeechConfig
}
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt that includes the names of the speakers.
val prompt = """
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
"""
// Call `generateContent` to generate the speech output based on your text prompt.
val response = model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
val part = response.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmData = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData)
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 類型。
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
MultiSpeakerVoiceConfig multiSpeakerVoiceConfig = new MultiSpeakerVoiceConfig(
Arrays.asList(
new SpeakerVoiceConfig("Joe", new Voice("Puck")),
new SpeakerVoiceConfig("Jane", new Voice("Kore"))
)
);
SpeechConfig multiSpeechConfig = new SpeechConfig(multiSpeakerVoiceConfig);
// Set `responseModalities` to include `AUDIO`.
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(multiSpeechConfig)
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt that includes the names of the speakers.
String prompt = "Joe: How's it going today Jane?\nJane: Not too bad, how about you?";
Content content = new Content.Builder().addText(prompt).build();
Executor executor = Executors.newSingleThreadExecutor();
// Call `generateContent` to generate the speech output based on your text prompt.
// Extract the audio data and handle it for downstream use.
ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
@Override
public void onSuccess(GenerateContentResponse result) {
Part part = result.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmData = ((InlineDataPart) part).getInlineData(); // Raw PCM bytes (24kHz, 1 channel, 16-bit)
String mimeType = ((InlineDataPart) part).getMimeType(); // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
@Override
public void onFailure(Throwable t) {
t.printStackTrace();
}
}, executor);
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
multiSpeakerVoiceConfig: {
speakerVoiceConfigs: [
{ speaker: "Joe", voiceConfig: { prebuiltVoiceConfig: { voiceName: "Puck" } } },
{ speaker: "Jane", voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } } }
]
},
languageCode: "en-US"
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt that includes the names of the speakers.
const prompt = `
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
`;
// Call `generateContent` to generate the speech output based on your text prompt.
const result = await model.generateContent(prompt);
const inlineDataParts = result.response.inlineDataParts();
// Extract the audio data and handle it for downstream use. For example:
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// To play back a PCM buffer, you'll need to write your own `playAudio` function.
playAudio(pcmBuffer);
}
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
final multiSpeechConfig = SpeechConfig.multiSpeaker(
multiSpeakerVoiceConfig: MultiSpeakerVoiceConfig(
speakerVoiceConfigs: [
SpeakerVoiceConfig(speaker: 'Joe', voiceName: 'Puck'),
SpeakerVoiceConfig(speaker: 'Jane', voiceName: 'Kore'),
],
),
languageCode: 'en-US',
);
// Set `responseModalities` to include `audio`.
final config = GenerationConfig(
responseModalities: [ResponseModalities.audio],
speechConfig: multiSpeechConfig,
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
generationConfig: config,
);
// Provide a text prompt that includes the names of the speakers.
final prompt = '''
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
''';
// Call `generateContent` to generate the speech output based on your text prompt.
final response = await model.generateContent([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
final part = response.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmData = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
await playAudio(pcmData);
}
Unity
using Firebase.AI;
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
var multiSpeakerVoiceConfig = new MultiSpeakerVoiceConfig(
new System.Collections.Generic.List<SpeakerVoiceConfig> {
SpeakerVoiceConfig.UsePrebuiltVoice("Joe", "Puck"),
SpeakerVoiceConfig.UsePrebuiltVoice("Jane", "Kore")
}
);
var multiSpeechConfig = SpeechConfig.UseMultiSpeakerVoice(multiSpeakerVoiceConfig);
// Set `responseModalities` to include `Audio`.
var config = new GenerationConfig(
responseModalities: new System.Collections.Generic.List<ResponseModality> { ResponseModality.Audio },
speechConfig: multiSpeechConfig
);
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt that includes the names of the speakers.
var prompt = "Joe: How's it going today Jane?\nJane: Not too bad, how about you?";
// Call `GenerateContentAsync` to generate the speech output based on your text prompt.
var response = await model.GenerateContentAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
if (response.Candidates.Count > 0) {
foreach (var part in response.Candidates[0].Content.Parts) {
if (part is ModelContent.InlineDataPart inlineData) {
byte[] pcmData = inlineData.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
}
逐句回覆
|
嘗試這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也需要點選所選Gemini API供應商的按鈕,這樣這個頁面才會顯示供應商專屬內容。 |
您可以串流生成中的音訊回覆,不必等待整個音訊檔案完成,藉此加快互動速度並降低延遲。
單一說話者和多位說話者設定都支援串流生成語音。只有在使用 Gemini 3.x TTS 模型時才支援。
如要串流語音回覆,請呼叫 generateContentStream 而非 generateContent,並在區塊送達時處理。以下範例說明如何串流單一說話者的回應:
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Set `responseModalities` to include `audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: SpeechConfig(voiceName: "Kore")
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt.
let prompt = "Tell me a story about a brave knight."
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
let responseStream = try model.generateContentStream(prompt)
// Extract the audio data and handle it for downstream use. For example:
for try await chunk in responseStream {
for part in chunk.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(data)
}
}
Kotlin
在 Kotlin 中,這個 SDK 的方法是暫停函式,必須從協同程式範圍呼叫。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = SpeechConfig(voice = Voice("Kore"))
}
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt.
val prompt = "Tell me a story about a brave knight."
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
// Extract the audio data and handle it for downstream use.
model.generateContentStream(prompt).collect { chunk ->
val part = chunk.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmChunk = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk)
}
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 類型。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(new SpeechConfig(new Voice("Kore")))
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt.
String prompt = "Tell me a story about a brave knight.";
Content content = new Content.Builder().addText(prompt).build();
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
Publisher<GenerateContentResponse> streamingResponse =
model.generateContentStream(content);
// Extract the audio data and handle it for downstream use.
streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
@Override
public void onSubscribe(Subscription s) {
s.request(Long.MAX_VALUE);
}
@Override
public void onNext(GenerateContentResponse chunk) {
Part part = chunk.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmChunk = ((InlineDataPart) part).getInlineData(); // Raw PCM bytes (24kHz, 1 channel, 16-bit)
String mimeType = ((InlineDataPart) part).getMimeType(); // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
@Override
public void onComplete() {
// Audio stream complete.
}
@Override
public void onError(Throwable t) {
t.printStackTrace();
}
});
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } }
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt.
const prompt = "Tell me a story about a brave knight.";
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
const result = await model.generateContentStream(prompt);
// Extract the audio data and handle it for downstream use. For example:
const playbackQueue = [];
for await (const chunk of result.stream) {
const inlineDataParts = chunk.inlineDataParts();
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// Append the audio chunk to your audio queue/buffer for playback.
playbackQueue.push(pcmBuffer);
}
}
// To play back an array of PCM buffers in sequence, you'll need to write your own `processPlaybackQueue` function.
processPlaybackQueue(playbackQueue);
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Set `responseModalities` to include `audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
final config = GenerationConfig(
responseModalities: [ResponseModalities.audio],
speechConfig: SpeechConfig(voiceName: 'Kore'),
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
generationConfig: config,
);
// Provide a text prompt.
final prompt = 'Tell me a story about a brave knight.';
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
final responseStream = model.generateContentStream([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
await for (final chunk in responseStream) {
final part = chunk.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmChunk = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
Unity
using System.Collections.Generic;
using System.Linq;
using Firebase.AI;
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Set `ResponseModalities` to include `Audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
var config = new GenerationConfig(
responseModalities: new List<ResponseModality> { ResponseModality.Audio },
speechConfig: SpeechConfig.UsePrebuiltVoice("Kore")
);
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt.
var prompt = "Tell me a story about a brave knight.";
// Call `GenerateContentStreamAsync` to generate the speech output stream based on your text prompt.
var responseStream = model.GenerateContentStreamAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
await foreach (var response in responseStream)
{
var audioParts = response.Candidates.FirstOrDefault().Content.Parts
.OfType<ModelContent.InlineDataPart>();
foreach (var part in audioParts)
{
byte[] pcmChunk = part.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
使用提示控制語音輸出
您可以使用特定的提示技術,調整生成語音的語氣、語速和風格。
以下有關提示結構和音訊標記的子節,將提供高階指引。如需詳細指引,請參閱這份提示詞指南。
提示結構
為獲得最佳結果,請使用下列元件建構提示:
Audio Profile:描述說話者的角色、核心身分和原型 (例如A warm, professional narrator)。Scene:描述環境和情緒氛圍 (例如In a quiet library或Amidst a noisy crowd)。Director's Notes:描述情緒、語速、風格和口音 (例如Speak slowly and with mystery)。Sample Context:為模型提供起點 (例如The speaker is greeting a close friend)。Transcript:要說出的實際文字。為獲得最佳效果,請確保文字的語氣和情境與語音設定檔和導演附註一致。
範例提示詞:
[Audio Profile: A young, energetic voice]
[Scene: A lively sports broadcast]
[Director's Notes: Speak fast, with high energy and excitement]
[Sample Context: The game just ended with a last-second touchdown]
Welcome back fans! What an incredible game we're witnessing today!
音訊標記
您可以直接在文字提示詞中插入格式標記,引導模型生成內容。
只有使用 Gemini 3.x TTS 模型時,才支援音訊標記。
常用標記包括:
[whispers]:輕聲說話[laughs]:增添歡樂[giggles]:增添歡樂[sighs]:加入嘆氣聲[gasp]:新增喘氣聲[shouting]:大聲說話[excited]:以興奮的語氣說話[serious]:認真說話[sighs whispers]:混合情緒 (可合併標記)
使用音訊代碼時,請注意下列事項:
沒有詳盡清單:支援的標記沒有固定清單。您可以嘗試不同的情緒和表情 (例如
[bored]、[sarcastically],甚至是[like dracula]),看看輸出內容會如何變化。非英文文字提示詞:即使文字提示詞不是英文,仍應使用英文音訊標記,才能獲得最佳結果。
範例提示詞:
I have a secret to tell you. [whispers] I found the hidden treasure. [laughs] I can't believe it!
限制與需求
使用語音生成功能時,請注意下列限制和規定:
多位說話者設定僅支援 2 位說話者。
下列功能僅支援 Gemini 3.x TTS 模型:串流、音訊標記和其他自動偵測語言。
gemini-3.1-flash-tts-preview 的限制
- 語音不一致:如果提示的語氣和情境與說話者設定檔不符 (例如深沉的男聲試圖模仿年輕女孩說話),模型的輸出內容可能不一定會嚴格符合所選說話者。請確認提示內容與語音相符。
- 輸出內容較長:如果音訊長度超過幾分鐘,語音品質和一致性可能會不穩定。建議將長文字提示詞拆成較小的部分。
- 偶爾傳回文字權杖:模型有時會傳回文字權杖,而非音訊權杖,導致要求失敗並顯示
500錯誤。由於這種情況會隨機發生在少數要求中,因此您應在應用程式中實作重試邏輯。 - 分類器誤拒:如果提示詞不夠明確,語音合成分類器可能會判定失敗,導致要求遭拒 (
PROHIBITED_CONTENT),或模型會朗讀樣式指示。為避免這種情況,請使用結構化提示,並在提示開頭加上明確的前導提示 (例如Audio Profile和Director's Notes)。
支援的語音和語言
Gemini TTS 模型會接收文字輸入並生成音訊輸出,因此回覆就是合成語音本身。下列子節列出 Gemini TTS 模型支援的語音和語言,可用於「說話」(或回覆)。
這些語音支援多種語言,因此您可以使用相同的語音,以任何支援的語言生成語音。舉例來說,您可以將語音設為 Kore,並以西班牙文、北印度文和越南文傳送一組文字提示詞。回覆都會以 Kore 語音呈現,但會使用不同的語言。
語音名稱
Gemini TTS 模型支援 30 種不同的合成 HD 語音,各有不同特色。展開下方區段,即可查看回覆語音選項清單,並聆聽各語音的示範。
語言
Gemini TTS 模型可「自動」偵測文字提示詞中的下列語言。生成的語音會使用該語言。
請注意,您也可以在語音設定中明確設定語言代碼。
所有音訊生成模型支援的語言
| 語言 | BCP-47 代碼 | 語言 | BCP-47 代碼 |
|---|---|---|---|
| 阿拉伯文 (埃及) | ar-EG | 德文 (德國) | de-DE |
| 英文 (美國) | en-US | 西班牙文 (美國) | es-US |
| 法文 (法國) | fr-FR | 北印度文 (印度) | hi-IN |
| 印尼文 (印尼) | id-ID | 義大利文 (義大利) | it-IT |
| 日文 (日本) | ja-JP | 韓文 (韓國) | ko-KR |
| 葡萄牙文 (巴西) | pt-BR | 俄文 (俄羅斯) | ru-RU |
| 荷蘭文 (荷蘭) | nl-NL | 波蘭文 (波蘭) | pl-PL |
| 泰文 (泰國) | th-TH | 土耳其文 (土耳其) | tr-TR |
| 越南文 (越南) | vi-VN | 羅馬尼亞文 (羅馬尼亞) | ro-RO |
| 烏克蘭文 (烏克蘭) | uk-UA | 孟加拉文 (孟加拉) | bn-BD |
| 英文 (印度) | en-IN 和 hi-IN 套件 | 馬拉地文 (印度) | mr-IN |
| 泰米爾文 (印度) | ta-IN | 泰盧固文 (印度) | te-IN |
音訊生成 3.x 模型支援的其他語言
| 語言 | BCP-47 代碼 | 語言 | BCP-47 代碼 |
|---|---|---|---|
| 南非荷蘭文 | 南非荷蘭文 | 菲律賓文 | fil |
| 阿爾巴尼亞文 | 阿爾巴尼亞文 | 芬蘭文 | 芬蘭文 |
| 阿姆哈拉文 | 阿姆哈拉文 | 加里西亞文 | 加里西亞文 |
| 亞美尼亞文 | 亞美尼亞文 | 喬治亞文 | 喬治亞文 |
| 亞塞拜然文 | 亞塞拜然文 | 希臘文 | 希臘文 |
| 巴斯克文 | 巴斯克文 | 古吉拉特文 | 古吉拉特文 |
| 白俄羅斯語 | be | 海地克里奧爾文 | ht |
| 保加利亞文 | 保加利亞文 | 希伯來文 | 他 |
| 緬甸文 | 緬甸文 | 匈牙利文 | 匈牙利文 |
| 加泰隆尼亞文 | 加泰隆尼亞文 | 冰島文 | 為 |
| 宿霧文 | ceb | 爪哇語 | jv |
| 中文 (華語) | cmn | 卡納達文 | 卡納達文 |
| 克羅埃西亞文 | 小時 | 貢根文 | kok |
| 捷克文 | 捷克文 | 寮文 | lo |
| 丹麥文 | da | 拉丁文 | la |
| 愛沙尼亞文 | 愛沙尼亞文 | 拉脫維亞文 | lv |
| 立陶宛文 | lt | 盧森堡文 | lb |
| 馬其頓文 | 馬其頓文 | 邁蒂利文 | mai |
| 馬達加斯加文 | mg | 馬來文 | ms |
| 馬拉雅拉姆文 | 馬拉雅拉姆文 | 蒙古文 | mn |
| 尼泊爾文 | 尼泊爾文 | 挪威文 (Bokmål) | nb |
| 挪威文 (新挪威文) | nn | 歐利亞文 | 或 |
| 普什圖文 | ps | 波斯文 | 波斯文 |
| 旁遮普文 | 旁遮普文 | 塞爾維亞文 | 塞爾維亞文 |
| 信德文 | sd | 錫蘭文 | 僧伽羅文 (錫蘭文) |
| 斯洛伐克文 | sk | 斯洛維尼亞文 | 斯洛維尼亞文 |
| 史瓦西里文 | sw | 瑞典文 | 瑞典文 |
| 烏都文 | 烏都文 |
(選用) 明確設定語言代碼
如未在語音設定中指定語言代碼,模型會自動偵測文字提示詞中的語言。
不過,您也可以選擇明確設定語言 (使用語音設定中的 languageCode 參數)。如要執行這項操作,請使用下列其中一個支援的 BCP-47 語言代碼:
- 阿拉伯文:
ar-XA - 孟加拉文:
bn-IN - 中文 (國語):
cmn-CN - 荷蘭文:
nl-NL - 英文:
en-US、en-GB、en-AU、en-IN - 法文:
fr-FR、fr-CA - 德文:
de-DE - 古吉拉特文:
gu-IN - 北印度文:
hi-IN - 印尼文:
id-ID - 義大利文:
it-IT - 日文:
ja-JP - Kannada:
kn-IN - 韓文:
ko-KR - 馬拉雅拉姆文:
ml-IN - 馬拉地文:
mr-IN - Polish:
pl-PL - 葡萄牙文:
pt-BR - 俄文:
ru-RU - 西班牙文:
es-US、es-ES - 泰米爾文:
ta-IN - 泰盧固文:
te-IN - 泰文:
th-TH - 土耳其文:
tr-TR - 越南文:
vi-VN
你還可以做些什麼?
- 瞭解如何先計算詞元,再將長提示傳送至模型。
-
開始思考如何為正式版做好準備 (請參閱正式版檢查清單):
- 盡早強制執行 Firebase App Check,協助保護 Gemini API,避免未經授權的用戶端濫用。
- 使用 Firebase Remote Config 或伺服器提示詞範本,即可隨選變更 AI 功能的設定 (例如模型名稱),不必發布新版應用程式。
試用其他功能
- 建立多輪對話 (即時通訊)。
- 根據僅限文字的提示生成文字。
- 根據文字和多模態提示生成結構化輸出內容 (例如 JSON)。
- 生成及編輯圖片:使用文字和多模態提示詞。
- 使用 Gemini Live API 串流輸入和輸出 (包括音訊)。
-
使用工具 (例如函式呼叫和
Google Search 或Google Maps 的建立基準功能),將 Gemini 模型連結至應用程式的其他部分、外部系統和資訊。
瞭解如何控管內容生成
您也可以使用 Google AI Studio 測試提示和模型設定,甚至取得生成的程式碼片段。進一步瞭解支援的模型
瞭解適用於各種用途的模型,以及配額和定價。提供「Firebase AI Logic」使用體驗的意見回饋