您可以要求 Gemini 文字轉語音模型根據文字提示詞生成語音 (音訊) 輸出內容。使用 Firebase AI Logic 時,您可以直接從應用程式提出這項要求。
文字轉語音 (TTS) 生成功能可控,也就是說,您提供要合成語音的確切文字。此外,您也可以在提示中使用自然語言,引導音訊輸出的風格、口音、語速和語調。您可以將 TTS 視為轉錄 (語音轉文字) 的相反功能。
這項功能適用於任何 Gemini -tts 模型,這些模型經過最佳化,可生成高品質的語音,且延遲時間短。
這項功能可協助你執行下列操作:
互動式說故事:製作身歷其境的有聲書或角色扮演遊戲,讓模型為不同角色變換聲音,或根據敘事內容調整語氣 (例如在懸疑情節中低語,或在笑話中大笑)。
語言學習:製作發音指南,以特定區域口音或較慢的速度朗讀文字,協助學習者練習困難的發音。
可辨識情境的內容朗讀器:使用符合內容的語音角色和情緒語氣朗讀新聞報導、食譜或網誌文章 (例如以嚴肅的語氣朗讀即時新聞,或以溫暖、耐心的語氣朗讀烹飪步驟說明)。
本指南說明如何從單一或多位說話者的文字輸入內容生成語音,以及如何串流音訊回應。
跳至單一音箱的程式碼 跳至多個音箱的程式碼 跳至串流回應的程式碼
比較 TTS 和 Live API
文字轉語音 (TTS) 模型和 Live API 模型都是低延遲的語音生成模型,可設定不同的回覆聲音和語言。不過,兩者適用於截然不同的用途。
文字轉語音 (TTS) 生成是單向的請求/回應互動 (輸入文字,輸出音訊)。這個模型專為需要準確朗讀所提供文字,並精細控制風格和聲音的應用情境而設計,例如 Podcast 旁白、有聲書或朗讀文章。
Live API 生成支援雙向串流,可進行即時語音對話 (語音輸入/輸出)。這項模型擅長處理動態對話情境,可決定要傳回的適用語音。請注意,最新模型也支援影片和圖片輸入內容。Live API
事前準備
|
按一下 Gemini API 供應商,即可在這個頁面查看供應商專屬內容和程式碼。 |
如果尚未完成,請參閱入門指南,瞭解如何設定 Firebase 專案、將應用程式連結至 Firebase、新增 SDK、為所選Gemini API供應商初始化後端服務,以及建立 GenerativeModel 執行個體。
如要測試及反覆調整提示,建議使用 Google AI Studio。
支援這項功能的機型
gemini-3.1-flash-tts-preview
根據文字生成語音
您可以使用 Gemini TTS 模型,從提供的文字生成語音。
生成單一說話者的語音
|
試用這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也會點選所選Gemini API供應商的按鈕,以便在本頁面查看供應商專屬內容。 |
您可以將模型設定為使用單一聲音輸出音訊。
在 GenerationConfig 中加入下列內容:
將
responseModalities設為包含AUDIO。使用下列項目設定
SpeechConfig:(必要) 回覆語音名稱 (例如
Kore)(選用) 語言代碼。
如未指定語言,Gemini TTS 模型會自動偵測提示中的語言。
使用文字提示詞呼叫 generateContent。模型會在回覆部分中傳回原始 PCM 音訊資料。
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: SpeechConfig(voiceName: "Kore", languageCode: "en-US")
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt.
let prompt = "Say cheerfully: Have a wonderful day!"
// Call `generateContent` to generate the speech output based on your text prompt.
let response = try await model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
for part in response.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
print("Received audio data with MIME type: \(mimeType)")
// To play back raw PCM audio bytes, you'll need to write your own `playRawPcm` function.
playRawPcm(data: data)
}
Kotlin
如果是 Kotlin,這個 SDK 中的方法是暫停函式,需要從 Coroutine 範圍呼叫。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = SpeechConfig(
voice = Voice("Kore"),
languageCode = "en-US"
)
}
// Initialize the Gemini Developer API backend service.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt.
val prompt = "Say cheerfully: Have a wonderful day!"
// Call `generateContent` to generate the speech output based on your text prompt.
val response = model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
val part = response.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmData = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData)
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 型別。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(new SpeechConfig(new Voice("Kore"), "en-US"))
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
// Use the GenerativeModelFutures Java compatibility layer.
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt.
String prompt = "Say cheerfully: Have a wonderful day!";
Content content = new Content.Builder().addText(prompt).build();
Executor executor = Executors.newSingleThreadExecutor();
// Call `generateContent` to generate the speech output based on your text prompt.
// Extract the audio data and handle it for downstream use.
ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
@Override
public void onSuccess(GenerateContentResponse result) {
Part part = result.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmData = ((InlineDataPart) part).getInlineData();
String mimeType = ((InlineDataPart) part).getMimeType();
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
@Override
public void onFailure(Throwable t) {
t.printStackTrace();
}
}, executor);
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } },
languageCode: "en-US"
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt.
const prompt = "Say cheerfully: Have a wonderful day!";
// Call `generateContent` to generate the speech output based on your text prompt.
const result = await model.generateContent(prompt);
const inlineDataParts = result.response.inlineDataParts();
// Extract the audio data and handle it for downstream use. For example:
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data;
// Decode base64 to ArrayBuffer
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// To play back a PCM buffer, you'll need to write your own `playAudio` function.
playAudio(pcmBuffer);
}
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
final config = GenerationConfig(
responseModalities: [ResponseModality.audio],
speechConfig: SpeechConfig(voiceName: 'Kore', languageCode: 'en-US'),
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
config: config,
);
// Provide a text prompt.
final prompt = 'Say cheerfully: Have a wonderful day!';
// Call `generateContent` to generate the speech output based on your text prompt.
final response = await model.generateContent([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
final part = response.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmData = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
await playAudio(pcmData);
}
Unity
using Firebase.AI;
// Set `responseModalities` to include `Audio`.
// Configure a `SpeechConfig` with your chosen voice name and language code.
var config = new GenerationConfig(
responseModalities: new System.Collections.Generic.List<ResponseModality> { ResponseModality.Audio },
speechConfig: SpeechConfig.UsePrebuiltVoice("Kore", "en-US")
);
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt.
var prompt = "Say cheerfully: Have a wonderful day!";
// Call `GenerateContentAsync` to generate the speech output based on your text prompt.
var response = await model.GenerateContentAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
if (response.Candidates.Count > 0) {
foreach (var part in response.Candidates[0].Content.Parts) {
if (part is ModelContent.InlineDataPart inlineData) {
byte[] pcmData = inlineData.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
}
生成多位說話者的語音
|
試用這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也會點選所選Gemini API供應商的按鈕,以便在本頁面查看供應商專屬內容。 |
您可以設定模型,讓文字中的不同說話者使用不同聲音。這項功能適合用來生成對話或交談的音訊。
建立
MultiSpeakerVoiceConfig,將說話者名稱 (您會在提示中使用) 對應至特定回覆語音名稱 (例如Kore)。多位說話者設定僅支援 2 位說話者。
在
GenerationConfig中加入下列內容:將
responseModalities設為包含AUDIO。使用下列項目設定
SpeechConfig:(必要) 傳遞
MultiSpeakerVoiceConfig。(選用) 語言代碼。
如未指定語言,Gemini TTS 模型會自動偵測提示中的語言。
在提示中,使用說話者名稱做為前置字元,指出說話者是誰 (例如
Joe: Hello. Jane: Hi.)。
使用文字提示詞呼叫 generateContent。模型會在回覆部分中傳回原始 PCM 音訊資料。
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
let multiSpeechConfig = SpeechConfig(
multiSpeakerVoiceConfig: MultiSpeakerVoiceConfig(
speakerVoiceConfigs: [
SpeakerVoiceConfig(speaker: "Joe", voiceName: "Puck"),
SpeakerVoiceConfig(speaker: "Jane", voiceName: "Kore")
]
),
languageCode: "en-US"
)
// Set `responseModalities` to include `audio`.
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: multiSpeechConfig
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt that includes the names of the speakers.
let prompt = """
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
"""
// Call `generateContent` to generate the speech output based on your text prompt.
let response = try await model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
for part in response.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
print("Received audio data with MIME type: \(mimeType)")
// To play back raw PCM audio bytes, you'll need to write your own `playRawPcm` function.
playRawPcm(data: data)
}
Kotlin
如果是 Kotlin,這個 SDK 中的方法是暫停函式,需要從 Coroutine 範圍呼叫。
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
val multiSpeechConfig = SpeechConfig(
multiSpeakerVoiceConfig = MultiSpeakerVoiceConfig(
speakerVoiceConfigs = listOf(
SpeakerVoiceConfig(speaker = "Joe", voice = Voice("Puck")),
SpeakerVoiceConfig(speaker = "Jane", voice = Voice("Kore"))
)
),
languageCode = "en-US"
)
// Set `responseModalities` to include `AUDIO`.
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = multiSpeechConfig
}
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt that includes the names of the speakers.
val prompt = """
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
"""
// Call `generateContent` to generate the speech output based on your text prompt.
val response = model.generateContent(prompt)
// Extract the audio data and handle it for downstream use. For example:
val part = response.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmData = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData)
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 型別。
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
MultiSpeakerVoiceConfig multiSpeakerVoiceConfig = new MultiSpeakerVoiceConfig(
Arrays.asList(
new SpeakerVoiceConfig("Joe", new Voice("Puck")),
new SpeakerVoiceConfig("Jane", new Voice("Kore"))
)
);
SpeechConfig multiSpeechConfig = new SpeechConfig(multiSpeakerVoiceConfig);
// Set `responseModalities` to include `AUDIO`.
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(multiSpeechConfig)
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt that includes the names of the speakers.
String prompt = "Joe: How's it going today Jane?\nJane: Not too bad, how about you?";
Content content = new Content.Builder().addText(prompt).build();
Executor executor = Executors.newSingleThreadExecutor();
// Call `generateContent` to generate the speech output based on your text prompt.
// Extract the audio data and handle it for downstream use.
ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
@Override
public void onSuccess(GenerateContentResponse result) {
Part part = result.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmData = ((InlineDataPart) part).getInlineData(); // Raw PCM bytes (24kHz, 1 channel, 16-bit)
String mimeType = ((InlineDataPart) part).getMimeType(); // for example: "audio/pcm"
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
@Override
public void onFailure(Throwable t) {
t.printStackTrace();
}
}, executor);
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
multiSpeakerVoiceConfig: {
speakerVoiceConfigs: [
{ speaker: "Joe", voiceConfig: { prebuiltVoiceConfig: { voiceName: "Puck" } } },
{ speaker: "Jane", voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } } }
]
},
languageCode: "en-US"
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt that includes the names of the speakers.
const prompt = `
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
`;
// Call `generateContent` to generate the speech output based on your text prompt.
const result = await model.generateContent(prompt);
const inlineDataParts = result.response.inlineDataParts();
// Extract the audio data and handle it for downstream use. For example:
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// To play back a PCM buffer, you'll need to write your own `playAudio` function.
playAudio(pcmBuffer);
}
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
final multiSpeechConfig = SpeechConfig.multiSpeaker(
multiSpeakerVoiceConfig: MultiSpeakerVoiceConfig(
speakerVoiceConfigs: [
SpeakerVoiceConfig(speaker: 'Joe', voiceName: 'Puck'),
SpeakerVoiceConfig(speaker: 'Jane', voiceName: 'Kore'),
],
),
languageCode: 'en-US',
);
// Set `responseModalities` to include `audio`.
final config = GenerationConfig(
responseModalities: [ResponseModality.audio],
speechConfig: multiSpeechConfig,
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
config: config,
);
// Provide a text prompt that includes the names of the speakers.
final prompt = '''
Joe: How's it going today Jane?
Jane: Not too bad, how about you?
''';
// Call `generateContent` to generate the speech output based on your text prompt.
final response = await model.generateContent([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
final part = response.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmData = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
await playAudio(pcmData);
}
Unity
using Firebase.AI;
// Configure a `SpeechConfig` for multiple speakers, assigning a voice to each speaker.
var multiSpeakerVoiceConfig = new MultiSpeakerVoiceConfig(
new System.Collections.Generic.List<SpeakerVoiceConfig> {
SpeakerVoiceConfig.UsePrebuiltVoice("Joe", "Puck"),
SpeakerVoiceConfig.UsePrebuiltVoice("Jane", "Kore")
}
);
var multiSpeechConfig = SpeechConfig.UseMultiSpeakerVoice(multiSpeakerVoiceConfig);
// Set `responseModalities` to include `Audio`.
var config = new GenerationConfig(
responseModalities: new System.Collections.Generic.List<ResponseModality> { ResponseModality.Audio },
speechConfig: multiSpeechConfig
);
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt that includes the names of the speakers.
var prompt = "Joe: How's it going today Jane?\nJane: Not too bad, how about you?";
// Call `GenerateContentAsync` to generate the speech output based on your text prompt.
var response = await model.GenerateContentAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
if (response.Candidates.Count > 0) {
foreach (var part in response.Candidates[0].Content.Parts) {
if (part is ModelContent.InlineDataPart inlineData) {
byte[] pcmData = inlineData.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// To play back PCM audio data, you'll need to write your own `playAudio` function.
playAudio(pcmData);
}
}
}
逐句顯示回覆
|
試用這個範例前,請先完成本指南的「事前準備」一節,設定專案和應用程式。 在該節中,您也會點選所選Gemini API供應商的按鈕,以便在本頁面查看供應商專屬內容。 |
您可以串流音訊回應,不必等待整個音訊檔案完成,藉此加快互動速度並縮短延遲時間。
無論是單一說話者或多位說話者,系統都支援串流生成的語音。只有在使用 Gemini 3.x TTS 模型時才支援。
如要串流語音回覆,請呼叫 generateContentStream 而不是 generateContent,並在接收到區塊時處理。下列範例說明如何串流單一說話者的回應:
Swift
import FirebaseAILogic
// Initialize the Gemini Developer API backend service.
let ai = FirebaseAI.firebaseAI(backend: .googleAI())
// Set `responseModalities` to include `audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
let config = GenerationConfig(
responseModalities: [.audio],
speechConfig: SpeechConfig(voiceName: "Kore")
)
// Create a `GenerativeModel` instance with a model that supports speech generation.
let model = ai.generativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
)
// Provide a text prompt.
let prompt = "Tell me a story about a brave knight."
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
let responseStream = try model.generateContentStream(prompt)
// Extract the audio data and handle it for downstream use. For example:
for try await chunk in responseStream {
for part in chunk.inlineDataParts {
let data = part.data // Raw PCM audio bytes (24kHz, 1 channel, 16-bit)
let mimeType = part.mimeType // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(data)
}
}
Kotlin
如果是 Kotlin,這個 SDK 中的方法是暫停函式,需要從 Coroutine 範圍呼叫。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
val config = generationConfig {
responseModalities = listOf(ResponseModality.AUDIO)
speechConfig = SpeechConfig(voice = Voice("Kore"))
}
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
val model = Firebase.ai(backend = GenerativeBackend.googleAI())
.generativeModel(
modelName = "gemini-3.1-flash-tts-preview",
generationConfig = config
)
// Provide a text prompt.
val prompt = "Tell me a story about a brave knight."
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
// Extract the audio data and handle it for downstream use.
model.generateContentStream(prompt).collect { chunk ->
val part = chunk.candidates.firstOrNull()?.content?.parts?.firstOrNull()
if (part is InlineDataPart) {
val pcmChunk = part.inlineData // Raw PCM bytes (24kHz, 1 channel, 16-bit)
val mimeType = part.mimeType // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk)
}
}
Java
如果是 Java,這個 SDK 中的串流方法會從 Reactive Streams 程式庫傳回Publisher 型別。
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
GenerationConfig config = new GenerationConfig.Builder()
.setResponseModalities(Collections.singletonList(ResponseModality.AUDIO))
.setSpeechConfig(new SpeechConfig(new Voice("Kore")))
.build();
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
GenerativeModel ai = FirebaseAI.getInstance(GenerativeBackend.googleAI())
.generativeModel("gemini-3.1-flash-tts-preview", config);
GenerativeModelFutures model = GenerativeModelFutures.from(ai);
// Provide a text prompt.
String prompt = "Tell me a story about a brave knight.";
Content content = new Content.Builder().addText(prompt).build();
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
Publisher<GenerateContentResponse> streamingResponse =
model.generateContentStream(content);
// Extract the audio data and handle it for downstream use.
streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
@Override
public void onSubscribe(Subscription s) {
s.request(Long.MAX_VALUE);
}
@Override
public void onNext(GenerateContentResponse chunk) {
Part part = chunk.getCandidates().get(0).getContent().getParts().get(0);
if (part instanceof InlineDataPart) {
byte[] pcmChunk = ((InlineDataPart) part).getInlineData(); // Raw PCM bytes (24kHz, 1 channel, 16-bit)
String mimeType = ((InlineDataPart) part).getMimeType(); // for example: "audio/pcm"
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
@Override
public void onComplete() {
// Audio stream complete.
}
@Override
public void onError(Throwable t) {
t.printStackTrace();
}
});
Web
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
// TODO(developer): Replace with your app's Firebase configuration
const firebaseConfig = { /* ... */ };
const firebaseApp = initializeApp(firebaseConfig);
// Initialize the Gemini Developer API backend service.
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Set `responseModalities` to include `AUDIO`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
const generationConfig = {
responseModalities: [ResponseModality.AUDIO],
speechConfig: {
voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } }
}
};
// Create a `GenerativeModel` instance with a model that supports speech generation.
const model = getGenerativeModel(ai, {
model: "gemini-3.1-flash-tts-preview",
generationConfig
});
// Provide a text prompt.
const prompt = "Tell me a story about a brave knight.";
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
const result = await model.generateContentStream(prompt);
// Extract the audio data and handle it for downstream use. For example:
const playbackQueue = [];
for await (const chunk of result.stream) {
const inlineDataParts = chunk.inlineDataParts();
if (inlineDataParts?.[0]) {
const pcmBase64 = inlineDataParts[0].inlineData.data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
const pcmBuffer = Uint8Array.from(atob(pcmBase64), c => c.charCodeAt(0)).buffer;
// Append the audio chunk to your audio queue/buffer for playback.
playbackQueue.push(pcmBuffer);
}
}
// To play back an array of PCM buffers in sequence, you'll need to write your own `processPlaybackQueue` function.
processPlaybackQueue(playbackQueue);
Dart
import 'package:firebase_ai/firebase_ai.dart';
// Set `responseModalities` to include `audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
final config = GenerationConfig(
responseModalities: [ResponseModality.audio],
speechConfig: SpeechConfig(voiceName: 'Kore'),
);
// Initialize the Gemini Developer API backend service.
// Create a `GenerativeModel` instance with a model that supports speech generation.
final model = FirebaseAI.googleAI().generativeModel(
model: 'gemini-3.1-flash-tts-preview',
config: config,
);
// Provide a text prompt.
final prompt = 'Tell me a story about a brave knight.';
// Call `generateContentStream` to generate the speech output stream based on your text prompt.
final responseStream = model.generateContentStream([Content.text(prompt)]);
// Extract the audio data and handle it for downstream use. For example:
await for (final chunk in responseStream) {
final part = chunk.candidates.first.content.parts.first;
if (part is InlineDataPart && part.mimeType.startsWith('audio/')) {
final Uint8List pcmChunk = part.bytes; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
Unity
using System.Collections.Generic;
using System.Linq;
using Firebase.AI;
// Initialize the Gemini Developer API backend service.
var ai = FirebaseAI.GetInstance(FirebaseAI.Backend.GoogleAI());
// Set `ResponseModalities` to include `Audio`.
// Configure a `SpeechConfig` with your chosen voice name (and optionally a language code).
var config = new GenerationConfig(
responseModalities: new List<ResponseModality> { ResponseModality.Audio },
speechConfig: SpeechConfig.UsePrebuiltVoice("Kore")
);
// Create a `GenerativeModel` instance with a model that supports speech generation.
var model = ai.GetGenerativeModel(
modelName: "gemini-3.1-flash-tts-preview",
generationConfig: config
);
// Provide a text prompt.
var prompt = "Tell me a story about a brave knight.";
// Call `GenerateContentStreamAsync` to generate the speech output stream based on your text prompt.
var responseStream = model.GenerateContentStreamAsync(prompt);
// Extract the audio data and handle it for downstream use. For example:
await foreach (var response in responseStream)
{
var audioParts = response.Candidates.FirstOrDefault().Content.Parts
.OfType<ModelContent.InlineDataPart>();
foreach (var part in audioParts)
{
byte[] pcmChunk = part.Data; // Raw PCM bytes (24kHz, 1 channel, 16-bit)
// Append the audio chunk to your audio queue/buffer for playback.
appendAudioChunk(pcmChunk);
}
}
使用提示控制語音輸出
您可以使用特定提示技巧,影響生成的語音語氣、語速和風格。
以下小節將說明提示結構和音訊標記,並提供高階指引。如需詳細指引,請參閱這份提示指南。
提示結構
為獲得最佳結果,請在提示中加入下列元素:
Audio Profile:描述說話者的角色、核心身分和原型 (例如A warm, professional narrator)。Scene:描述環境和情緒氛圍 (例如In a quiet library或Amidst a noisy crowd)。Director's Notes:描述情緒、語速、風格和口音 (例如Speak slowly and with mystery)。Sample Context:為模型提供起點 (例如The speaker is greeting a close friend)。轉錄稿:要朗讀的實際文字。為獲得最佳效能,請確保文字的語氣和情境與語音設定檔和導演筆記一致。
範例提示詞:
[Audio Profile: A young, energetic voice]
[Scene: A lively sports broadcast]
[Director's Notes: Speak fast, with high energy and excitement]
[Sample Context: The game just ended with a last-second touchdown]
Welcome back fans! What an incredible game we're witnessing today!
音訊標記
您可以直接在文字提示中插入格式化標記,引導模型執行作業。
只有在使用 Gemini 3.x TTS 模型時,系統才會支援音訊標記。
常用的標記包括:
[whispers]:以氣音說話[laughs]:加入笑聲[giggles]:新增笑聲[sighs]:新增嘆氣聲[gasp]:新增喘息聲[shouting]:大聲喊叫[excited]:表示興奮[serious]:認真說話[sighs whispers]:綜合情緒 (可合併標記)
使用音訊標記時,請注意下列事項:
沒有詳盡的清單:支援的標記沒有固定清單。您可以嘗試使用不同的情緒和表情符號 (例如
[bored]、[sarcastically],甚至是[like dracula]),看看輸出內容會如何變化。非英文文字提示詞:如果文字提示詞不是英文,建議使用英文音訊標記,以獲得最佳結果。
範例提示詞:
I have a secret to tell you. [whispers] I found the hidden treasure. [laughs] I can't believe it!
限制與需求
使用語音生成功能時,請注意下列限制和規定:
多位說話者設定僅支援 2 位說話者。
使用 Gemini 3.x TTS 模型時,系統僅支援下列功能:串流、音訊標記和額外自動偵測語言。
「gemini-3.1-flash-tts-preview」的限制
- 聲音不一致:如果提示的語氣和情境與所選講者的個人資料不一致 (例如,嘗試以年輕女孩的聲音說話的低沉男聲),模型輸出內容可能不會完全符合所選講者。確認提示內容與語音相符。
- 輸出內容較長:如果音訊長度超過幾分鐘,語音品質和一致性可能會下降。建議將長篇文字提示拆成多個小段。
- 偶爾會傳回文字權杖:模型偶爾會傳回文字權杖,而非音訊權杖,導致要求失敗並出現
500錯誤。由於只有少數要求會隨機發生這種情況,建議您在應用程式中導入重試邏輯。 - 分類器誤判拒絕:如果提示內容含糊不清,語音合成分類器可能會拒絕要求 (
PROHIBITED_CONTENT),或是導致模型大聲朗讀樣式指令。為避免這種情況,請在提示開頭使用結構化提示,並加入清楚的前言 (例如Audio Profile和Director's Notes)。
支援的語音和語言
Gemini TTS 模型會接收文字輸入內容並生成音訊輸出內容,因此回應本身就是合成語音。下列小節列出 Gemini TTS 模型可「說」的語音和語言 (或用來回覆)。
這些聲音支援多種語言,因此你可以使用同一種聲音,以任何支援的語言生成語音。舉例來說,您可以將語音設為 Kore,並以西班牙文、北印度文和越南文傳送一組文字提示。回覆內容都會以 Kore 聲音呈現,但會使用不同語言。
聲音名稱
GeminiTTS 模型支援 30 種不同的合成 HD 語音,各有不同特色。展開下方部分,即可查看回覆語音選項清單,並試聽各個語音。
語言
Gemini TTS 模型可自動偵測文字提示詞中的下列語言。系統會以該語言生成語音。
請注意,您也可以選擇在語音設定中明確設定語言代碼。
所有音訊生成模型支援的語言
| 語言 | BCP-47 代碼 | 語言 | BCP-47 代碼 |
|---|---|---|---|
| 阿拉伯文 (埃及) | ar-EG | 德文 (德國) | de-DE |
| 英文 (美國) | en-US | 西班牙文 (美國) | es-US |
| 法文 (法國) | fr-FR | 北印度文 (印度) | hi-IN |
| 印尼文 (印尼) | id-ID | 義大利文 (義大利) | it-IT |
| 日文 (日本) | ja-JP | 韓文 (韓國) | ko-KR |
| 葡萄牙文 (巴西) | pt-BR | 俄文 (俄羅斯) | ru-RU |
| 荷蘭文 (荷蘭) | nl-NL | 波蘭文 (波蘭) | pl-PL |
| 泰文 (泰國) | th-TH | 土耳其文 (土耳其) | tr-TR |
| 越南文 (越南) | vi-VN | 羅馬尼亞文 (羅馬尼亞) | ro-RO |
| 烏克蘭文 (烏克蘭) | uk-UA | 孟加拉文 (孟加拉) | bn-BD |
| 英文 (印度) | en-IN 和 hi-IN 套裝組合 | 馬拉地文 (印度) | mr-IN |
| 泰米爾文 (印度) | ta-IN | 泰盧固文 (印度) | te-IN |
音訊生成 3.x 模型支援的其他語言
| 語言 | BCP-47 代碼 | 語言 | BCP-47 代碼 |
|---|---|---|---|
| 南非荷蘭文 | 南非荷蘭文 | 菲律賓文 | fil |
| 阿爾巴尼亞文 | 阿爾巴尼亞文 | 芬蘭文 | 芬蘭文 |
| 阿姆哈拉文 | 阿姆哈拉文 | 加里西亞文 | 加里西亞文 |
| 亞美尼亞文 | 亞美尼亞文 | 喬治亞文 | 喬治亞文 |
| 亞塞拜然文 | 亞塞拜然文 | 希臘文 | 希臘文 |
| 巴斯克文 | 巴斯克文 | 古吉拉特文 | 古吉拉特文 |
| 白俄羅斯語 | be | 海地克里奧爾文 | ht |
| 保加利亞文 | 保加利亞文 | 希伯來文 | 他 |
| 緬甸文 | 緬甸文 | 匈牙利文 | 匈牙利文 |
| 加泰隆尼亞文 | 加泰隆尼亞文 | 冰島文 | 為 |
| 宿霧文 | ceb | 爪哇語 | jv |
| 中文 (國語) | cmn | 卡納達文 | 卡納達文 |
| 克羅埃西亞文 | 小時 | 貢根文 | kok |
| 捷克文 | 捷克文 | 寮文 | lo |
| 丹麥文 | da | 拉丁文 | la |
| 愛沙尼亞文 | 愛沙尼亞文 | 拉脫維亞文 | lv |
| 立陶宛文 | lt | 盧森堡文 | lb |
| 馬其頓文 | 馬其頓文 | 邁蒂利文 | mai |
| 馬達加斯加文 | mg | 馬來文 | ms |
| 馬拉雅拉姆文 | 馬拉雅拉姆文 | 蒙古文 | mn |
| 尼泊爾文 | 尼泊爾文 | 挪威文 (巴克摩) | nb |
| 挪威文 (新挪威文) | nn | 歐利亞文 | 或 |
| 普什圖文 | ps | 波斯文 | 波斯文 |
| 旁遮普文 | 旁遮普文 | 塞爾維亞文 | 塞爾維亞文 |
| 信德文 | sd | 錫蘭文 | 僧伽羅文 (錫蘭文) |
| 斯洛伐克文 | sk | 斯洛維尼亞文 | 斯洛維尼亞文 |
| 史瓦西里文 | sw | 瑞典文 | 瑞典文 |
| 烏都文 | 烏都文 |
(選用) 明確設定語言代碼
如未在語音設定中指定語言代碼,模型會自動偵測文字提示詞中的語言。
不過,您也可以選擇明確設定語言 (使用語音設定中的 languageCode 參數)。如要進行這項作業,您必須使用下列其中一個支援的 BCP-47 語言代碼:
- 阿拉伯文:
ar-XA - 孟加拉文:
bn-IN - 中文 (普通話):
cmn-CN - 荷蘭文:
nl-NL - 英文:
en-US、en-GB、en-AU、en-IN - 法文:
fr-FR、fr-CA - 德文:
de-DE - 古吉拉特文:
gu-IN - 北印度文:
hi-IN - 印尼文:
id-ID - 義大利文:
it-IT - 日文:
ja-JP - 卡納達文:
kn-IN - 韓文:
ko-KR - 馬拉雅拉姆文:
ml-IN - 馬拉地文:
mr-IN - 波蘭文:
pl-PL - 葡萄牙文:
pt-BR - 俄文:
ru-RU - 西班牙文:
es-US、es-ES - 泰米爾文:
ta-IN - 泰盧固文:
te-IN - 泰文:
th-TH - 土耳其文:
tr-TR - 越南文:
vi-VN
你還可以做些什麼?
- 瞭解如何在將長提示傳送至模型前計算權杖數量。
-
開始思考如何準備供實際運作 (請參閱製片檢查清單):
- 盡早強制執行 Firebase App Check,防止未經授權的用戶端濫用 Gemini API。
- 使用 Firebase Remote Config 或伺服器提示範本,即可視需要變更 AI 功能的設定 (例如模型名稱),不必發布新版應用程式。
試試其他功能
- 建構多輪對話 (即時通訊)。
- 使用純文字提示生成文字。
- 從文字和多模態提示生成結構化輸出內容 (例如 JSON)。
- 生成及編輯圖片: 使用文字和多模態提示生成及編輯圖片。
- 使用 Gemini Live API 串流輸入和輸出 (包括音訊)。
-
使用工具 (例如函式呼叫
和 Grounding with
Google Search 或Google Maps ) ,將 Gemini 模型連結至應用程式的其他部分,以及外部系統和資訊。
瞭解如何控管內容生成
您也可以使用 Google AI Studio 測試提示和模型設定,甚至取得生成的程式碼片段。進一步瞭解支援的機型
瞭解各種用途適用的模型,以及這些模型的配額和價格。提供有關 Firebase AI Logic 的使用體驗意見回饋