Firebase.AI.LiveSession

Manages asynchronous communication with Gemini model over a WebSocket connection.

Summary

Warning: This API is in Public Preview, which means that the feature is not subject to any SLA or deprecation policy and could change in backwards-incompatible ways.

Inheritance

Inherits from: IDisposable

Protected functions
Dispose(bool disposing)
virtual void

Public functions
CloseAsync(CancellationToken cancellationToken)
Task
Close the LiveSession.
Dispose()
void
ReceiveAsync(CancellationToken cancellationToken)
async IAsyncEnumerable< LiveSessionResponse >
Receives a stream of responses from the server.
ResumeSessionAsync(SessionResumptionConfig sessionResumption, CancellationToken cancellationToken)
async Task
Resumes an existing live session with the server.
SendAsync(ModelContent? content, bool turnComplete, CancellationToken cancellationToken)
async Task
Sends a single piece of content to the server.
SendAudioAsync(float[] audioData, CancellationToken cancellationToken)
Task
Convenience function for sending audio data in a float[] to the server.
SendAudioRealtimeAsync(ModelContent.InlineDataPart audio, CancellationToken cancellationToken)
async Task
Sends audio data to the server in realtime.
SendMediaChunksAsync(List< ModelContent.InlineDataPart > mediaChunks, CancellationToken cancellationToken)
async Task
Deprecated. Use SendAudioRealtimeAsync, SendVideoRealtimeAsync, or SendTextRealtimeAsync instead.
Send realtime input to the server.
SendStartActivityRealtimeAsync(CancellationToken cancellationToken)
async Task
Manually marks the start of user activity, using the realtime API.
SendStopActivityRealtimeAsync(CancellationToken cancellationToken)
async Task
Manually marks the end of user activity, using the realtime API.
SendTextRealtimeAsync(string text, CancellationToken cancellationToken)
async Task
Sends text data to the server in realtime.
SendVideoRealtimeAsync(ModelContent.InlineDataPart video, CancellationToken cancellationToken)
async Task
Sends video data to the server in realtime.

Protected functions

Dispose

virtual void Dispose(
  bool disposing
)

Public functions

CloseAsync

Task CloseAsync(
  CancellationToken cancellationToken
)

Close the LiveSession.

Details
Parameters
cancellationToken
A token to cancel the operation.

Dispose

void Dispose()

ReceiveAsync

async IAsyncEnumerable< LiveSessionResponse > ReceiveAsync(
  CancellationToken cancellationToken
)

Receives a stream of responses from the server.

Having multiple of these ongoing will result in unexpected behavior. Closes upon receiving a TurnComplete from the server.

Details
Parameters
cancellationToken
A token to cancel the operation.
Returns
A stream of LiveContentResponses from the backend.

ResumeSessionAsync

async Task ResumeSessionAsync(
  SessionResumptionConfig sessionResumption,
  CancellationToken cancellationToken
)

Resumes an existing live session with the server.

This closes the current WebSocket connection and establishes a new one using the same configuration as the original session.

Details
Parameters
sessionResumption
The configuration for session resumption.
cancellationToken
A token to cancel the operation.

SendAsync

async Task SendAsync(
  ModelContent? content,
  bool turnComplete,
  CancellationToken cancellationToken
)

Sends a single piece of content to the server.

Details
Parameters
content
The content to send.
turnComplete
Indicates to the server that the client's turn is complete.
cancellationToken
A token to cancel the send operation.

SendAudioAsync

Task SendAudioAsync(
  float[] audioData,
  CancellationToken cancellationToken
)

Convenience function for sending audio data in a float[] to the server.

Details
Parameters
audioData
The audio data to send. Expected format: 16 bit PCM audio at 16kHz little-endian.
cancellationToken
A token to cancel the send operation.

SendAudioRealtimeAsync

async Task SendAudioRealtimeAsync(
  ModelContent.InlineDataPart audio,
  CancellationToken cancellationToken
)

Sends audio data to the server in realtime.

Check https://ai.google.dev/api/live#bidigeneratecontentrealtimeinput for details about the realtime input usage.

Details
Parameters
audio
The audio data to send.
cancellationToken
A token to cancel the send operation.

SendMediaChunksAsync

async Task SendMediaChunksAsync(
  List< ModelContent.InlineDataPart > mediaChunks,
  CancellationToken cancellationToken
)

Send realtime input to the server.

Details
Parameters
mediaChunks
A list of media chunks to send.
cancellationToken
A token to cancel the send operation.

Use SendAudioRealtimeAsync, SendVideoRealtimeAsync, or SendTextRealtimeAsync instead.

Deprecated. Use SendAudioRealtimeAsync, SendVideoRealtimeAsync, or SendTextRealtimeAsync instead.

SendStartActivityRealtimeAsync

async Task SendStartActivityRealtimeAsync(
  CancellationToken cancellationToken
)

Manually marks the start of user activity, using the realtime API.

The start of user activity is effectively the start of a user's turn, but depending on the configuration defined in RealtimeInputConfig, it may not be interpreted as an interruption. An example of the start of user activity could be the user speaking (not silence).

Should be followed with a call to SendStopActivityRealtimeAsync(CancellationToken); after all the data has been sent for the user's turn.

Only required when automatic activity detection is disabled via ActivityDetectionConfig.Disabled().

SendStopActivityRealtimeAsync

async Task SendStopActivityRealtimeAsync(
  CancellationToken cancellationToken
)

Manually marks the end of user activity, using the realtime API.

The end of user activity is effectively the end of a user's turn, and signals that the model can start sending responses.

Should follow after a previous call to SendStartActivityRealtimeAsync(CancellationToken).

Only required when automatic activity detection is disabled via ActivityDetectionConfig.Disabled().

SendTextRealtimeAsync

async Task SendTextRealtimeAsync(
  string text,
  CancellationToken cancellationToken
)

Sends text data to the server in realtime.

Check https://ai.google.dev/api/live#bidigeneratecontentrealtimeinput for details about the realtime input usage.

Details
Parameters
text
The text data to send.
cancellationToken
A token to cancel the send operation.

SendVideoRealtimeAsync

async Task SendVideoRealtimeAsync(
  ModelContent.InlineDataPart video,
  CancellationToken cancellationToken
)

Sends video data to the server in realtime.

Check https://ai.google.dev/api/live#bidigeneratecontentrealtimeinput for details about the realtime input usage.

Details
Parameters
video
The video data to send.
cancellationToken
A token to cancel the send operation.