OnDeviceConfig

@PublicPreviewAPI
public final class OnDeviceConfig

Configuration for on-device AI model inference.

Summary

Nested types
public static class OnDeviceConfig.Companion

Public fields
static final @NonNull OnDeviceConfig

A default configuration that only uses in-cloud inference.
final int

The number of generated responses to return.
final Integer

The maximum number of tokens to generate in the response.
final @NonNull InferenceMode

The InferenceMode to use for the model.
final Integer

The seed to use for generation to ensure reproducibility.
final Float

A parameter controlling the degree of randomness in token selection.
final Integer

The topK parameter changes how the model selects tokens for output.

Public constructors
OnDeviceConfig(
    @NonNull InferenceMode mode,
    Integer maxOutputTokens,
    Float temperature,
    Integer topK,
    Integer seed,
    int candidateCount
)

Public fields

IN_CLOUD

public static final @NonNull OnDeviceConfig IN_CLOUD

A default configuration that only uses in-cloud inference.

candidateCount

public final int candidateCount

The number of generated responses to return. See GenerationConfig for more detail. By default it's set to 1.

maxOutputTokens

public final Integer maxOutputTokens

The maximum number of tokens to generate in the response. See GenerationConfig for more detail.

mode

public final @NonNull InferenceMode mode

The InferenceMode to use for the model.

seed

public final Integer seed

The seed to use for generation to ensure reproducibility. See GenerationConfig for more detail.

temperature

public final Float temperature

A parameter controlling the degree of randomness in token selection. See GenerationConfig for more detail.

topK

public final Integer topK

The topK parameter changes how the model selects tokens for output. See GenerationConfig for more detail.

Public constructors

OnDeviceConfig

public OnDeviceConfig(
    @NonNull InferenceMode mode,
    Integer maxOutputTokens,
    Float temperature,
    Integer topK,
    Integer seed,
    int candidateCount
)