[Video](https://www.youtube.com/watch?v=wBfqpPxUwqM)

You can build AI-powered Web apps and features with hybrid inference using
Firebase AI Logic. Hybrid inference enables running inference using
on-device models when available and seamlessly falling back to cloud-hosted
models otherwise (and vice versa).

This page describes how to
[get started using the client SDK](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#get-started). After completing this standard
setup, check out the
[additional configuration options and capabilities](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt).

Note that on-device inference via Firebase AI Logic is supported for
web apps using Firebase AI Logic SDK v12.5.0+ and running on
Chrome on Desktop (version 139 or higher).

[Jump to view code examples](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#initialize-service-and-model)

### Use cases and supported capabilities

Depending on your app's use cases, you can take advantage of several benefits
of on-device or hybrid experiences:

- Benefits of using an **on-device model** for inference:

  - Enhanced privacy
  - Local context
  - Inference at no-cost
  - Offline functionality
- Benefits of using a **hybrid** approach:

  - Provide a similar app experience, regardless of the end-user's device
  - Improve availability of generative AI features, regardless of internet connectivity, quota limitations, or device capabilities

#### Supported on-device inference capabilities

On-device inference only supports **single-turn text generation (*not* chat)**
(with streaming or non-streaming output), specifically the following
capabilities:

- Generating [text from text-only input](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#text-in-text-out).

- Generating [text from text-and-image input](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#image-in-text-out), specifically
  input image types of JPEG and PNG.

- Generating [structured output](https://firebase.google.com/docs/ai-logic/hybrid/web/generate-structured-output.md.txt),
  including JSON and enums.

Make sure to review the list of
[not-yet-supported features for hybrid or on-device inference](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#features-not-yet-supported)
at the bottom of this page.

### Supported APIs and platforms

The following sections describe the supported APIs, as well as the supported
platforms and browsers.

#### Supported APIs for in-cloud vs on-device inference

- In-cloud inference uses your chosen Gemini API provider (either the
  Gemini Developer API or the Agent Platform Gemini API (formerly Vertex AI)).

- On-device inference uses the
  [Prompt API from Chrome](https://developer.chrome.com/docs/ai/prompt-api).

#### Supported platforms and browsers

For on-device inference (which uses the Prompt API from Chrome), the end-user
of your app must be running Chrome on Desktop (version 139 or higher).
Learn more in the
[Chrome documentation](https://developer.chrome.com/docs/ai/prompt-api).

## Get started on localhost

These get started steps describe the required general setup for any supported
prompt request that you want to send.

You can learn more about
[using APIs on localhost](https://developer.chrome.com/docs/ai/get-started#use_apis_on_localhost)
in the Chrome documentation.

### **Step 1**: Set up Chrome and the Prompt API for on-device inference

1. Make sure you're using a recent version of Chrome. Update in
   <chrome://settings/help>.  

   On-device inference is available from Chrome v139 and higher.

2. Enable the on-device multimodal model by setting the following flag to
   **Enabled**:

   - `chrome://flags/#prompt-api-for-gemini-nano-multimodal-input`
3. Restart Chrome.

4. *(Optional)* Download the on-device model before the first request.

   The Prompt API is built into Chrome; however, the on-device model isn't
   available by default. If you haven't yet downloaded the model before your
   first request for on-device inference, the request will automatically start
   the model download in the background.

   > [!NOTE]
   > **Note:** Downloading the model can take several minutes, so waiting to auto-download with the first request can significantly delay receiving a response to that request.

   <br />

   View instructions to download the on-device model

   <br />

   1. Open **Developer Tools \> Console**.

   2. Run the following:

      ```javascript
      await LanguageModel.availability();
      ```
   3. Make sure that the output is `available`, `downloading`, or
      `downloadable`.

   4. If the output is `downloadable`, start the model download by running:

      ```javascript
      await LanguageModel.create();
      ```
   5. You can use the following `monitor` callback to listen for download
      progress and make sure that the model is `available` before making
      requests:

      ```javascript
      const session = await LanguageModel.create({
        monitor(m) {
          m.addEventListener("downloadprogress", (e) => {
            console.log(`Downloaded ${e.loaded * 100}%`);
          });
        },
      });
      ```

   <br />

   <br />

### **Step 2**: Set up a Firebase project and connect your app to Firebase

1. Sign into the [Firebase console](https://console.firebase.google.com/),
   and then select your Firebase project.

   <br />

   Don't already have a Firebase project?

   <br />

   If you don't already have a Firebase project, click the button to create a
   new Firebase project, and then use either of the following options:
   - **Option 1** : Create a wholly new Firebase project (and its underlying
     Google Cloud project automatically) by entering a new project name in the
     first step of the workflow.

   - **Option 2** : "Add Firebase" to an existing Google Cloud project by
     clicking **Add Firebase to Google Cloud project** (at bottom of page).
     In the first step of the workflow, start entering the **project name** of
     the existing project, and then select the project from the displayed list.

   Complete the remaining steps of the on-screen workflow to create a Firebase
   project. Note that when prompted, you do ***not*** need to set up
   Google Analytics to use the Firebase AI Logic SDKs.

   <br />

   <br />

2. In the Firebase console, go to **AI Services** \>
   [**AI Logic**](https://console.firebase.google.com/project/_/ailogic/?useAutoProject=true).

3. Click **Get started** to launch a guided workflow that helps you set up the
   [required APIs](https://firebase.google.com/docs/ai-logic/faq-and-troubleshooting.md.txt#required-apis)
   and resources for your project.

4. If prompted, follow the on-screen instructions to register your app and add
   your Firebase configuration to your app.

5. When asked to choose a "Gemini API provider", we recommend selecting the
   **Gemini Developer API**, which lets you get started quickly at no cost.

   At any point later, you can always
   [set up the Agent Platform Gemini API (formerly Vertex AI)](https://console.firebase.google.com/project/_/ailogic?openVertexAiOnboarding=true&useAutoProject=true)
   (and its requirement for billing).
6. Continue in the workflow to set up the required APIs and associated services
   for Firebase AI Logic.

   This stage of the workflow automatically
   [enforces Firebase App Check for AI Logic](https://firebase.google.com/docs/ai-logic/app-check.md.txt),
   which is a *critical* service to help protect the Gemini API when it's
   directly accessed from your app. As part of getting started (see steps later
   in this guide), you'll need to configure the App Check *debug provider*
   for local development when App Check is enforced.

   > [!NOTE]
   > **Note:** To release your app's AI feature to end users, you need to [register your app with a production App Check attestation provider](https://firebase.google.com/docs/ai-logic/app-check.md.txt) so that they can use the feature when App Check is enforced.

7. Continue to the next step in this guide to add the required SDKs to your
   app.

### **Step 3**: Add the SDK

The Firebase library provides access to the APIs for interacting with generative
models. The library is included as part of the Firebase JavaScript SDK for Web.

1. Install the Firebase JS SDK for Web using npm:

   ```console
   npm install firebase
   ```
2. Initialize Firebase in your app:

   ```javascript
   import { initializeApp } from "firebase/app";
   import { initializeAppCheck } from "firebase/app-check";

   // TODO(developer) Replace the following with your app's Firebase configuration
   // See: https://firebase.google.com/docs/web/learn-more#config-object
   const firebaseConfig = {
     // ...
   };

   // Initialize FirebaseApp
   const firebaseApp = initializeApp(firebaseConfig);
   ```

### **Step 4** : Configure the App Check debug provider for local development

As part of the guided setup workflow for AI Logic in the console, Firebase App Check is automatically enforced to protect the Gemini API.
For local development, you need to configure the
App Check *debug provider* to bypass attestation while still maintaining the
enforcement of App Check.

> [!IMPORTANT]
> **Important:** Make sure that App Check is enforced for AI Logic. In the Firebase console, go to the **Security** \> **App Check** \> [**APIs** tab](https://console.firebase.google.com/project/_/appcheck/apps/products/?useAutoProject=true), and then find the row for Firebase AI Logic. If it says `Unenforced`, then follow the [instructions to enforce App Check yourself](https://firebase.google.com/docs/ai-logic/app-check.md.txt#enforce-app-check-for-debug-provider-only).

Here's how to use the debug provider while running your app from `localhost`
interactively (for example, during local development):

> [!WARNING]
> **Warning:** Do ***not*** try to enable `localhost` debugging by adding `localhost` to the allowed domains for reCAPTCHA. Doing so would allow anyone to run your app from their local machines!

1. In your debug build, enable debug mode by setting
   `self.FIREBASE_APPCHECK_DEBUG_TOKEN` to `true` before you initialize
   App Check. For example:

   ```javascript
   self.FIREBASE_APPCHECK_DEBUG_TOKEN = true;
   initializeAppCheck(firebaseApp, { /* App Check options */ });
   ```
2. Visit your web app locally and open the browser's developer tools. In the
   debug console, you'll see a debug token:

       AppCheck debug token: "123a4567-b89c-12d3-e456-789012345678".
       You will need to safelist it in the Firebase console for it to work.

3. Register your debug token with App Check:

   1. In the Firebase console, go to the
      **Security** \> **App Check** \>
      [**Apps** tab](https://console.firebase.google.com/project/_/appcheck/apps/?useAutoProject=true).

   2. Find your app, click the overflow menu
      (), and then select
      **Manage debug tokens**.

   3. Follow the on-screen instructions to register your debug token.

For details about the debug provider (including how to get a new debug token),
check out the
[official App Check docs](https://firebase.google.com/docs/app-check/web/debug-provider.md.txt).

> [!CAUTION]
> **Here are some critical points about the
> App Check debug provider:**
>
> - **Keep your debug token and debug build private.** Don't commit your debug token to a public repository, and don't ship your debug token or debug build in production builds of your app.
> - **Register your app with a production attestation provider before
>   releasing to end users.** You'll need to [register your app with a production App Check attestation provider](https://firebase.google.com/docs/ai-logic/app-check.md.txt) (for example, Play Integrity) so that your end-users can use your feature with App Check enforced.

### **Step 5**: Initialize the service and create a model instance

|---|
| *Click your Gemini API provider to view provider-specific content and code on this page.* <button value="dev" default="">Gemini Developer API</button> <button value="vertex">Agent Platform Gemini API</button> |

Set up the following before you send a prompt request to the model:

1. Initialize the service for your chosen API provider.

2. Create a `GenerativeModel` instance. Make sure to set the `mode` to one of:

   - **`PREFER_ON_DEVICE`** : Use the on-device model if it's available;
     otherwise, *fall back to the cloud-hosted model*.

   - **`ONLY_ON_DEVICE`** : Use the on-device model if it's available;
     otherwise, *throw an exception*.

   - **`PREFER_IN_CLOUD`** : Use the cloud-hosted model if it's available;
     otherwise, *fall back to the on-device model*.

   - **`ONLY_IN_CLOUD`** : Use the cloud-hosted model if it's available;
     otherwise, *throw an exception*.

```javascript
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, InferenceMode } from "firebase/ai";

// TODO(developer) Replace the following with your app's Firebase configuration
// See: https://firebase.google.com/docs/web/learn-more#config-object
const firebaseConfig = {
  // ...
};

// Initialize FirebaseApp
const firebaseApp = initializeApp(firebaseConfig);

// Initialize the Gemini Developer API backend service
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
// Create a \`GenerativeModel\` instance
// Set the mode (for example, use the on-device model if it's available)
const model = getGenerativeModel(ai, { mode: InferenceMode.PREFER_ON_DEVICE });
```

> [!NOTE]
> **Note** : Keep the following in mind:
>
> - Before implementing hybrid or on-device inference, make sure to review the [list of not-yet-supported features](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#features-not-yet-supported).
> - The default cloud-hosted model is `gemini-3.5-flash-lite`, but you can [override the default](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#override-default-cloud-model).
> - As part of the response, the SDK tells you [whether on-device or in-cloud inference was used](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#determine-inference-mode).

### **Step 6**: Initialize the on-device model

You must call `initializeDeviceModel()` *after or on* an end-user page
interaction (like a button click) and *before* you send a prompt request to the
model. Learn more about the
[user activation requirement](https://developer.chrome.com/docs/ai/get-started#user-activation)
in the Chrome documentation.

```javascript
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, InferenceMode } from "firebase/ai";

// TODO(developer) Replace the following with your app's Firebase configuration
// See: https://firebase.google.com/docs/web/learn-more#config-object
const firebaseConfig = {
  // ...
};

// Initialize FirebaseApp
const firebaseApp = initializeApp(firebaseConfig);

// Initialize the Gemini Developer API backend service
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });

// Create a `GenerativeModel` instance
// Set the mode (for example, use the on-device model if it's available)
const model = getGenerativeModel(ai, { mode: InferenceMode.PREFER_ON_DEVICE });

// \`initializeDeviceModel\` must be called:
// (1) after or on an end-user page interaction such as a button click
// and
// (2) before any queries to the model (such as \`generateContent()\`)
// You may want to \`await\` this promise if using \`ONLY_ON_DEVICE\` (see note below).
model.initializeDeviceModel((val) =\>
// Example: "Download progress: 72.62%""
console.log(\`Download progress: ${Math.round(val\*10000) / 100}%\`)
);
```

> [!NOTE]
> **Downloading the on-device model can take several minutes.**
>
>
> If you haven't yet
> [downloaded the model
> before your first request for on-device inference](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started#instructions-to-download-on-device-model),
> calling `initializeDeviceModel()` will automatically start the
> model download in the background. Waiting for
> `initializeDeviceModel()` to complete ensures the model is ready
> for immediate queries but may take several minutes.
>
>
> Calling methods like `generateContent()` before the download is
> complete will have different results based on the mode:
>
> - `ONLY_ON_DEVICE`: the request will wait until the download is complete, and then send the prompt to the on-device model and return the result. This may take several minutes.
> - `PREFER_IN_CLOUD` or `PREFER_ON_DEVICE`: the request will *not* wait for the download to complete and will automatically fall back to use the cloud-hosted model.
> - `ONLY_IN_CLOUD`: the request will automatically fall back to use the cloud-hosted model.

### **Step 7**: Send a prompt request to a model

This section shows you how to send various types of input to generate different
types of output, including:

- [Generate text from text-only input](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#text-in-text-out)
- [Generate text from text-and-image (multimodal) input](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#image-in-text-out)

If you want to generate structured output (like JSON or enums), then
use one of the following "generate text" examples and additionally
[configure the model to respond according to a provided schema](https://firebase.google.com/docs/ai-logic/hybrid/web/generate-structured-output.md.txt).

#### Generate text from text-only input

|---|
| *Before trying this sample, make sure that you've completed the [Get started](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#get-started) section of this guide.* |

You can use
[`generateContent()`](https://firebase.google.com/docs/reference/js/ai.generativemodel.md.txt#generativemodelgeneratecontent)
to generate text from a prompt that contains text:

```javascript
// Imports + initialization of FirebaseApp and backend service + creation of model instance

// Wrap in an async function so you can use await
async function run() {
  // Provide a prompt that contains text
  const prompt = "Write a story about a magic backpack."

  // To generate text output, call `generateContent` with the text input
  const result = await model.generateContent(prompt);

  const response = result.response;
  const text = response.text();
  console.log(text);
}

run();
```

Note that Firebase AI Logic also supports streaming of text responses using
[`generateContentStream`](https://firebase.google.com/docs/reference/js/ai.generativemodel.md.txt#generativemodelgeneratecontentstream)
(instead of `generateContent`).

#### Generate text from text-and-image (multimodal) input

|---|
| *Before trying this sample, make sure that you've completed the [Get started](https://firebase.google.com/docs/ai-logic/hybrid/web/get-started.md.txt#get-started) section of this guide.* |

You can use
[`generateContent()`](https://firebase.google.com/docs/reference/js/ai.generativemodel.md.txt#generativemodelgeneratecontent)
to generate text from a prompt that contains text and image files---providing each
input file's `mimeType` and the file itself.

The supported input image types for on-device inference are PNG and JPEG.

> [!IMPORTANT]
> **Important:** For on-device inference, the maximum token limit is 9216 tokens.

```javascript
// Imports + initialization of FirebaseApp and backend service + creation of model instance

// Converts a File object to a Part object.
async function fileToGenerativePart(file) {
  const base64EncodedDataPromise = new Promise((resolve) => {
    const reader = new FileReader();
    reader.onloadend = () => resolve(reader.result.split(',')[1]);
    reader.readAsDataURL(file);
  });
  return {
    type: "inlineData",
    inlineData: { data: await base64EncodedDataPromise, mimeType: file.type },
  };
}

async function run() {
  // Provide a text prompt to include with the image
  const prompt = "Write a poem about this picture:";

  const fileInputEl = document.querySelector("input[type=file]");
  const imagePart = await fileToGenerativePart(fileInputEl.files[0]);

  // To generate text output, call `generateContent` with the text and image
  const result = await model.generateContent([prompt, imagePart]);

  const response = result.response;
  const text = response.text();
  console.log(text);
}

run();
```

Note that Firebase AI Logic also supports streaming of text responses using
[`generateContentStream`](https://firebase.google.com/docs/reference/js/ai.generativemodel.md.txt#generativemodelgeneratecontentstream)
(instead of `generateContent`).

## What else can you do?

You can use various additional configuration options and capabilities for your
hybrid experiences:

- [Set an inference mode.](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#inference-modes)

- [Determine whether on-device or in-cloud inference was used.](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#determine-inference-mode)

- [Override the default cloud-hosted fallback model.](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#override-default-cloud-model)

- [Use model configuration to control responses.](https://firebase.google.com/docs/ai-logic/hybrid/web/configuration-options.md.txt#model-config)

- [Generate structured output (like JSON and enums).](https://firebase.google.com/docs/ai-logic/hybrid/web/generate-structured-output.md.txt)

## Features not yet supported

Not all the capabilities of the Web SDK or cloud-hosted models are available for
hybrid or on-device inference.

### Features not yet supported for **hybrid** implementations

The following features cannot be used with hybrid implementations (neither
on-device nor via cloud fallback):

- **Gemini Live API**: Bi-directional real-time streaming is not supported.

- **Server prompt templates**: Managed server prompt templates are not supported.

- **Count tokens**: Count tokens shouldn't be relied upon because the count will
  differ between cloud-hosted and on-device models, so there's no intuitive
  fallback.

### Features not yet supported for **on-device** inference

**The following features are *not yet supported for on-device inference*, but
they are available for cloud-hosted models.**

> [!NOTE]
> **Note:** For many of these not-yet-supported features, if you set the mode to `PREFER_ON_DEVICE`, the SDK will automatically fall back to use the cloud-hosted model. However, to provide a more consistent experience for your users, consider directly using a cloud-hosted model.

- Generating images using Gemini Nano Banana models

  - Can fall back to the cloud-hosted model; however, `ONLY_ON_DEVICE` mode will throw an error.
- Generating text from image file input types other than JPEG and PNG

  - Can fall back to the cloud-hosted model; however, `ONLY_ON_DEVICE` mode will throw an error.
- Generating text from multimodal inputs of audio, video, and
  documents (PDFs)

  - Can fall back to the cloud-hosted model; however, `ONLY_ON_DEVICE` mode will throw an error.
- Providing files using URLs in multimodal requests. You must provide files as
  inline data to on-device models.

- Multi-turn chat

  - Can fall back to the cloud-hosted model; however, `ONLY_ON_DEVICE` mode will throw an error.
- Providing the model with *tools* to help it generate its response
  (like function calling, code execution, URL context,
  Grounding with Google Search, and Grounding with Google Maps)

- AI monitoring in the Firebase console does ***not*** show any data for
  on-device inference (including on-device logs). However, any inference that
  uses a cloud-hosted model can be monitored just like other inference via
  Firebase AI Logic.

### Additional limitations

In addition to the above, ***on-device* inference has the following
limitations**:

- The end-user of your app must be running Chrome on Desktop (version 139 or
  higher) with the required flags enabled.

- The on-device model requires sufficient storage space on the user's device for
  the initial model download.

<br />

[Give feedback
about your experience with Firebase AI Logic](https://firebase.google.com/docs/ai-logic/feedback)

<br />