使用 Imagen 替换图片的背景


本页介绍了如何使用 Imagen 通过 替换图片的背景 使用 Firebase AI Logic SDK。

背景替换是一种 基于蒙版的修改（具体而言是 修复）。蒙版是一种数字叠加层，用于定义您要修改的特定区域。

工作原理：您提供原始图片和相应的蒙版图片，该蒙版图片定义了背景上的蒙版，您可以使用自动背景检测功能，也可以自行提供背景蒙版。您还可以提供文本提示，描述您想要更改的内容。 然后，模型会生成并应用新的背景。

例如，您可以更改正文或对象周围的设置，而不会 影响前景（例如，在产品图片中）。

跳转到自动检测背景的代码 跳转到提供背景蒙版的代码

准备工作

仅当使用 Agent Platform Gemini API (formerly Vertex AI) 作为您的 API 提供方时可用。

如果您尚未完成 入门指南，请先完成该指南。该指南介绍了如何设置 Firebase 项目、将应用连接到 Firebase、 添加 SDK、为所选的 API 提供方初始化后端服务，以及 创建 ImagenModel 实例。

支持此功能的模型

Imagen 通过其 capability 模型提供图片修改功能：

  • imagen-3.0-capability-001

请注意，对于 Imagen 模型，global 位置 不支持。

使用自动背景检测功能替换背景

在尝试此示例之前，请先完成本指南的 准备工作部分， 以设置您的项目和应用。

以下示例展示了如何使用自动背景检测功能替换图片的背景。 您提供原始图片和文本提示，Imagen 会自动检测并创建背景蒙版，以修改 原始图片。

Swift

使用 Imagen 模型进行图片修改不支持 Swift。请在今年晚些时候回来查看！

Kotlin

如需使用自动背景检测功能替换背景，请指定 ImagenBackgroundMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.INPAINT_INSERTION。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // Provide the prompt describing the new background.
    val prompt = "space background"

    // Use the editImage API to replace the background.
    // Pass the original image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and background replacement.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_INSERTION)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如需使用自动背景检测功能替换背景，请指定 ImagenBackgroundMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.INPAINT_INSERTION。

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your image Bitmap here");

// Provide the prompt describing the new background.
String prompt = "space background";

// Define the list of sources for the editImage call.
// This includes the original image and the auto-generated mask.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
// Use ImagenBackgroundMask() to auto-generate the mask.
ImagenBackgroundMask rawMaskedImage = new ImagenBackgroundMask();

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to replace the background.
// Pass the original image, the auto-generated masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

Imagen 模型不支持使用 Web 应用进行图片修改。请在今年晚些时候回来查看！

Dart

如需使用自动背景检测功能替换背景，请指定 ImagenBackgroundMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.inpaintInsertion。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// Provide the prompt describing the new background.
final prompt = 'space background';

try {
  // Use the editImage API to replace the background.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and background replacement.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintInsertion,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Imagen 模型不支持使用 Unity 进行图片修改。请在今年晚些时候回来查看！

使用提供的蒙版替换背景

在尝试此示例之前，请先完成本指南的 准备工作部分， 以设置您的项目和应用。

以下示例展示了如何使用您提供的图片中定义的背景蒙版替换图片的背景。 您提供原始图片、文本提示和蒙版图片。

Swift

使用 Imagen 模型进行图片修改不支持 Swift。请在今年晚些时候回来查看！

Kotlin

如需使用您提供的蒙版替换背景，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.INPAINT_INSERTION。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
    // In a real app, this might come from the user's device or a URL.
    val maskImage: Bitmap = TODO("Load your masked image Bitmap here")

    // Provide the prompt describing the new background.
    val prompt = "space background"

    // Use the editImage API to replace the background.
    // Pass the original image, the masked image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenRawMask(maskImage.toImagenInlineImage()), // Use ImagenRawMask() to provide your own masked image.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and background replacement.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_INSERTION)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如需使用您提供的蒙版替换背景，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.INPAINT_INSERTION。

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your original image Bitmap here");

// This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
// In a real app, this might come from the user's device or a URL.
Bitmap maskImage = null; // TODO("Load your masked image Bitmap here");

// Provide the prompt describing the new background.
String prompt = "space background";

// Define the list of source images for the editImage call.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
// Use ImagenRawMask() to provide your own masked image.
ImagenBackgroundMask rawMaskedImage =
    new ImagenRawMask(ImagenInlineImageKt.toImagenInlineImage(maskImage));

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to replace the background.
// Pass the original image, the masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

Imagen 模型不支持使用 Web 应用进行图片修改。请在今年晚些时候回来查看！

Dart

如需使用您提供的蒙版替换背景，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将修改配置设置为使用 ImagenEditMode.INPAINT_INSERTION。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// This example assumes 'maskImage' is a pre-loaded Uint8List that contains the masked area.
// In a real app, this might come from the user's device or a URL.
final Uint8List maskImage = Uint8List(0); // TODO: Load your masked image data here.

// Provide the prompt describing the new background.
final prompt = 'space background';

try {
  // Use the editImage API to replace the background.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenRawMask(maskImage), // Use ImagenRawMask() to provide your own masked image.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and background replacement.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintInsertion,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Imagen 模型不支持使用 Unity 进行图片修改。请在今年晚些时候回来查看！

最佳实践和限制

我们建议您在修改图片时扩大蒙版。这有助于平滑 修改的边界，使其看起来更逼真。一般来说，建议使用 1% 或 2% 的扩大值（0.01 或 0.02）。


提供反馈 有关您的使用体验Firebase AI Logic