使用 Imagen 从图片中移除对象


本页介绍了如何使用 Firebase AI Logic SDK 通过 Imagen 进行图像修复，以从图像中移除对象。

修补是一种基于蒙版的修改。蒙版是一种数字叠加层，用于定义您要修改的特定区域。

运作方式：您提供一张原始图片和一张相应的蒙版图片（自动生成或由您提供），其中定义了要移除的对象或主题的蒙版。您还可以选择提供文本提示来描述要移除的内容，或者让模型智能检测要移除的对象。然后，模型会移除对象，并使用新的、符合上下文的内容填充该区域。

例如，您可以遮盖一个球，并将其替换为空白墙壁或草地。

跳转到自动生成的遮罩的代码 跳转到提供遮罩的代码

准备工作

仅在将 Vertex AI Gemini API 用作 API 提供方时可用。

如果您尚未完成入门指南，请先完成该指南。该指南介绍了如何设置 Firebase 项目、将应用连接到 Firebase、添加 SDK、为所选的 API 提供程序初始化后端服务，以及创建 ImagenModel 实例。

支持此功能的模型

Imagen 通过其 capability 模型提供图片编辑功能：

  • imagen-3.0-capability-001

请注意，对于 Imagen 模型，global 位置不受支持。

使用自动生成的蒙版移除对象

在试用此示例之前，请完成本指南的准备工作部分，以设置您的项目和应用。

以下示例展示了如何使用修复功能从图片中移除内容，并使用自动蒙版生成功能。 您提供原始图片和文本提示，Imagen 会自动检测并创建蒙版区域来修改原始图片。

Swift

Swift 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

Kotlin

如需移除具有自动生成的遮罩的对象，请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Vertex AI Gemini API backend service
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // Provide the prompt describing the content to be removed.
    val prompt = "ball"

    // Use the editImage API to remove the unwanted content.
    // Pass the original image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and insertion.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如需移除具有自动生成的遮罩的对象，请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。

// Initialize the Vertex AI Gemini API backend service
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your image Bitmap here");

// Provide the prompt describing the content to be removed.
String prompt = "ball";

// Define the list of sources for the editImage call.
// This includes the original image and the auto-generated mask.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
// Use ImagenBackgroundMask() to auto-generate the mask.
ImagenBackgroundMask rawMaskedImage = new ImagenBackgroundMask();

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to remove the unwanted content.
// Pass the original image, the auto-generated masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

Web 应用不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

Dart

如需移除具有自动生成的遮罩的对象，请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.inpaintRemoval。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Vertex AI Gemini API backend service
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

TODO - FLUTTER// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// Provide the prompt describing the content to be removed.
final prompt = 'ball';

try {
  // Use the editImage API to remove the unwanted content.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenBackgroundMask(),  // Use ImagenBackgroundMask() to auto-generate the mask.
    ],
    prompt,
    // Define the editing configuration for inpainting and removal.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintRemoval,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

使用提供的遮罩移除对象

在试用此示例之前，请完成本指南的准备工作部分，以设置您的项目和应用。

以下示例展示了如何使用修复功能从图片中移除内容，方法是使用您提供的图片中定义的蒙版。您提供原始图片、文本提示和蒙版图片。

如果您提供遮盖图片，则可以选择是否提供文本提示。 Imagen 可以智能检测要从遮罩区域中移除的对象。不过，如果您要移除的对象不明显，或者您只想移除蒙版区域中的特定对象，请提供文本提示，帮助模型移除正确的对象。

Swift

Swift 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

Kotlin

如需移除对象并提供您自己的蒙版图片，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Vertex AI Gemini API backend service
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
    // In a real app, this might come from the user's device or a URL.
    val maskImage: Bitmap = TODO("Load your masked image Bitmap here")

    // Provide the prompt describing the content to be removed.
    val prompt = "ball"

    // Use the editImage API to remove the unwanted content.
    // Pass the original image, the masked image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenRawMask(maskImage.toImagenInlineImage()), // Use ImagenRawMask() to provide your own masked image.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and removal.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如需移除对象并提供您自己的蒙版图片，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。

// Initialize the Vertex AI Gemini API backend service
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your original image Bitmap here");

// This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
// In a real app, this might come from the user's device or a URL.
Bitmap maskImage = null; // TODO("Load your masked image Bitmap here");

// Provide the prompt describing the content to be removed.
String prompt = "ball";

// Define the list of source images for the editImage call.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
 // Use ImagenRawMask() to provide your own masked image.
ImagenBackgroundMask rawMaskedImage =
    new ImagenRawMask(ImagenInlineImageKt.toImagenInlineImage(maskImage));

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to remove the unwanted content.
// Pass the original image, the masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

Web 应用不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

Dart

如需移除对象并提供您自己的蒙版图片，请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.inpaintRemoval。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Vertex AI Gemini API backend service
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// This example assumes 'maskImage' is a pre-loaded Uint8List that contains the masked area.
// In a real app, this might come from the user's device or a URL.
final Uint8List maskImage = Uint8List(0); // TODO: Load your masked image data here.

// Provide the prompt describing the content to be removed.
final prompt = 'ball';

try {
  // Use the editImage API to remove the unwanted content.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenRawMask(maskImage), // Use ImagenRawMask() to provide your own masked image.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and removal.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintRemoval,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看！

最佳做法和限制

建议您在编辑图片时扩大遮罩。这有助于平滑编辑的边界，使其看起来更逼真。一般来说，建议将扩张值设置为 1% 或 2%（0.01 或 0.02）。


就您使用 Firebase AI Logic 的体验提供反馈