使用 Imagen 移除圖片中的物件


本頁說明如何使用 Imagen 進行修復，並使用 Firebase AI Logic SDK 從圖片中移除物件。

修復是以遮罩為基礎的編輯類型，遮罩是數位疊加層，可定義要編輯的特定區域。

運作方式：您提供原始圖片和相應的遮罩圖片 (系統自動生成或您提供)，定義要移除的物體或主體遮罩。你也可以選擇提供文字提示，說明要移除的內容，或是讓模型智慧偵測要移除的物件。模型隨即會移除物件，並以符合情境的新內容填補該區域。

舉例來說，你可以遮蓋一顆球，並將其替換成空白牆面或草地。

跳至自動生成遮罩的程式碼 跳至提供遮罩的程式碼

事前準備

只有在使用 Vertex AI Gemini API 做為 API 供應商時，才能使用這項功能。

如果尚未完成，請參閱入門指南，瞭解如何設定 Firebase 專案、將應用程式連結至 Firebase、新增 SDK、初始化所選 API 供應商的後端服務，以及建立 ImagenModel 執行個體。

支援這項功能的模型

Imagen 可透過 capability 模型編輯圖片：

  • imagen-3.0-capability-001

請注意，Imagen 型號global不支援位置資訊。

使用自動生成的遮罩移除物件

在試用這個範例之前，請先完成本指南的「事前準備」一節，設定專案和應用程式。

以下範例說明如何使用修復功能，透過自動產生遮罩，從圖片中移除內容。您提供原始圖片和文字提示詞，Imagen 就會自動偵測並建立遮罩區域，修改原始圖片。

Swift

Swift 不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

Kotlin

如要使用自動生成的遮罩移除物件，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_REMOVAL。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Vertex AI Gemini API backend service
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // Provide the prompt describing the content to be removed.
    val prompt = "ball"

    // Use the editImage API to remove the unwanted content.
    // Pass the original image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        sources = listOf(
            ImagenRawImage(originalImage),
            ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and insertion.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如要使用自動生成的遮罩移除物件，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_REMOVAL。

// Initialize the Vertex AI Gemini API backend service
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your image Bitmap here");

// Provide the prompt describing the content to be removed.
String prompt = "ball";

// Define the list of sources for the editImage call.
// This includes the original image and the auto-generated mask.
ImagenRawImage rawOriginalImage = new ImagenRawImage(originalImage);
ImagenBackgroundMask rawMaskedImage = new ImagenBackgroundMask();  // Use ImagenBackgroundMask() to auto-generate the mask.

// Define the editing configuration for inpainting and removal.
ImagenEditingConfig config = new ImagenEditingConfig.Builder()
        .setEditMode(ImagenEditMode.INPAINT_REMOVAL)
        .build();

// Use the editImage API to remove the unwanted content.
// Pass the original image, the auto-generated masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config), new FutureCallback<ImagenGenerationResponse>() {
    @Override
    public void onSuccess(ImagenGenerationResponse result) {
        if (result.getImages().isEmpty()) {
            Log.d("ImageEditor", "No images generated");
        }
        Bitmap editedImage = result.getImages().get(0).asBitmap();
        // Process and use the bitmap to display the image in your UI
    }

    @Override
    public void onFailure(Throwable t) {
        // ...
    }
}, Executors.newSingleThreadExecutor());

Web

網頁應用程式不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

Dart

如要使用自動生成的遮罩移除物件，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.inpaintRemoval。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Vertex AI Gemini API backend service
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

TODO - FLUTTER// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// Provide the prompt describing the content to be removed.
final prompt = 'ball';

try {
  // Use the editImage API to remove the unwanted content.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenBackgroundMask(),  // Use ImagenBackgroundMask() to auto-generate the mask.
    ],
    prompt,
    // Define the editing configuration for inpainting and removal.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintRemoval,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

使用提供的遮罩移除物件

在試用這個範例之前，請先完成本指南的「事前準備」一節，設定專案和應用程式。

以下範例說明如何使用修復功能，從圖片中移除內容，方法是使用您提供的圖片中定義的遮罩。您提供原始圖片、文字提示和遮罩圖片。

如果提供遮罩圖片，則文字提示為選填。 Imagen 可智慧偵測要從遮罩區域移除的物體。不過，如果想移除的物件不明顯，或只想移除遮罩區域中的特定物件，請提供文字提示，協助模型移除正確的物件。

Swift

Swift 不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

Kotlin

如要移除物件並提供自己的遮罩圖片，請使用遮罩圖片指定 ImagenRawMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_REMOVAL。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Vertex AI Gemini API backend service
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
    // In a real app, this might come from the user's device or a URL.
    val maskImage: Bitmap = TODO("Load your masked image Bitmap here")

    // Provide the prompt describing the content to be removed.
    val prompt = "ball"

    // Use the editImage API to remove the unwanted content.
    // Pass the original image, the masked image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenRawMask(maskImage.toImagenInlineImage()), // Use ImagenRawMask() to provide your own masked image.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and removal.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如要移除物件並提供自己的遮罩圖片，請使用遮罩圖片指定 ImagenRawMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_REMOVAL。

// Initialize the Vertex AI Gemini API backend service
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your original image Bitmap here");

// This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
// In a real app, this might come from the user's device or a URL.
Bitmap maskImage = null; // TODO("Load your masked image Bitmap here");

// Provide the prompt describing the content to be removed.
String prompt = "ball";

// Define the list of source images for the editImage call.
ImagenRawImage rawOriginalImage = new ImagenRawImage(originalImage);
ImagenBackgroundMask rawMaskedImage = new ImagenRawMask(maskImage); // Use ImagenRawMask() to provide your own masked image.

// Define the editing configuration for inpainting and removal.
ImagenEditingConfig config = new ImagenEditingConfig.Builder()
        .setEditMode(ImagenEditMode.INPAINT_REMOVAL)
        .build();

// Use the editImage API to remove the unwanted content.
// Pass the original image, the masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config), new FutureCallback<ImagenGenerationResponse>() {
    @Override
    public void onSuccess(ImagenGenerationResponse result) {
        if (result.getImages().isEmpty()) {
            Log.d("ImageEditor", "No images generated");
        }
        Bitmap editedImage = result.getImages().get(0).asBitmap();
        // Process and use the bitmap to display the image in your UI
    }

    @Override
    public void onFailure(Throwable t) {
        // ...
    }
}, Executors.newSingleThreadExecutor());

Web

網頁應用程式不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

Dart

如要移除物件並提供自己的遮罩圖片，請使用遮罩圖片指定 ImagenRawMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.inpaintRemoval。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Vertex AI Gemini API backend service
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// This example assumes 'maskImage' is a pre-loaded Uint8List that contains the masked area.
// In a real app, this might come from the user's device or a URL.
final Uint8List maskImage = Uint8List(0); // TODO: Load your masked image data here.

// Provide the prompt describing the content to be removed.
final prompt = 'ball';

try {
  // Use the editImage API to remove the unwanted content.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenRawMask(maskImage), // Use ImagenRawMask() to provide your own masked image.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and removal.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintRemoval,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支援使用 Imagen 模型編輯圖片。請於今年稍晚再回來查看！

最佳做法和限制

編輯圖片時，建議擴大遮罩範圍。這有助於平滑化編輯內容的邊界，讓編輯結果看起來更逼真。一般來說，建議使用 1% 或 2% 的擴張值 (0.01 或 0.02)。


提供有關 Firebase AI Logic 的使用體驗意見回饋