Join us in person and online for Firebase Summit on October 18, 2022. Learn how Firebase can help you accelerate app development, release your app with confidence, and scale with ease. Register now

在 Android 上使用 Firebase ML 識別地標

透過集合功能整理內容 你可以依據偏好儲存及分類內容。

您可以使用 Firebase ML 識別圖像中的知名地標。

在你開始之前

  1. 如果您還沒有,請將 Firebase 添加到您的 Android 項目中。
  2. 使用Firebase Android BoM ,在您的模塊(應用級)Gradle 文件(通常是app/build.gradle )中聲明 Firebase ML Vision Android 庫的依賴項。
    dependencies {
        // Import the BoM for the Firebase platform
        implementation platform('com.google.firebase:firebase-bom:30.3.1')
    
        // Declare the dependency for the Firebase ML Vision library
        // When using the BoM, you don't specify versions in Firebase library dependencies
        implementation 'com.google.firebase:firebase-ml-vision'
    }
    

    通過使用Firebase Android BoM ,您的應用將始終使用兼容版本的 Firebase Android 庫。

    (替代)使用 BoM 的情況下聲明 Firebase 庫依賴項

    如果您選擇不使用 Firebase BoM,則必須在其依賴行中指定每個 Firebase 庫版本。

    請注意,如果您在應用中使用多個Firebase 庫,我們強烈建議您使用 BoM 來管理庫版本,以確保所有版本都兼容。

    dependencies {
        // Declare the dependency for the Firebase ML Vision library
        // When NOT using the BoM, you must specify versions in Firebase library dependencies
        implementation 'com.google.firebase:firebase-ml-vision:24.1.0'
    }
    
  3. 如果您尚未為您的項目啟用基於雲的 API,請立即執行此操作:

    1. 打開 Firebase 控制台的Firebase ML API 頁面
    2. 如果您尚未將項目升級到 Blaze 定價計劃,請單擊升級以執行此操作。 (僅當您的項目不在 Blaze 計劃中時,系統才會提示您升級。)

      只有 Blaze 級項目可以使用基於雲的 API。

    3. 如果尚未啟用基於雲的 API,請單擊啟用基於雲的 API

配置地標檢測器

默認情況下,雲檢測器使用模型的STABLE版本並返回最多 10 個結果。如果您想更改這些設置中的任何一個,請使用FirebaseVisionCloudDetectorOptions對象指定它們。

例如,要更改這兩個默認設置,請構建一個FirebaseVisionCloudDetectorOptions對象,如下例所示:

Java

FirebaseVisionCloudDetectorOptions options =
        new FirebaseVisionCloudDetectorOptions.Builder()
                .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
                .setMaxResults(15)
                .build();

Kotlin+KTX

val options = FirebaseVisionCloudDetectorOptions.Builder()
        .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
        .setMaxResults(15)
        .build()

要使用默認設置,您可以在下一步中使用FirebaseVisionCloudDetectorOptions.DEFAULT

運行地標檢測器

要識別圖像中的地標,請從Bitmapmedia.ImageByteBuffer 、字節數組或設備上的文件創建一個FirebaseVisionImage對象。然後,將FirebaseVisionImage對像傳遞給FirebaseVisionCloudLandmarkDetectordetectInImage方法。

  1. 從您的圖像創建一個FirebaseVisionImage對象。

    • 要從media.Image對象創建FirebaseVisionImage對象,例如從設備的攝像頭捕獲圖像時,請將media.Image對象和圖像的旋轉傳遞給FirebaseVisionImage.fromMediaImage()

      如果您使用CameraX庫,則OnImageCapturedListenerImageAnalysis.Analyzer類會為您計算旋轉值,因此您只需在調用FirebaseVisionImage.fromMediaImage()之前將旋轉轉換為 Firebase ML 的ROTATION_常量之一:

      Java

      private class YourAnalyzer implements ImageAnalysis.Analyzer {
      
          private int degreesToFirebaseRotation(int degrees) {
              switch (degrees) {
                  case 0:
                      return FirebaseVisionImageMetadata.ROTATION_0;
                  case 90:
                      return FirebaseVisionImageMetadata.ROTATION_90;
                  case 180:
                      return FirebaseVisionImageMetadata.ROTATION_180;
                  case 270:
                      return FirebaseVisionImageMetadata.ROTATION_270;
                  default:
                      throw new IllegalArgumentException(
                              "Rotation must be 0, 90, 180, or 270.");
              }
          }
      
          @Override
          public void analyze(ImageProxy imageProxy, int degrees) {
              if (imageProxy == null || imageProxy.getImage() == null) {
                  return;
              }
              Image mediaImage = imageProxy.getImage();
              int rotation = degreesToFirebaseRotation(degrees);
              FirebaseVisionImage image =
                      FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
              // Pass image to an ML Vision API
              // ...
          }
      }
      

      Kotlin+KTX

      private class YourImageAnalyzer : ImageAnalysis.Analyzer {
          private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
              0 -> FirebaseVisionImageMetadata.ROTATION_0
              90 -> FirebaseVisionImageMetadata.ROTATION_90
              180 -> FirebaseVisionImageMetadata.ROTATION_180
              270 -> FirebaseVisionImageMetadata.ROTATION_270
              else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
          }
      
          override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
              val mediaImage = imageProxy?.image
              val imageRotation = degreesToFirebaseRotation(degrees)
              if (mediaImage != null) {
                  val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
                  // Pass image to an ML Vision API
                  // ...
              }
          }
      }
      

      如果您不使用為您提供圖像旋轉的相機庫,您可以根據設備的旋轉和設備中相機傳感器的方向來計算它:

      Java

      private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
      static {
          ORIENTATIONS.append(Surface.ROTATION_0, 90);
          ORIENTATIONS.append(Surface.ROTATION_90, 0);
          ORIENTATIONS.append(Surface.ROTATION_180, 270);
          ORIENTATIONS.append(Surface.ROTATION_270, 180);
      }
      
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      private int getRotationCompensation(String cameraId, Activity activity, Context context)
              throws CameraAccessException {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
          int rotationCompensation = ORIENTATIONS.get(deviceRotation);
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
          int sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION);
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          int result;
          switch (rotationCompensation) {
              case 0:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  break;
              case 90:
                  result = FirebaseVisionImageMetadata.ROTATION_90;
                  break;
              case 180:
                  result = FirebaseVisionImageMetadata.ROTATION_180;
                  break;
              case 270:
                  result = FirebaseVisionImageMetadata.ROTATION_270;
                  break;
              default:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  Log.e(TAG, "Bad rotation value: " + rotationCompensation);
          }
          return result;
      }

      Kotlin+KTX

      private val ORIENTATIONS = SparseIntArray()
      
      init {
          ORIENTATIONS.append(Surface.ROTATION_0, 90)
          ORIENTATIONS.append(Surface.ROTATION_90, 0)
          ORIENTATIONS.append(Surface.ROTATION_180, 270)
          ORIENTATIONS.append(Surface.ROTATION_270, 180)
      }
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      @Throws(CameraAccessException::class)
      private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          val deviceRotation = activity.windowManager.defaultDisplay.rotation
          var rotationCompensation = ORIENTATIONS.get(deviceRotation)
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
          val sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          val result: Int
          when (rotationCompensation) {
              0 -> result = FirebaseVisionImageMetadata.ROTATION_0
              90 -> result = FirebaseVisionImageMetadata.ROTATION_90
              180 -> result = FirebaseVisionImageMetadata.ROTATION_180
              270 -> result = FirebaseVisionImageMetadata.ROTATION_270
              else -> {
                  result = FirebaseVisionImageMetadata.ROTATION_0
                  Log.e(TAG, "Bad rotation value: $rotationCompensation")
              }
          }
          return result
      }

      然後,將media.Image對象和旋轉值傳遞給FirebaseVisionImage.fromMediaImage()

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
    • 要從文件 URI 創建FirebaseVisionImage對象,請將應用上下文和文件 URI 傳遞給FirebaseVisionImage.fromFilePath() 。當您使用ACTION_GET_CONTENT意圖提示用戶從他們的圖庫應用中選擇圖像時,這很有用。

      Java

      FirebaseVisionImage image;
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri);
      } catch (IOException e) {
          e.printStackTrace();
      }

      Kotlin+KTX

      val image: FirebaseVisionImage
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri)
      } catch (e: IOException) {
          e.printStackTrace()
      }
    • 要從ByteBuffer或字節數組創建FirebaseVisionImage對象,請首先按照上面針對media.Image輸入的描述計算圖像旋轉。

      然後,創建一個FirebaseVisionImageMetadata對象,其中包含圖像的高度、寬度、顏色編碼格式和旋轉:

      Java

      FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
              .setWidth(480)   // 480x360 is typically sufficient for
              .setHeight(360)  // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build();

      Kotlin+KTX

      val metadata = FirebaseVisionImageMetadata.Builder()
              .setWidth(480) // 480x360 is typically sufficient for
              .setHeight(360) // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build()

      使用緩衝區或數組以及元數據對象來創建FirebaseVisionImage對象:

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
      // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
      // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
    • 要從Bitmap對象創建FirebaseVisionImage對象:

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromBitmap(bitmap)
      Bitmap對象表示的圖像必須是直立的,不需要額外的旋轉。

  2. 獲取FirebaseVisionCloudLandmarkDetector的實例:

    Java

    FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
            .getVisionCloudLandmarkDetector();
    // Or, to change the default settings:
    // FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
    //         .getVisionCloudLandmarkDetector(options);

    Kotlin+KTX

    val detector = FirebaseVision.getInstance()
            .visionCloudLandmarkDetector
    // Or, to change the default settings:
    // val detector = FirebaseVision.getInstance()
    //         .getVisionCloudLandmarkDetector(options)
  3. 最後,將圖像傳遞給detectInImage方法:

    Java

    Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image)
            .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() {
                @Override
                public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) {
                    // Task completed successfully
                    // ...
                }
            })
            .addOnFailureListener(new OnFailureListener() {
                @Override
                public void onFailure(@NonNull Exception e) {
                    // Task failed with an exception
                    // ...
                }
            });

    Kotlin+KTX

    val result = detector.detectInImage(image)
            .addOnSuccessListener { firebaseVisionCloudLandmarks ->
                // Task completed successfully
                // ...
            }
            .addOnFailureListener { e ->
                // Task failed with an exception
                // ...
            }

獲取有關已識別地標的信息

如果地標識別操作成功, FirebaseVisionCloudLandmark對象列表將傳遞給成功偵聽器。每個FirebaseVisionCloudLandmark對象代表在圖像中識別的地標。對於每個地標,您可以獲得其在輸入圖像中的邊界坐標、地標的名稱、其緯度和經度、其知識圖實體 ID (如果可用)以及匹配的置信度分數。例如:

Java

for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) {

    Rect bounds = landmark.getBoundingBox();
    String landmarkName = landmark.getLandmark();
    String entityId = landmark.getEntityId();
    float confidence = landmark.getConfidence();

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (FirebaseVisionLatLng loc: landmark.getLocations()) {
        double latitude = loc.getLatitude();
        double longitude = loc.getLongitude();
    }
}

Kotlin+KTX

for (landmark in firebaseVisionCloudLandmarks) {

    val bounds = landmark.boundingBox
    val landmarkName = landmark.landmark
    val entityId = landmark.entityId
    val confidence = landmark.confidence

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (loc in landmark.locations) {
        val latitude = loc.latitude
        val longitude = loc.longitude
    }
}

下一步