在 Android 上使用 Firebase ML 辨識地標

你可以使用 Firebase ML 辨識圖片中的知名地標。

事前準備

如果您尚未將 Firebase 新增至 Android 專案，請先新增。
在模組 (應用程式層級) Gradle 檔案 (通常是 <project>/<app-module>/build.gradle.kts 或 <project>/<app-module>/build.gradle) 中，加入 Android 適用的 Firebase ML Vision 程式庫依附元件。建議使用 Firebase Android BoM 控制程式庫版本。
```
dependencies {
    // Import the BoM for the Firebase platform
    implementation(platform("com.google.firebase:firebase-bom:34.10.0"))

    // Add the dependency for the Firebase ML Vision library
    // When using the BoM, you don't specify versions in Firebase library dependencies
    implementation 'com.google.firebase:firebase-ml-vision'
}
```
只要使用 Firebase Android BoM，應用程式就會一律使用相容的 Firebase Android 程式庫版本。
(替代做法) 不使用 BoM 新增 Firebase 程式庫依附元件

如果選擇不使用 Firebase BoM，則必須在依附元件行中指定每個 Firebase 程式庫版本。

請注意，如果應用程式使用多個 Firebase 程式庫，強烈建議使用 BoM 管理程式庫版本，確保所有版本都相容。
```
dependencies {
    // Add the dependency for the Firebase ML Vision library
    // When NOT using the BoM, you must specify versions in Firebase library dependencies
    implementation 'com.google.firebase:firebase-ml-vision:24.1.0'
}
```
如果尚未為專案啟用雲端 API，請立即啟用：
1. 在 Firebase 控制台中開啟「APIs」(API) Firebase ML頁面。
2. 如果尚未將專案升級至即付即用 Blaze 定價方案，請按一下「升級」。只有在專案未採用 Blaze 定價方案時，系統才會提示您升級。
  
  只有採用 Blaze 定價方案的專案才能使用雲端 API。
3. 如果尚未啟用雲端 API，請按一下「啟用雲端 API」。
在將使用 Cloud API 的應用程式部署至正式環境之前，請先採取幾個額外步驟，防範未經授權的 API 存取活動，並減輕其影響。

設定地標偵測器

根據預設，Cloud 偵測器會使用 STABLE 版模型，最多傳回 10 項結果。如要變更這些設定，請使用 FirebaseVisionCloudDetectorOptions 物件指定設定。

舉例來說，如要變更這兩項預設設定，請建構 FirebaseVisionCloudDetectorOptions 物件，如下列範例所示：

Kotlin

val options = FirebaseVisionCloudDetectorOptions.Builder()
    .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
    .setMaxResults(15)
    .build()MainActivity.kt

Java

FirebaseVisionCloudDetectorOptions options =
        new FirebaseVisionCloudDetectorOptions.Builder()
                .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
                .setMaxResults(15)
                .build();MainActivity.java

如要使用預設設定，請在下一個步驟使用 FirebaseVisionCloudDetectorOptions.DEFAULT。

執行地標偵測器

如要辨識圖片中的地標，請從 Bitmap、media.Image、ByteBuffer、位元組陣列或裝置上的檔案建立 FirebaseVisionImage 物件。然後，將 FirebaseVisionImage 物件傳遞至 FirebaseVisionCloudLandmarkDetector 的 detectInImage 方法。

從圖片建立 FirebaseVisionImage 物件。

如要從 media.Image 物件建立 FirebaseVisionImage 物件 (例如從裝置的相機擷取圖片時)，請將 media.Image 物件和圖片的旋轉角度傳遞至 FirebaseVisionImage.fromMediaImage()。

如果您使用 CameraX 程式庫，OnImageCapturedListener 和 ImageAnalysis.Analyzer 類別會為您計算旋轉值，因此您只需在呼叫 FirebaseVisionImage.fromMediaImage() 前，將旋轉值轉換為 Firebase ML 的 ROTATION_ 常數之一：

Kotlin

private class YourImageAnalyzer : ImageAnalysis.Analyzer {
    private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
        0 -> FirebaseVisionImageMetadata.ROTATION_0
        90 -> FirebaseVisionImageMetadata.ROTATION_90
        180 -> FirebaseVisionImageMetadata.ROTATION_180
        270 -> FirebaseVisionImageMetadata.ROTATION_270
        else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
    }

    override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
        val mediaImage = imageProxy?.image
        val imageRotation = degreesToFirebaseRotation(degrees)
        if (mediaImage != null) {
            val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
            // Pass image to an ML Vision API
            // ...
        }
    }
}

Java

private class YourAnalyzer implements ImageAnalysis.Analyzer {

    private int degreesToFirebaseRotation(int degrees) {
        switch (degrees) {
            case 0:
                return FirebaseVisionImageMetadata.ROTATION_0;
            case 90:
                return FirebaseVisionImageMetadata.ROTATION_90;
            case 180:
                return FirebaseVisionImageMetadata.ROTATION_180;
            case 270:
                return FirebaseVisionImageMetadata.ROTATION_270;
            default:
                throw new IllegalArgumentException(
                        "Rotation must be 0, 90, 180, or 270.");
        }
    }

    @Override
    public void analyze(ImageProxy imageProxy, int degrees) {
        if (imageProxy == null || imageProxy.getImage() == null) {
            return;
        }
        Image mediaImage = imageProxy.getImage();
        int rotation = degreesToFirebaseRotation(degrees);
        FirebaseVisionImage image =
                FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
        // Pass image to an ML Vision API
        // ...
    }
}

如果您使用的相機程式庫未提供圖片的旋轉角度，可以根據裝置的旋轉角度和裝置中相機感應器的方向計算：

Kotlin

private val ORIENTATIONS = SparseIntArray()

init {
    ORIENTATIONS.append(Surface.ROTATION_0, 90)
    ORIENTATIONS.append(Surface.ROTATION_90, 0)
    ORIENTATIONS.append(Surface.ROTATION_180, 270)
    ORIENTATIONS.append(Surface.ROTATION_270, 180)
}
/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
@Throws(CameraAccessException::class)
private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    val deviceRotation = activity.windowManager.defaultDisplay.rotation
    var rotationCompensation = ORIENTATIONS.get(deviceRotation)

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
    val sensorOrientation = cameraManager
        .getCameraCharacteristics(cameraId)
        .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360

    // Return the corresponding FirebaseVisionImageMetadata rotation value.
    val result: Int
    when (rotationCompensation) {
        0 -> result = FirebaseVisionImageMetadata.ROTATION_0
        90 -> result = FirebaseVisionImageMetadata.ROTATION_90
        180 -> result = FirebaseVisionImageMetadata.ROTATION_180
        270 -> result = FirebaseVisionImageMetadata.ROTATION_270
        else -> {
            result = FirebaseVisionImageMetadata.ROTATION_0
            Log.e(TAG, "Bad rotation value: $rotationCompensation")
        }
    }
    return result
}VisionImage.kt

Java

private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
static {
    ORIENTATIONS.append(Surface.ROTATION_0, 90);
    ORIENTATIONS.append(Surface.ROTATION_90, 0);
    ORIENTATIONS.append(Surface.ROTATION_180, 270);
    ORIENTATIONS.append(Surface.ROTATION_270, 180);
}

/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private int getRotationCompensation(String cameraId, Activity activity, Context context)
        throws CameraAccessException {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int rotationCompensation = ORIENTATIONS.get(deviceRotation);

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
    int sensorOrientation = cameraManager
            .getCameraCharacteristics(cameraId)
            .get(CameraCharacteristics.SENSOR_ORIENTATION);
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

    // Return the corresponding FirebaseVisionImageMetadata rotation value.
    int result;
    switch (rotationCompensation) {
        case 0:
            result = FirebaseVisionImageMetadata.ROTATION_0;
            break;
        case 90:
            result = FirebaseVisionImageMetadata.ROTATION_90;
            break;
        case 180:
            result = FirebaseVisionImageMetadata.ROTATION_180;
            break;
        case 270:
            result = FirebaseVisionImageMetadata.ROTATION_270;
            break;
        default:
            result = FirebaseVisionImageMetadata.ROTATION_0;
            Log.e(TAG, "Bad rotation value: " + rotationCompensation);
    }
    return result;
}VisionImage.java

接著，將 media.Image 物件和旋轉值傳遞至 FirebaseVisionImage.fromMediaImage()：

Kotlin

val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)VisionImage.kt

Java

FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);VisionImage.java

如要從檔案 URI 建立 FirebaseVisionImage 物件，請將應用程式內容和檔案 URI 傳遞至 FirebaseVisionImage.fromFilePath()。當您使用 ACTION_GET_CONTENT 意圖提示使用者從相簿應用程式選取圖片時，這項功能就非常實用。

Kotlin

val image: FirebaseVisionImage
try {
    image = FirebaseVisionImage.fromFilePath(context, uri)
} catch (e: IOException) {
    e.printStackTrace()
}VisionImage.kt

Java

FirebaseVisionImage image;
try {
    image = FirebaseVisionImage.fromFilePath(context, uri);
} catch (IOException e) {
    e.printStackTrace();
}VisionImage.java

如要從 ByteBuffer 或位元組陣列建立 FirebaseVisionImage 物件，請先計算圖片旋轉角度，如上文所述，以做為 media.Image 輸入內容。

接著，建立包含圖片高度、寬度、色彩編碼格式和旋轉角度的 FirebaseVisionImageMetadata 物件：

Kotlin

val metadata = FirebaseVisionImageMetadata.Builder()
    .setWidth(480) // 480x360 is typically sufficient for
    .setHeight(360) // image recognition
    .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
    .setRotation(rotation)
    .build()VisionImage.kt

Java

FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
        .setWidth(480)   // 480x360 is typically sufficient for
        .setHeight(360)  // image recognition
        .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
        .setRotation(rotation)
        .build();VisionImage.java

使用緩衝區或陣列和中繼資料物件，建立 FirebaseVisionImage 物件：

Kotlin

val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
// Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)VisionImage.kt

Java

FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
// Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);VisionImage.java

如要從 Bitmap 物件建立 FirebaseVisionImage 物件：

Kotlin

val image = FirebaseVisionImage.fromBitmap(bitmap)VisionImage.kt

Java

FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);VisionImage.java

Bitmap 物件代表的圖片必須直立，不需額外旋轉。

取得 FirebaseVisionCloudLandmarkDetector 的例項：

Kotlin

val detector = FirebaseVision.getInstance()
    .visionCloudLandmarkDetector
// Or, to change the default settings:
// val detector = FirebaseVision.getInstance()
//         .getVisionCloudLandmarkDetector(options)LandmarkRecognitionActivity.kt

Java

FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
        .getVisionCloudLandmarkDetector();
// Or, to change the default settings:
// FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
//         .getVisionCloudLandmarkDetector(options);LandmarkRecognitionActivity.java

最後，將圖片傳遞至 detectInImage 方法：

Kotlin

val result = detector.detectInImage(image)
    .addOnSuccessListener { firebaseVisionCloudLandmarks ->
        // Task completed successfully
        // ...
    }
    .addOnFailureListener { e ->
        // Task failed with an exception
        // ...
    }LandmarkRecognitionActivity.kt

Java

Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image)
        .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() {
            @Override
            public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) {
                // Task completed successfully
                // ...
            }
        })
        .addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                // Task failed with an exception
                // ...
            }
        });LandmarkRecognitionActivity.java

取得辨識出的地標相關資訊

如果地標辨識作業成功，系統會將 FirebaseVisionCloudLandmark 物件清單傳遞至成功監聽器。每個 FirebaseVisionCloudLandmark 物件都代表圖片中辨識到的地標。針對每個地標，您可以取得輸入圖片中的地標邊界座標、地標名稱、經緯度、知識圖譜實體 ID (如有) 和比對的信賴分數。例如：

Kotlin

for (landmark in firebaseVisionCloudLandmarks) {
    val bounds = landmark.boundingBox
    val landmarkName = landmark.landmark
    val entityId = landmark.entityId
    val confidence = landmark.confidence

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (loc in landmark.locations) {
        val latitude = loc.latitude
        val longitude = loc.longitude
    }
}LandmarkRecognitionActivity.kt

Java

for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) {

    Rect bounds = landmark.getBoundingBox();
    String landmarkName = landmark.getLandmark();
    String entityId = landmark.getEntityId();
    float confidence = landmark.getConfidence();

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (FirebaseVisionLatLng loc: landmark.getLocations()) {
        double latitude = loc.getLatitude();
        double longitude = loc.getLongitude();
    }
}LandmarkRecognitionActivity.java

後續步驟

在正式環境中部署使用 Cloud API 的應用程式之前，請先採取幾個額外步驟，防範未經授權的 API 存取活動，並減輕其影響。

在 Android 上使用 Firebase ML 辨識地標 透過集合功能整理內容 你可以依據偏好儲存及分類內容。

事前準備

設定地標偵測器

Kotlin

Java

執行地標偵測器

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

Kotlin

Java

取得辨識出的地標相關資訊

Kotlin

Java

後續步驟

在 Android 上使用 Firebase ML 辨識地標