Recognize Landmarks with ML Kit on Android

You can use ML Kit to recognize well-known landmarks in an image.

See the ML Kit quickstart sample on GitHub for an example of this API in use.

Before you begin

  1. If you have not already added Firebase to your app, do so by following the steps in the getting started guide.
  2. Include the dependencies for ML Kit in your app-level build.gradle file:
    dependencies {
      // ...
      implementation ''
  3. If you have not already enabled Cloud-based APIs for your project, do so now:

    1. Open the ML Kit APIs page of the Firebase console.
    2. If you have not already upgraded your project to a Blaze plan, click Upgrade to do so. (You will be prompted to upgrade only if your project isn't on the Blaze plan.)

      Only Blaze-level projects can use Cloud-based APIs.

    3. If Cloud-based APIs aren't already enabled, click Enable Cloud-based APIs.

Configure the landmark detector

By default, the Cloud detector uses the STABLE version of the model and returns up to 10 results. If you want to change either of these settings, specify them with a FirebaseVisionCloudDetectorOptions object.

For example, to change both of the default settings, build a FirebaseVisionCloudDetectorOptions object as in the following example:

FirebaseVisionCloudDetectorOptions options =
        new FirebaseVisionCloudDetectorOptions.Builder()

To use the default settings, you can use FirebaseVisionCloudDetectorOptions.DEFAULT in the next step.

Run the landmark detector

To recognize landmarks in an image, create a FirebaseVisionImage object from either a Bitmap, media.Image, ByteBuffer, byte array, or a file on the device. Then, pass the FirebaseVisionImage object to the FirebaseVisionCloudLandmarkDetector's detectInImage method.

  1. Create a FirebaseVisionImage object from your image.

    • To create a FirebaseVisionImage object from a Bitmap object:
      FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
      The image represented by the Bitmap object must be upright, with no additional rotation required.
    • To create a FirebaseVisionImage object from a media.Image object, such as when capturing an image from a device's camera, first determine the angle the image must be rotated to compensate for both the device's rotation and the orientation of camera sensor in the device:
      private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
      static {
          ORIENTATIONS.append(Surface.ROTATION_0, 90);
          ORIENTATIONS.append(Surface.ROTATION_90, 0);
          ORIENTATIONS.append(Surface.ROTATION_180, 270);
          ORIENTATIONS.append(Surface.ROTATION_270, 180);
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      private int getRotationCompensation(String cameraId, Activity activity, Context context)
              throws CameraAccessException {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
          int rotationCompensation = ORIENTATIONS.get(deviceRotation);
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
          int sensorOrientation = cameraManager
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          int result;
          switch (rotationCompensation) {
              case 0:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
              case 90:
                  result = FirebaseVisionImageMetadata.ROTATION_90;
              case 180:
                  result = FirebaseVisionImageMetadata.ROTATION_180;
              case 270:
                  result = FirebaseVisionImageMetadata.ROTATION_270;
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  Log.e(TAG, "Bad rotation value: " + rotationCompensation);
          return result;

      Then, pass the media.Image object and the rotation value to FirebaseVisionImage.fromMediaImage():

      FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
    • To create a FirebaseVisionImage object from a ByteBuffer or a byte array, first calculate the image rotation as described above.

      Then, create a FirebaseVisionImageMetadata object that contains the image's height, width, color encoding format, and rotation:

      FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
              .setWidth(480)   // 480x360 is typically sufficient for
              .setHeight(360)  // image recognition

      Use the buffer or array, and the metadata object, to create a FirebaseVisionImage object:

      FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
      // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
    • To create a FirebaseVisionImage object from a file, pass the app context and file URI to FirebaseVisionImage.fromFilePath():
      FirebaseVisionImage image;
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri);
      } catch (IOException e) {

  2. Get an instance of FirebaseVisionCloudLandmarkDetector:

    FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
    // Or, to change the default settings:
    // FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
    //         .getVisionCloudLandmarkDetector(options);

  3. Finally, pass the image to the detectInImage method:

    Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image)
            .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() {
                public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) {
                    // Task completed successfully
                    // ...
            .addOnFailureListener(new OnFailureListener() {
                public void onFailure(@NonNull Exception e) {
                    // Task failed with an exception
                    // ...

Get information about the recognized landmarks

If the landmark recognition operation succeeds, a list of FirebaseVisionCloudLandmark objects will be passed to the success listener. Each FirebaseVisionCloudLandmark object represents a landmark that was recognized in the image. For each landmark, you can get its bounding coordinates in the input image, the landmark's name, its latitude and longitude, its Knowledge Graph entity ID (if available), and the confidence score of the match. For example:

for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) {

    Rect bounds = landmark.getBoundingBox();
    String landmarkName = landmark.getLandmark();
    String entityId = landmark.getEntityId();
    float confidence = landmark.getConfidence();

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (FirebaseVisionLatLng loc: landmark.getLocations()) {
        double latitude = loc.getLatitude();
        double longitude = loc.getLongitude();

Next steps

Before you deploy to production an app that uses a Cloud API, you should take some additional steps to prevent and mitigate the effect of unauthorized API access.

Оставить отзыв о...

Текущей странице
Нужна помощь? Обратитесь в службу поддержки.