转到控制台

在 iOS 上使用机器学习套件识别地标

您可以使用机器学习套件来识别图片中的知名地标。

如需了解此 API 的实际应用示例,请查看 GitHub 上的机器学习套件快速入门示例

准备工作

  1. 如果您尚未将 Firebase 添加到自己的应用中,请按照入门指南中的步骤执行此操作。
  2. 在 Podfile 中添加机器学习套件库:
    pod 'Firebase/Core'
    pod 'Firebase/MLVision'
    
    安装或更新项目的 Pod 之后,请务必使用 Xcode 项目的 .xcworkspace 来打开项目。
  3. 在您的应用中导入 Firebase:

    Swift

    import Firebase

    Objective-C

    @import Firebase;
  4. 如果您尚未为项目启用云端 API,请立即完成以下操作:

    1. 打开 Firebase 控制台的机器学习套件 API 页面
    2. 如果您尚未将项目升级到 Blaze 方案,请点击升级以执行此操作。(只有在您的项目未采用 Blaze 方案中时,系统才会提示您进行升级。)

      只有 Blaze 级项目才能使用云端 API。

    3. 如果尚未启用云端 API,请点击启用云端 API

配置地标检测器

默认情况下,Cloud 检测器使用稳定版模型并最多返回 10 个结果。如果您想更改这两个设置中的任何一个,请按照以下示例使用 VisionCloudDetectorOptions 对象进行指定:

Swift

let options = VisionCloudDetectorOptions()
options.modelType = .latest
options.maxResults = 20

Objective-C

  FIRVisionCloudDetectorOptions *options =
      [[FIRVisionCloudDetectorOptions alloc] init];
  options.modelType = FIRVisionCloudModelTypeLatest;
  options.maxResults = 20;
  

在下一步中,在创建 Cloud 检测器对象时传递 VisionCloudDetectorOptions 对象。

运行地标检测器

要识别图片中的地标,请将图片作为 UIImageCMSampleBufferRef 传递给 VisionCloudLandmarkDetectordetect(in:) 方法:

  1. 获取 VisionCloudLandmarkDetector 的一个实例:

    Swift

    lazy var vision = Vision.vision()let cloudDetector = vision.cloudLandmarkDetector(options: options)
    // Or, to use the default settings:
    // let cloudDetector = vision.cloudLandmarkDetector()
    

    Objective-C

    FIRVision *vision = [FIRVision vision];
    FIRVisionCloudLandmarkDetector *landmarkDetector = [vision cloudLandmarkDetector];
    // Or, to change the default settings:
    // FIRVisionCloudLandmarkDetector *landmarkDetector =
    //     [vision cloudLandmarkDetectorWithOptions:options];
    
  2. 使用 UIImageCMSampleBufferRef 创建一个 VisionImage 对象。

    要使用 UIImage,请按以下步骤操作:

    1. 在必要时旋转图片,以使其 imageOrientation 属性为 .up
    2. 使用方向正确的 UIImage 创建一个 VisionImage 对象。不要指定任何旋转元数据,必须使用默认值 .topLeft

      Swift

      let image = VisionImage(image: uiImage)

      Objective-C

      FIRVisionImage *image = [[FIRVisionImage alloc] initWithImage:uiImage];
      

    要使用 CMSampleBufferRef,请按以下步骤操作:

    1. 创建一个 VisionImageMetadata 对象,用其指定 CMSampleBufferRef 缓冲区中所含图片数据的方向。

      例如,如果您使用的是从设备的后置摄像头捕获的图片数据:

      Swift

      let metadata = VisionImageMetadata()
      
      // Using back-facing camera
      let devicePosition: AVCaptureDevice.Position = .back
      
      let deviceOrientation = UIDevice.current.orientation
      switch deviceOrientation {
      case .portrait:
        metadata.orientation = devicePosition == .front ? .leftTop : .rightTop
      case .landscapeLeft:
        metadata.orientation = devicePosition == .front ? .bottomLeft : .topLeft
      case .portraitUpsideDown:
        metadata.orientation = devicePosition == .front ? .rightBottom : .leftBottom
      case .landscapeRight:
        metadata.orientation = devicePosition == .front ? .topRight : .bottomRight
      case .faceDown, .faceUp, .unknown:
        metadata.orientation = .leftTop
      }
      

      Objective-C

      // Calculate the image orientation
      FIRVisionDetectorImageOrientation orientation;
      
      // Using front-facing camera
      AVCaptureDevicePosition devicePosition = AVCaptureDevicePositionFront;
      
      UIDeviceOrientation deviceOrientation = UIDevice.currentDevice.orientation;
      switch (deviceOrientation) {
          case UIDeviceOrientationPortrait:
              if (devicePosition == AVCaptureDevicePositionFront) {
                  orientation = FIRVisionDetectorImageOrientationLeftTop;
              } else {
                  orientation = FIRVisionDetectorImageOrientationRightTop;
              }
              break;
          case UIDeviceOrientationLandscapeLeft:
              if (devicePosition == AVCaptureDevicePositionFront) {
                  orientation = FIRVisionDetectorImageOrientationBottomLeft;
              } else {
                  orientation = FIRVisionDetectorImageOrientationTopLeft;
              }
              break;
          case UIDeviceOrientationPortraitUpsideDown:
              if (devicePosition == AVCaptureDevicePositionFront) {
                  orientation = FIRVisionDetectorImageOrientationRightBottom;
              } else {
                  orientation = FIRVisionDetectorImageOrientationLeftBottom;
              }
              break;
          case UIDeviceOrientationLandscapeRight:
              if (devicePosition == AVCaptureDevicePositionFront) {
                  orientation = FIRVisionDetectorImageOrientationTopRight;
              } else {
                  orientation = FIRVisionDetectorImageOrientationBottomRight;
              }
              break;
          default:
              orientation = FIRVisionDetectorImageOrientationTopLeft;
              break;
      }
      
      FIRVisionImageMetadata *metadata = [[FIRVisionImageMetadata alloc] init];
      metadata.orientation = orientation;
      
    2. 使用 CMSampleBufferRef 对象和旋转元数据创建一个 VisionImage 对象:

      Swift

      let image = VisionImage(buffer: bufferRef)
      image.metadata = metadata
      

      Objective-C

      FIRVisionImage *image = [[FIRVisionImage alloc] initWithBuffer:buffer];
      image.metadata = metadata;
      
  3. 然后,将图片传递给 detect(in:) 方法:

    Swift

    cloudDetector.detect(in: visionImage) { landmarks, error in
      guard error == nil, let landmarks = landmarks, !landmarks.isEmpty else {
        // ...
        return
      }
    
      // Recognized landmarks
      // ...
    }
    

    Objective-C

    [landmarkDetector detectInImage:image
                         completion:^(NSArray<FIRVisionCloudLandmark *> *landmarks,
                                      NSError *error) {
      if (error != nil) {
        return;
      } else if (landmarks != nil) {
        // Got landmarks
      }
    }];
    

获取识别出的地标的相关信息

如果成功识别出了地标,则系统会向完成处理程序传递一组 VisionCloudLandmark 对象。从每个对象中,您可以获取图片中识别出的地标的相关信息。

例如:

Swift

for landmark in landmarks {
  let landmarkDesc = landmark.landmark
  let boundingPoly = landmark.frame
  let entityId = landmark.entityId

  // A landmark can have multiple locations: for example, the location the image
  // was taken, and the location of the landmark depicted.
  for location in landmark.locations {
    let latitude = location.latitude
    let longitude = location.longitude
  }

  let confidence = landmark.confidence
}

Objective-C

for (FIRVisionCloudLandmark *landmark in landmarks) {
   NSString *landmarkDesc = landmark.landmark;
   CGRect frame = landmark.frame;
   NSString *entityId = landmark.entityId;

   // A landmark can have multiple locations: for example, the location the image
   // was taken, and the location of the landmark depicted.
   for (FIRVisionLatitudeLongitude *location in landmark.locations) {
     double latitude = [location.latitude doubleValue];
     double longitude = [location.longitude doubleValue];
   }

   float confidence = [landmark.confidence floatValue];
}

后续步骤

在向生产环境中部署使用 Cloud API 的应用之前,您应该采取一些额外步骤来防止未经授权的 API 访问并减轻其造成的影响