Google is committed to advancing racial equity for Black communities. See how.
本頁面由 Cloud Translation API 翻譯而成。
Switch to English

在Android上使用ML Kit檢測人臉

{%setvar new_api_link%} 在Android上使用ML Kit檢測面孔 {%endsetvar%}

您可以使用ML Kit來檢測圖像和視頻中的面部。

在你開始之前

  1. 如果尚未將Firebase添加到您的Android項目中
  2. 在項目級別的build.gradle文件中,確保在buildscriptallprojects部分中都包含Google的Maven存儲庫。
  3. 將ML Kit Android庫的依賴項添加到模塊(應用程序級)Gradle文件(通常為app/build.gradle )中:
    apply plugin: 'com.android.application'
    apply plugin: 'com.google.gms.google-services'
    
    dependencies {
      // ...
    
      implementation 'com.google.firebase:firebase-ml-vision:24.0.3'
      // If you want to detect face contours (landmark detection and classification
      // don't require this additional model):
      implementation 'com.google.firebase:firebase-ml-vision-face-model:20.0.1'
    }
    
  4. 可選但推薦 :將應用配置為從Play商店安裝應用後自動將ML模型下載到設備上。

    為此,請將以下聲明添加到應用程序的AndroidManifest.xml文件中:

    <application ...>
      ...
      <meta-data
          android:name="com.google.firebase.ml.vision.DEPENDENCIES"
          android:value="face" />
      <!-- To use multiple models: android:value="face,model2,model3" -->
    </application>
    
    如果您未啟用安裝時模型下載,則將在您首次運行檢測器時下載模型。您在下載完成之前提出的請求將不會產生任何結果。

輸入圖像準則

為了使ML Kit準確檢測人臉,輸入圖像必須包含由足夠的像素數據表示的人臉。通常,您要在圖像中檢測到的每張臉至少應為100x100像素。如果要檢測臉部輪廓,則ML Kit需要更高分辨率的輸入:每個臉部至少應為200x200像素。

如果要在實時應用程序中檢測人臉,則可能還需要考慮輸入圖像的整體尺寸。較小的圖像可以更快地處理,因此可以減少延遲,以較低的分辨率捕獲圖像(請牢記上述精度要求),並確保對象的面部盡可能多地佔據圖像。另請參閱提高實時性能的提示

圖像聚焦不良會影響準確性。如果沒有得到滿意的結果,請嘗試要求用戶重新捕獲圖像。

臉部相對於相機的方向也會影響ML Kit檢測到的臉部特徵。請參閱面部檢測概念

1.配置面部檢測器

在將面部檢測應用於圖像之前,如果要更改面部檢測器的任何默認設置,請使用FirebaseVisionFaceDetectorOptions對象指定這些設置。您可以更改以下設置:

設定值
表演模式 FAST (默認)| ACCURATE

檢測臉部時提高速度或準確性。

檢測地標 NO_LANDMARKS (默認)| ALL_LANDMARKS

是否嘗試識別面部“地標”:眼睛,耳朵,鼻子,臉頰,嘴巴等。

檢測輪廓 NO_CONTOURS (默認)| ALL_CONTOURS

是否檢測面部特徵的輪廓。僅檢測到圖像中最突出的面部輪廓。

分類面孔 NO_CLASSIFICATIONS (默認)| ALL_CLASSIFICATIONS

是否將面部分類為“微笑”和“睜開眼睛”等類別。

最小臉型 float (默認值: 0.1f

要檢測的臉部相對於圖像的最小尺寸。

啟用臉部追踪 false (默認)| true

是否分配人臉ID,可用於跟踪圖像中的人臉。

請注意,啟用輪廓檢測後,只會檢測到一張臉,因此臉部跟踪不會產生有用的結果。因此,為了提高檢測速度,請不要同時啟用輪廓檢測和麵部跟踪。

例如:

爪哇

// High-accuracy landmark detection and face classification
FirebaseVisionFaceDetectorOptions highAccuracyOpts =
        new FirebaseVisionFaceDetectorOptions.Builder()
                .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
                .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
                .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
                .build();

// Real-time contour detection of multiple faces
FirebaseVisionFaceDetectorOptions realTimeOpts =
        new FirebaseVisionFaceDetectorOptions.Builder()
                .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
                .build();

Kotlin + KTX

// High-accuracy landmark detection and face classification
val highAccuracyOpts = FirebaseVisionFaceDetectorOptions.Builder()
        .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
        .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
        .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
        .build()

// Real-time contour detection of multiple faces
val realTimeOpts = FirebaseVisionFaceDetectorOptions.Builder()
        .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
        .build()

2.運行面部檢測器

要檢測圖像中的面部,請從Bitmapmedia.ImageByteBuffer ,字節數組或設備上的文件中創建FirebaseVisionImage對象。然後,將FirebaseVisionImage對像傳遞給FirebaseVisionFaceDetectordetectInImage方法。

對於面部識別,您應該使用尺寸至少為480x360像素的圖像。如果您要實時識別人臉,則以此最低分辨率捕獲幀可以幫助減少延遲。

  1. 從您的圖像創建一個FirebaseVisionImage對象。

    • 要從media.Image對象創建FirebaseVisionImage對象(例如,從設備的相機捕獲圖像時),請將media.Image對象和圖像的旋轉傳遞給FirebaseVisionImage.fromMediaImage()

      如果使用CameraX庫,則OnImageCapturedListenerImageAnalysis.Analyzer類將為您計算旋轉值,因此您只需要在調用FirebaseVisionImage.fromMediaImage()之前將旋轉值轉換為ML Kit的ROTATION_常量之一即可:

      爪哇

      private class YourAnalyzer implements ImageAnalysis.Analyzer {
      
          private int degreesToFirebaseRotation(int degrees) {
              switch (degrees) {
                  case 0:
                      return FirebaseVisionImageMetadata.ROTATION_0;
                  case 90:
                      return FirebaseVisionImageMetadata.ROTATION_90;
                  case 180:
                      return FirebaseVisionImageMetadata.ROTATION_180;
                  case 270:
                      return FirebaseVisionImageMetadata.ROTATION_270;
                  default:
                      throw new IllegalArgumentException(
                              "Rotation must be 0, 90, 180, or 270.");
              }
          }
      
          @Override
          public void analyze(ImageProxy imageProxy, int degrees) {
              if (imageProxy == null || imageProxy.getImage() == null) {
                  return;
              }
              Image mediaImage = imageProxy.getImage();
              int rotation = degreesToFirebaseRotation(degrees);
              FirebaseVisionImage image =
                      FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
              // Pass image to an ML Kit Vision API
              // ...
          }
      }
      

      Kotlin + KTX

      private class YourImageAnalyzer : ImageAnalysis.Analyzer {
          private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
              0 -> FirebaseVisionImageMetadata.ROTATION_0
              90 -> FirebaseVisionImageMetadata.ROTATION_90
              180 -> FirebaseVisionImageMetadata.ROTATION_180
              270 -> FirebaseVisionImageMetadata.ROTATION_270
              else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
          }
      
          override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
              val mediaImage = imageProxy?.image
              val imageRotation = degreesToFirebaseRotation(degrees)
              if (mediaImage != null) {
                  val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
                  // Pass image to an ML Kit Vision API
                  // ...
              }
          }
      }
      

      如果不使用提供圖像旋轉角度的相機庫,則可以根據設備的旋轉角度和設備中相機傳感器的方向來計算圖像:

      爪哇

      private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
      static {
          ORIENTATIONS.append(Surface.ROTATION_0, 90);
          ORIENTATIONS.append(Surface.ROTATION_90, 0);
          ORIENTATIONS.append(Surface.ROTATION_180, 270);
          ORIENTATIONS.append(Surface.ROTATION_270, 180);
      }
      
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      private int getRotationCompensation(String cameraId, Activity activity, Context context)
              throws CameraAccessException {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
          int rotationCompensation = ORIENTATIONS.get(deviceRotation);
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
          int sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION);
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          int result;
          switch (rotationCompensation) {
              case 0:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  break;
              case 90:
                  result = FirebaseVisionImageMetadata.ROTATION_90;
                  break;
              case 180:
                  result = FirebaseVisionImageMetadata.ROTATION_180;
                  break;
              case 270:
                  result = FirebaseVisionImageMetadata.ROTATION_270;
                  break;
              default:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  Log.e(TAG, "Bad rotation value: " + rotationCompensation);
          }
          return result;
      }

      Kotlin + KTX

      private val ORIENTATIONS = SparseIntArray()
      
      init {
          ORIENTATIONS.append(Surface.ROTATION_0, 90)
          ORIENTATIONS.append(Surface.ROTATION_90, 0)
          ORIENTATIONS.append(Surface.ROTATION_180, 270)
          ORIENTATIONS.append(Surface.ROTATION_270, 180)
      }
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      @Throws(CameraAccessException::class)
      private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          val deviceRotation = activity.windowManager.defaultDisplay.rotation
          var rotationCompensation = ORIENTATIONS.get(deviceRotation)
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
          val sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          val result: Int
          when (rotationCompensation) {
              0 -> result = FirebaseVisionImageMetadata.ROTATION_0
              90 -> result = FirebaseVisionImageMetadata.ROTATION_90
              180 -> result = FirebaseVisionImageMetadata.ROTATION_180
              270 -> result = FirebaseVisionImageMetadata.ROTATION_270
              else -> {
                  result = FirebaseVisionImageMetadata.ROTATION_0
                  Log.e(TAG, "Bad rotation value: $rotationCompensation")
              }
          }
          return result
      }

      然後,將media.Image對象和旋轉值傳遞給FirebaseVisionImage.fromMediaImage()

      爪哇

      FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);

      Kotlin + KTX

      val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
    • 要從文件URI創建FirebaseVisionImage對象,請將應用上下文和文件URI傳遞給FirebaseVisionImage.fromFilePath() 。當您使用ACTION_GET_CONTENT意圖提示用戶從其圖庫應用中選擇圖像時,此功能很有用。

      爪哇

      FirebaseVisionImage image;
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri);
      } catch (IOException e) {
          e.printStackTrace();
      }

      Kotlin + KTX

      val image: FirebaseVisionImage
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri)
      } catch (e: IOException) {
          e.printStackTrace()
      }
    • 要從ByteBuffer或字節數組創建FirebaseVisionImage對象,請首先按照上述media.Image輸入計算圖像旋轉media.Image

      然後,創建一個FirebaseVisionImageMetadata對象,該對象包含圖像的高度,寬度,顏色編碼格式和旋轉度:

      爪哇

      FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
              .setWidth(480)   // 480x360 is typically sufficient for
              .setHeight(360)  // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build();

      Kotlin + KTX

      val metadata = FirebaseVisionImageMetadata.Builder()
              .setWidth(480) // 480x360 is typically sufficient for
              .setHeight(360) // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build()

      使用緩衝區或數組以及元數據對象創建FirebaseVisionImage對象:

      爪哇

      FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
      // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);

      Kotlin + KTX

      val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
      // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
    • 要從Bitmap對象創建FirebaseVisionImage對象,請執行以下操作:

      爪哇

      FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);

      Kotlin + KTX

      val image = FirebaseVisionImage.fromBitmap(bitmap)
      Bitmap對象表示的圖像必須是直立的,不需要額外旋轉。
  2. 獲取FirebaseVisionFaceDetector的實例:

    爪哇

    FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
            .getVisionFaceDetector(options);

    Kotlin + KTX

    val detector = FirebaseVision.getInstance()
            .getVisionFaceDetector(options)
  3. 最後,將圖像傳遞給detectInImage方法:

    爪哇

    Task<List<FirebaseVisionFace>> result =
            detector.detectInImage(image)
                    .addOnSuccessListener(
                            new OnSuccessListener<List<FirebaseVisionFace>>() {
                                @Override
                                public void onSuccess(List<FirebaseVisionFace> faces) {
                                    // Task completed successfully
                                    // ...
                                }
                            })
                    .addOnFailureListener(
                            new OnFailureListener() {
                                @Override
                                public void onFailure(@NonNull Exception e) {
                                    // Task failed with an exception
                                    // ...
                                }
                            });

    Kotlin + KTX

    val result = detector.detectInImage(image)
            .addOnSuccessListener { faces ->
                // Task completed successfully
                // ...
            }
            .addOnFailureListener { e ->
                // Task failed with an exception
                // ...
            }

3.獲取有關檢測到的面部的信息

如果人臉識別操作成功, FirebaseVisionFace對象列表將傳遞給成功偵聽器。每個FirebaseVisionFace對象代表在圖像中檢測到的一張臉。對於每個臉部,您都可以在輸入圖像中獲取其邊界坐標,以及配置了臉部檢測器以查找的任何其他信息。例如:

爪哇

for (FirebaseVisionFace face : faces) {
    Rect bounds = face.getBoundingBox();
    float rotY = face.getHeadEulerAngleY();  // Head is rotated to the right rotY degrees
    float rotZ = face.getHeadEulerAngleZ();  // Head is tilted sideways rotZ degrees

    // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
    // nose available):
    FirebaseVisionFaceLandmark leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR);
    if (leftEar != null) {
        FirebaseVisionPoint leftEarPos = leftEar.getPosition();
    }

    // If contour detection was enabled:
    List<FirebaseVisionPoint> leftEyeContour =
            face.getContour(FirebaseVisionFaceContour.LEFT_EYE).getPoints();
    List<FirebaseVisionPoint> upperLipBottomContour =
            face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).getPoints();

    // If classification was enabled:
    if (face.getSmilingProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        float smileProb = face.getSmilingProbability();
    }
    if (face.getRightEyeOpenProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        float rightEyeOpenProb = face.getRightEyeOpenProbability();
    }

    // If face tracking was enabled:
    if (face.getTrackingId() != FirebaseVisionFace.INVALID_ID) {
        int id = face.getTrackingId();
    }
}

Kotlin + KTX

for (face in faces) {
    val bounds = face.boundingBox
    val rotY = face.headEulerAngleY // Head is rotated to the right rotY degrees
    val rotZ = face.headEulerAngleZ // Head is tilted sideways rotZ degrees

    // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
    // nose available):
    val leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR)
    leftEar?.let {
        val leftEarPos = leftEar.position
    }

    // If contour detection was enabled:
    val leftEyeContour = face.getContour(FirebaseVisionFaceContour.LEFT_EYE).points
    val upperLipBottomContour = face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).points

    // If classification was enabled:
    if (face.smilingProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        val smileProb = face.smilingProbability
    }
    if (face.rightEyeOpenProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        val rightEyeOpenProb = face.rightEyeOpenProbability
    }

    // If face tracking was enabled:
    if (face.trackingId != FirebaseVisionFace.INVALID_ID) {
        val id = face.trackingId
    }
}

面部輪廓示例

啟用面部輪廓檢測後,您將獲得每個檢測到的面部特徵的點列表。這些點表示要素的形狀。有關如何表示輪廓的詳細信息,請參見面部檢測概念概述

下圖說明了這些點如何映射到臉部(單擊圖像放大):

實時人臉檢測

如果要在實時應用程序中使用面部檢測,請遵循以下準則以獲得最佳幀率:

  • 配置面部檢測器以使用面部輪廓檢測或分類和界標檢測,但不能同時使用以下兩種方法:

    輪廓檢測
    地標檢測
    分類
    地標檢測和分類
    輪廓檢測和界標檢測
    輪廓檢測和分類
    輪廓檢測,界標檢測和分類

  • 啟用FAST模式(默認情況下啟用)。

  • 考慮以較低的分辨率捕獲圖像。但是,請記住此API的圖像尺寸要求。

  • 節氣門呼叫檢測器。如果在運行檢測器時有新的視頻幀可用,請放下該幀。
  • 如果要使用檢測器的輸出在輸入圖像上疊加圖形,請首先從ML Kit中獲取結果,然後在一個步驟中渲染圖像並疊加。這樣,對於每個輸入幀,只渲染一次到顯示表面。
  • 如果使用Camera2 API,請以ImageFormat.YUV_420_888格式捕獲圖像。

    如果使用較舊的Camera API,請以ImageFormat.NV21格式捕獲圖像。