Google is committed to advancing racial equity for Black communities. See how.

Analyzing Test Results

Whether you run your tests through the Firebase console or the Firebase CLI, you can find your detailed test results in the Firebase console. Read on to learn how to analyze your test results.

Key concepts

To see test results head to the Test Lab section of the Firebase console.

You'll find a list of all of your previous test runs there. To understand the the results, it helps to know a bit about test matrices: Devices × Test Executions = Test Matrix

A device you run a test on, such as a phone, a tablet, or a wearable device. Devices in a test matrix are identified by device model, OS version, locale, and screen orientation.
Test execution
A test run on a device. In a typical test matrix there is one test execution per selected device.
Test matrix
A set of test executions. If any test execution in a matrix fails, the whole matrix fails as well.

Interpret test matrix results

If you start your tests in Firebase console, Test Lab takes you right to your in-progress test matrix, where you can watch your tests' progress as it happens. If you used the command line tool instead, you can follow the URL it provides to get to the in-progress test matrix page.

The matrix has a line for each test execution Test Lab runs. The icon before the execution name displays that execution's status:

  • play_circle_outline In progress: The test execution is still running. Executions can take up to an hour to complete, depending on the number and complexity of tests in your app.
  • check_circle Passed: None of the execution's tests failed.
  • warning Failed: At least one of the execution's tests failed.
  • error Inconclusive: The test results were inconclusive, possibly due to a Test Lab error.
  • block Skipped: Test Lab skipped your test, because the device/OS version combination you selected is unavailable.

Interpreting results from a single test execution

From the test matrix results page, click one of the test executions to see the result of that specific test execution.

The page shows you stats for that test execution, including issues encountered in testing, a list of test cases, logs from the execution, and a video of the test running.

Detailed test results

Detailed test results are available in a Google Cloud Storage (GCS) bucket for 90 days. Click View Source Files in a test execution result page to see the bucket.

To retain detailed test results for longer than 90 days, send the test results to a more-permanent GCS bucket that you own using the --results-bucket gcloud command-line option. You can then set the Age setting to determine how long GCS stores the results. For more information on how to change the Age setting, see Lifecycle conditions.