Firebase Test Lab provides cloud-based infrastructure for testing Android apps. This document describes how to get started with Test Lab using the Firebase console.
Create a Firebase project
If you don't have a Firebase project for your app, go to the Firebase console and click Create New Project to create one now. You will need ownership or edit permissions in your project.
You can use Test Lab for a limited number of daily test executions on the Spark and Flame plans. To use Test Lab without daily quota limits, you must upgrade to the Firebase Blaze plan.
Run a test
Firebase Test Lab lets you run the following types of tests:
- Instrumentation test: A test you write that allows you to drive the UI of your app with the actions you specify. An instrumentation test can also make explicit assertions about the state of your app to verify correct functionality using AndroidJUnitRunnerAPIs. Test Lab supports Espresso and UI Automator 2.0 instrumentation test frameworks.
- Robo test: A test that analyzes your app's interface and then explores it automatically by simulating user activities.
- Game loop test: A test that uses a "demo mode" to simulate player actions in game apps.
To run a test, see Run tests with Firebase Test Lab for Android.
New: instrumentation tests with Orchestrator
Android Test Orchestrator runs each of your app's instrumentation tests independently, which has several benefits, but also one drawback:
Benefits
No shared state: Each test runs in its own Instrumentation instance, so shared state doesn't accumulate across tests.
Isolated crashes: If a test crashes, it takes down only its own instance of Instrumentation, so the other tests in your suite still run.
Drawback
- Longer runtime: Each test running its own Instrumentation instance means that the testing process takes slightly longer overall. The increased run time could impact your quota usage or billed time and might cause you to hit devices' time-out limits.
Test Lab always uses the latest version of Orchestrator. To enable Orchestrator, select Run with Orchestrator from the Additional options screen in instrumentation test setup.
Investigate your test results
When the test starts, you're automatically redirected to the test results page. Tests can take a number of minutes to run, depending on the number of different configurations you have selected and the test timeout duration set for your tests. After your tests have run, you can review test results. See Analyzing Firebase Test Lab Results to learn more about how to interpret the test results.