Firebase Test Lab provides cloud-based infrastructure for testing Android apps. This document describes how to get started with Test Lab using the Firebase console.
Test Lab lets you run the following types of tests:
Instrumentation test: A test you write that allows you to drive the UI of your app with the actions you specify. An instrumentation test can also make explicit assertions about the state of your app to verify correct functionality using AndroidJUnitRunnerAPIs. Test Lab supports Espresso and UI Automator instrumentation test frameworks.
Robo test: A test that analyzes your app's interface and then explores it automatically by simulating user activities.
Game loop test: A test that uses a "demo mode" to simulate player actions in game apps.
Before you begin
Before you get started, you must enable billing for your project. If you don't have an active billing account, add one and then connect your project to that billing account. You will need owner or edit permissions in your project.
Step 1. Create a Firebase project
If you haven't yet, go to the Firebase console and create a new Firebase project.
Step 2. Run a test
Instrumentation test
On the Firebase console navigation bar, click Test Lab, and then click Get Started -> Run an Instrumentation test.
Click Browse, and then browse to your app APK or AAB and test APK or AAB before clicking Continue.
Define your test matrix by selecting which devices, Android API levels, screen orientations and locales you want to test your app against. You can select only those device and Android API level combinations that you want to target for testing.
(Optional) Click Show advanced options to change the Test timeout that determines the maximum duration of each test execution.
(Optional) To help you identify and locate your test matrices in the Firebase console, you can add a label to your test matrix by entering a label name in the Test matrix label (optional) field.
Click Start N Tests, where "N" is the number of valid test configurations from the test matrix that you define on this screen. Each pending test is shown with a blue clock icon while it is waiting to run, and that icon changes to a green check icon when the test has completed.
After each test has run, click the device listed in the Test Execution column to see test results, including test cases, logs, screenshots and videos.
Robo test
On the Firebase console navigation bar, click Test Lab, and then click Get Started -> Run a Robo test.
Click Browse, browse to your app APK, and then click Continue.
Define your test matrix by selecting which devices, Android API levels, screen orientations and locales you want to test your app against.
(Optional) Click Show advanced options to change the following options:
- Test timeout determines the maximum duration of each test execution.
- Test account credentials are usedis used to provide credentials for a test account.
- Additional fields are used to provide text input for other text fields in your app.
(Optional) To help you identify and locate your test matrices in the Firebase console, you can add a label to your test matrix by entering a label name in the Test matrix label (optional) field.
Click Start N Tests, where "N" is the number of valid test configurations from the test matrix that you define on this screen. Each pending test is shown with a blue clock icon while it is waiting to run, and that icon changes to a green check when the test has completed.
After each test finishes running, click the device listed in the Test Execution column to see test results, including test cases, logs, screenshots and videos.
If you want to create a script to guide the Robo test, see Record a Robo script using Test Lab in Android Studio.
Game Loop test
On the Test Lab page of the Firebase console, click Run Your First Test > Run an Android Game Loop.
In the Upload App section, click Browse, then select your app's APK file (if you haven't already, generate an APK file for your app).
(Optional) To help you identify and locate your test matrices in the Firebase console, you can add a label to your test matrix by entering a label name in the Test matrix label (optional) field.
(Optional) If you want to run multiple loops or scenarios at a time, or select specific loops to run, enter the loop numbers in the Scenarios field.
For example, when you enter "1-3, 5", Test Lab runs loops 1, 2, 3, and 5. By default (if you don't enter anything in the Scenarios field), Test Lab only runs loop 1.
In the Devices section, select one or more physical devices you want to test your app on, then click Start Tests.
Step 3. Investigate your test results
When the test starts, you're automatically redirected to the test results page. Tests can take a number of minutes to run, depending on the number of different configurations you have selected and the test timeout duration set for your tests. After your tests have run, you can review test results. See Analyzing Firebase Test Lab Results to learn more about how to interpret the test results.