This guide describes how to automatically distribute builds to testers using the App Distribution Automated Tester feature in the Firebase console. The Automated Tester feature provides an always-ready, easy-to-use smoke testing tool for your app.
Before you begin
If you haven't already, add Firebase to your Android project.
If you aren't using any other Firebase products, you only have to create a project and register your app. However, if you decide to use additional products in the future, be sure to complete all of the steps in Add Firebase using the Firebase console.
When you're ready to distribute a pre-release version of your app to testers, build your APK using your normal process. You must sign the APK with your debug key or app signing key.
Distribute your app to Automated Tester using the Firebase console
To distribute your app to Automated Tester, upload your APK file using the Firebase console:
- Open the App Distribution page of the Firebase console. Select your Firebase project when prompted.
- On the Releases page, select the app you want to distribute from the drop-down menu.
- Drag your app's APK file to the console to upload it.
- When the upload completes, select Automated Tester in the Add testers or groups search bar.
- To customize your automated test, click the Edit icon and select the customizations in the Customize your automated test dialog.
- (Optional) In the device configuration dialog, select the API level, device orientation, and locale that meets your testing specifications.
- (Optional) Click the Login credentials drop-down menu to create a custom username and password that can be used during the test if your app requires login credentials.
- Click Distribute. You can also add an optional release note for your app.
Distribute your app to Automated Tester using the Firebase CLI
Run the appdistribution:distribute command to upload your app and
distribute it to testers. Use the following parameters to
configure your distribution to the Automated Tester feature:
| appdistribution:distribute options | |
|---|---|
--test-devices or
--test-devices-file
|
The test devices you want to distribute builds to the Automated Tester feature. You can specify the test devices as a semicolon-separated list of test devices: --test-devices: "model=shiba, version=34, locale=en, orientation=portrait;model=b0q, version=33, locale=en, orientation=portrait"
Or, you can specify the path to a plain text file containing a semicolon-separated list of test devices: --test-devices-file: "/path/to/test-devices.txt"
|
--test-username
|
The username for automatic login to be used during automated tests. |
--test-password or
--test-password-file
|
The password for automatic login to be used during automated tests. Or, you can specify the path to a plain text file containing a password: --test-password-file: "/path/to/test-password.txt"
|
--test-username-resource
|
Resource name for the username field for automatic login to be used during automated tests. |
--test-password-resource
|
Resource name for the password field for automatic login to be used during automated tests. |
--test-non-blocking |
Run automated tests asynchronously. Visit the Firebase console for the automatic test results. |
For more information on getting started with the Firebase CLI and different ways to configure your distribution, see Distribute Android apps to testers using the Firebase CLI.
Distribute your app to Automated Tester using Gradle
You can configure App Distribution by adding at least one firebaseAppDistribution
section and use the following parameters to configure the distribution to the
Automated Tester feature:
| App Distribution Build Parameters | |
|---|---|
testDevices or
testDevicesFile
|
The test devices you want to distribute builds to using automated tests. You can specify the test devices as a semicolon-separated list of device specifications testDevices="model=shiba, version=34, locale=en, orientation=portrait;model=b0q, version=33, locale=en, orientation=portrait"
Or you can specify the path to a file containing a semicolon-separated list of device specifications: testDevicesFile="/path/to/testDevices.txt"
|
testUsername
|
The username for automatic login to be used during automated tests. |
testUsernameResource
|
Resource name for the username field for automatic login to be used during automated tests. |
testPassword or
testPasswordFile
|
The password for automatic login to be used during automated tests. Or, you can specify the path to a plain text file containing a password: testPasswordFile="/path/to/testPassword.txt" |
testPasswordResource
|
Resource name for the password field for automatic login to be used during automated tests. |
testNonBlocking |
Run automated tests asynchronously. Visit the FIrebase console for the automatic test results. |
For more information on getting started with Gradle and different ways to configure your distribution, see Distribute Android apps to testers using Gradle.
Distribute your app to Automated Tester using fastlane
In a ./fastlane/Fastfile lane, add a
firebase_app_distribution block. Use the following parameters to
configure your distribution to the Automated Tester feature:
| firebase_app_distribution parameters | |
|---|---|
test_devices or
test_devices_file
|
The test devices you want to distribute builds to the Automated Tester feature. You can specify the test devices as a semicolon-separated list of test devices: test_devices: "model=shiba, version=34, locale=en, orientation=portrait;model=b0q, version=33, locale=en, orientation=portrait"
Or, you can specify the path to a plain text file containing a semicolon-separated list of test devices: test_devices_file: "/path/to/test-devices.txt"
|
test_username
|
The username for automatic login to be used during automated tests. |
test_password or
test_password_file
|
The password for automatic login to be used during automated tests. Or, you can specify the path to a plain text file containing a password: test_password_file: "/path/to/test-password.txt"
|
test_username_resource
|
Resource name for the username field for automatic login to be used during automated tests. |
test_password_resource
|
Resource name for the password field for automatic login to be used during automated tests. |
test_non_blocking |
Run automated tests asynchronously. Visit the FIrebase console for the automatic test results. |
For more information on getting started with fastlane and different ways to configure your distribution, see Distribute Android apps to testers using fastlane.
Automatic test crawl statuses
To help you interpret your test results, the Automated test results tab shows any test issues, screenshots of the app, and a video of the test crawl for each device. The status of the test can be found in your test results page:
| Crawl status | Description |
|---|---|
| Pending | The automatic test is still running and your results will be available soon. |
| Passed | The automatic test successfully crawled your app and did not find any crashes. |
| Failed | The automatic test failed because your app crashed during the test. |
| Inconclusive | The automatic test failed because of infrastructure errors. |