Get started with Firebase Test Lab from the gcloud Command Line

Firebase Test Lab lets you test your iOS app on multiple devices through the gcloud command line interface. Keep reading to find out how to get started.

Create a Firebase project

If you don't have a Firebase project for your app, go to the Firebase console and click Create New Project to create one now. You will need ownership or edit permissions in your project.

You can use Test Lab for a limited number of daily test executions on the Spark and Flame plans. To use Test Lab without daily quota limits, you must upgrade to the Firebase Blaze plan.

Configure your local Google Cloud SDK environment

  1. Download the gcloud command line tool, if you haven't yet: Google Cloud SDK
  2. Make sure your Cloud SDK installation is up-to-date and includes the gcloud firebase command:
    gcloud components update
  3. Make sure your authentication credentials are current:
    gcloud auth login
  4. Set your current project in gcloud:
    gcloud config set project PROJECT_ID
  5. Configure gcloud to use your current project as its billing/quota project:
    gcloud config set billing/quota_project PROJECT_ID

Build XCtests for your app

To build your test, use the following command in the terminal:

project

xcodebuild -project PATH/TO/YOUR_WORKSPACE.xcodeproj \
  -scheme YOUR_SCHEME \
  -derivedDataPath FOLDER_WITH_TEST_OUTPUT \
  -sdk iphoneos build-for-testing

workspace

xcodebuild -workspace PATH/TO/YOUR_WORKSPACE.xcworkspace \
  -scheme YOUR_SCHEME \
  -derivedDataPath FOLDER_WITH_TEST_OUTPUT \
  -sdk iphoneos build-for-testing

After your test successfully builds, zip it for upload to Test Lab:

cd FOLDER_WITH_TEST_OUTPUT/Build/Products ; \
zip -r MyTests.zip Debug-iphoneos YOUR_SCHEME_iphoneosDEPLOYMENT_TARGET-arm64.xctestrun

Run your tests

Before you run your tests, you have to choose a set of devices to run them on. To see a list of devices, enter the following command in your terminal:

gcloud alpha firebase test ios models list

Once you've chosen, have Test Lab run the tests:

gcloud alpha firebase test ios run --test PATH/TO/MyTests.zip \
  --device model=MODEL_ID_1,version=VERSION_ID_1 \
  --device model=MODEL_ID_2,version=VERSION_ID_2 \
  etc...

Analyze your test results

When the test is done, the gcloud tool prints a basic summary of your test results. As part of the summary, the tool includes a link to view more detailed results in the Firebase console. To learn more about how to interpret these results, see Analyzing Firebase Test Results.

Automate future tests

Scripting gcloud commands with Test Lab

You can use shell scripts or batch files to automate mobile app testing commands that you would otherwise run using the gcloud command line. This sample bash script runs an XCTest with a two-minute timeout, and reports if the test run completed successfully:

if gcloud alpha firebase test ios run --test MyTest.zip --timeout 2m
then
    echo "Test matrix successfully finished"
else
    echo "Test matrix exited abnormally with non-zero exit code: " $?
fi

Script exit codes

Test Lab provides several exit codes that you can use to better understand the results of tests that you run using scripts or batch files.

Scripting exit codes for Test Lab

Exit code Notes
0 All test executions passed.
1 A general failure occurred. Possible causes include: a filename that does not exist or an HTTP/network error.
2 Testing exited because unknown commands or arguments were provided.
10 One or more test cases (tested classes or class methods) within a test execution did not pass.
15 Firebase Test Lab could not determine if the test matrix passed or failed, because of an unexpected error.
19 The test matrix was canceled by the user.
20 A test infrastructure error occurred.

Next step

Explore the Test Lab for iOS command line SDK: gcloud alpha firebase test ios

Send feedback about...

Need help? Visit our support page.