Properties

clientInfo

non-null functions.testLab.ClientInfo

Information about the client that invoked the test.

createTime

string

When this test matrix was initially created (ISO8601 timestamp).

invalidMatrixDetails

(string or undefined)

For 'INVALID' matrices only, describes why the matrix is invalid.

Possible values:

  • DETAILS_UNAVAILABLE: The matrix is INVALID, but there are no further details available.
  • MALFORMED_APK: The input app APK could not be parsed.
  • MALFORMED_TEST_APK: The input test APK could not be parsed.
  • NO_MANIFEST: The AndroidManifest.xml could not be found.
  • NO_PACKAGE_NAME: The APK manifest does not declare a package name.
  • INVALID_PACKAGE_NAME: The APK application ID is invalid.
  • TEST_SAME_AS_APP: The test package and app package are the same.
  • NO_INSTRUMENTATION: The test APK does not declare an instrumentation.
  • NO_SIGNATURE: The input app APK does not have a signature.
  • INSTRUMENTATION_ORCHESTRATOR_INCOMPATIBLE: The test runner class specified by user or in the test APK's manifest file is not compatible with Android Test Orchestrator.
  • NO_TEST_RUNNER_CLASS: The test APK does not contain the test runner class specified by user or in the manifest file.
  • NO_LAUNCHER_ACTIVITY: A main launcher activity could not be found.
  • FORBIDDEN_PERMISSIONS: The app declares one or more permissions that are not allowed.
  • INVALID_ROBO_DIRECTIVES: There is a conflict in the provided robo_directives.
  • INVALID_RESOURCE_NAME: There is at least one invalid resource name in the provided robo directives.
  • INVALID_DIRECTIVE_ACTION: Invalid definition of action in the robo directives, e.g. a click or ignore action includes an input text field.
  • TEST_LOOP_INTENT_FILTER_NOT_FOUND: There is no test loop intent filter, or the one that is given is not formatted correctly.
  • SCENARIO_LABEL_NOT_DECLARED: The request contains a scenario label that was not declared in the manifest.
  • SCENARIO_LABEL_MALFORMED: There was an error when parsing a label value.
  • SCENARIO_NOT_DECLARED: The request contains a scenario number that was not declared in the manifest.
  • DEVICE_ADMIN_RECEIVER: Device administrator applications are not allowed.
  • MALFORMED_XC_TEST_ZIP: The zipped XCTest was malformed. The zip did not contain a single .xctestrun file and the contents of the DerivedData/Build/Products directory.
  • BUILT_FOR_IOS_SIMULATOR: The zipped XCTest was built for the iOS simulator rather than for a physical device.
  • NO_TESTS_IN_XC_TEST_ZIP: The .xctestrun file did not specify any test targets.
  • USE_DESTINATION_ARTIFACTS: One or more of the test targets defined in the .xctestrun file specifies "UseDestinationArtifacts", which is disallowed.
  • TEST_NON_APP_HOSTED: XC tests which run on physical devices must have "IsAppHostedTestBundle" == "true" in the xctestrun file.
  • PLIST_CANNOT_BE_PARSED: An Info.plist file in the XCTest zip could not be parsed.
  • NO_CODE_APK: APK contains no code.
  • INVALID_INPUT_APK: Either the provided input APK path was malformed, the APK file does not exist, or the user does not have permission to access the APK file.
  • INVALID_APK_PREVIEW_SDK: APK is built for a preview SDK which is unsupported.

outcomeSummary

(string or undefined)

The overall outcome of the test matrix run. Only set when the test matrix state is FINISHED.

Possible values:

  • SUCCESS: The test matrix run was successful, for instance:
    • All the test cases passed.
    • Robo did not detect a crash of the application under test.
  • FAILURE: The test run failed, for instance:
    • One or more test cases failed.
    • A test timed out.
    • The application under test crashed.
  • INCONCLUSIVE: Something unexpected happened. The run should still be considered unsuccessful but this is likely a transient problem and re-running the test might be successful.
  • SKIPPED: All tests were skipped, for instance:
    • All device configurations were incompatible.

resultStorage

non-null functions.testLab.ResultStorage

Where the results for the matrix are located.

state

string

Indicates the current progress of the test matrix.

Possible values:

  • VALIDATING: The matrix is being validated.
  • PENDING: The matrix is waiting for resources to become available.
  • FINISHED: The matrix has terminated normally. This means that the matrix level processing completed normally, but individual executions may be in an ERROR state.
  • ERROR: The matrix has stopped because it encountered an infrastructure failure.
  • INVALID: The matrix was not run because the provided inputs are not valid. E.g. the input file is not of the expected type, or is malformed/corrupt.

testMatrixId

string

Unique ID set by the service.