REST Resource: projects.histories.executions.steps

Resource: Step

A Step represents a single operation performed as part of Execution. A step can be used to represent the execution of a tool ( for example a test runner execution or an execution of a compiler).

Steps can overlap (for instance two steps might have the same start time if some operations are done in parallel).

Here is an example, let's consider that we have a continuous build is executing a test runner for each iteration. The workflow would look like: - user creates a Execution with id 1 - user creates a TestExecutionStep with id 100 for Execution 1 - user update TestExecutionStep with id 100 to add a raw xml log + the service parses the xml logs and returns a TestExecutionStep with updated TestResult(s). - user update the status of TestExecutionStep with id 100 to COMPLETE

A Step can be updated until its state is set to COMPLETE at which points it becomes immutable.

JSON representation 
{
  "stepId": string,
  "creationTime": {
    object (Timestamp)
  },
  "completionTime": {
    object (Timestamp)
  },
  "name": string,
  "description": string,
  "state": enum (State),
  "outcome": {
    object (Outcome)
  },
  "hasImages": boolean,
  "labels": {
    string: string,
    ...
  },
  "dimensionValue": {
    string: string,
    ...
  },
  "runDuration": {
    object (Duration)
  },
  "deviceUsageDuration": {
    object (Duration)
  },
  "multiStep": {
    object (MultiStep)
  },

  // Union field step can be only one of the following:
  "testExecutionStep": {
    object (TestExecutionStep)
  },
  "toolExecutionStep": {
    object (ToolExecutionStep)
  }
  // End of list of possible types for union field step.
}
Fields
stepId

string

A unique identifier within a Execution for this Step.

Returns INVALID_ARGUMENT if this field is set or overwritten by the caller.

  • In response: always set
  • In create/update request: never set
creationTime

object (Timestamp)

The time when the step was created.

  • In response: always set
  • In create/update request: never set
completionTime

object (Timestamp)

The time when the step status was set to complete.

This value will be set automatically when state transitions to COMPLETE.

  • In response: set if the execution state is COMPLETE.
  • In create/update request: never set
name

string

A short human-readable name to display in the UI. Maximum of 100 characters. For example: Clean build

A PRECONDITION_FAILED will be returned upon creating a new step if it shares its name and dimensionValue with an existing step. If two steps represent a similar action, but have different dimension values, they should share the same name. For instance, if the same set of tests is run on two different platforms, the two steps should have the same name.

  • In response: always set
  • In create request: always set
  • In update request: never set
description

string

A description of this tool For example: mvn clean package -D skipTests=true

  • In response: present if set by create/update request
  • In create/update request: optional
state

enum (State)

The initial state is IN_PROGRESS. The only legal state transitions are * IN_PROGRESS -> COMPLETE

A PRECONDITION_FAILED will be returned if an invalid transition is requested.

It is valid to create Step with a state set to COMPLETE. The state can only be set to COMPLETE once. A PRECONDITION_FAILED will be returned if the state is set to COMPLETE multiple times.

  • In response: always set
  • In create/update request: optional
outcome

object (Outcome)

Classification of the result, for example into SUCCESS or FAILURE

  • In response: present if set by create/update request
  • In create/update request: optional
hasImages

boolean

Whether any of the outputs of this step are images whose thumbnails can be fetched with thumbnails.list.

  • In response: always set
  • In create/update request: never set
labels

map (key: string, value: string)

Arbitrary user-supplied key/value pairs that are associated with the step.

Users are responsible for managing the key namespace such that keys don't accidentally collide.

An INVALID_ARGUMENT will be returned if the number of labels exceeds 100 or if the length of any of the keys or values exceeds 100 characters.

  • In response: always set
  • In create request: optional
  • In update request: optional; any new key/value pair will be added to the map, and any new value for an existing key will update that key's value

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
dimensionValue

map (key: string, value: string)

If the execution containing this step has any dimension_definition set, then this field allows the child to specify the values of the dimensions.

The keys must exactly match the dimension_definition of the execution.

For example, if the execution has dimension_definition = ['attempt', 'device'] then a step must define values for those dimensions, eg. dimensionValue = ['attempt': '1', 'device': 'Nexus 6']

If a step does not participate in one dimension of the matrix, the value for that dimension should be empty string. For example, if one of the tests is executed by a runner which does not support retries, the step could have dimensionValue = ['attempt': '', 'device': 'Nexus 6']

If the step does not participate in any dimensions of the matrix, it may leave dimensionValue unset.

A PRECONDITION_FAILED will be returned if any of the keys do not exist in the dimension_definition of the execution.

A PRECONDITION_FAILED will be returned if another step in this execution already has the same name and dimensionValue, but differs on other data fields, for example, step field is different.

A PRECONDITION_FAILED will be returned if dimensionValue is set, and there is a dimension_definition in the execution which is not specified as one of the keys.

  • In response: present if set by create
  • In create request: optional
  • In update request: never set

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
runDuration

object (Duration)

How long it took for this step to run.

If unset, this is set to the difference between creationTime and completionTime when the step is set to the COMPLETE state. In some cases, it is appropriate to set this value separately: For instance, if a step is created, but the operation it represents is queued for a few minutes before it executes, it would be appropriate not to include the time spent queued in its runDuration.

PRECONDITION_FAILED will be returned if one attempts to set a runDuration on a step which already has this field set.

  • In response: present if previously set; always present on COMPLETE step
  • In create request: optional
  • In update request: optional
deviceUsageDuration

object (Duration)

How much the device resource is used to perform the test.

This is the device usage used for billing purpose, which is different from the runDuration, for example, infrastructure failure won't be charged for device usage.

PRECONDITION_FAILED will be returned if one attempts to set a device_usage on a step which already has this field set.

  • In response: present if previously set.
  • In create request: optional
  • In update request: optional
multiStep

object (MultiStep)

Details when multiple steps are run with the same configuration as a group. These details can be used identify which group this step is part of. It also identifies the groups 'primary step' which indexes all the group members.

  • In response: present if previously set.
  • In create request: optional, set iff this step was performed more than once.
  • In update request: optional

Union field step.

step can be only one of the following:
testExecutionStep

object (TestExecutionStep)

An execution of a test runner.
toolExecutionStep

object (ToolExecutionStep)

An execution of a tool (used for steps we don't explicitly support).

TestExecutionStep

A step that represents running tests.

It accepts ant-junit xml files which will be parsed into structured test results by the service. Xml file paths are updated in order to append more files, however they can't be deleted.

Users can also add test results manually by using the test_result field.

JSON representation 
{
  "testSuiteOverviews": [
    {
      object (TestSuiteOverview)
    }
  ],
  "toolExecution": {
    object (ToolExecution)
  },
  "testIssues": [
    {
      object (TestIssue)
    }
  ],
  "testTiming": {
    object (TestTiming)
  }
}
Fields
testSuiteOverviews[]

object (TestSuiteOverview)

List of test suite overview contents. This could be parsed from xUnit XML log by server, or uploaded directly by user. This references should only be called when test suites are fully parsed or uploaded.

The maximum allowed number of test suite overviews per step is 1000.

  • In response: always set
  • In create request: optional
  • In update request: never (use publishXunitXmlFiles custom method instead)
toolExecution

object (ToolExecution)

Represents the execution of the test runner.

The exit code of this tool will be used to determine if the test passed.

  • In response: always set
  • In create/update request: optional
testIssues[]

object (TestIssue)

Issues observed during the test execution.

For example, if the mobile app under test crashed during the test, the error message and the stack trace content can be recorded here to assist debugging.

  • In response: present if set by create or update
  • In create/update request: optional
testTiming

object (TestTiming)

The timing break down of the test execution.

  • In response: present if set by create or update
  • In create/update request: optional

ToolExecution

An execution of an arbitrary tool. It could be a test runner or a tool copying artifacts or deploying code.

JSON representation 
{
  "commandLineArguments": [
    string
  ],
  "toolLogs": [
    {
      object (FileReference)
    }
  ],
  "exitCode": {
    object (ToolExitCode)
  },
  "toolOutputs": [
    {
      object (ToolOutputReference)
    }
  ]
}
Fields
commandLineArguments[]

string

The full tokenized command line including the program name (equivalent to argv in a C program).

  • In response: present if set by create request
  • In create request: optional
  • In update request: never set
toolLogs[]

object (FileReference)

References to any plain text logs output the tool execution.

This field can be set before the tool has exited in order to be able to have access to a live view of the logs while the tool is running.

The maximum allowed number of tool logs per step is 1000.

  • In response: present if set by create/update request
  • In create request: optional
  • In update request: optional, any value provided will be appended to the existing list
exitCode

object (ToolExitCode)

Tool execution exit code. This field will be set once the tool has exited.

  • In response: present if set by create/update request
  • In create request: optional
  • In update request: optional, a FAILED_PRECONDITION error will be returned if an exitCode is already set.
toolOutputs[]

object (ToolOutputReference)

References to opaque files of any format output by the tool execution.

The maximum allowed number of tool outputs per step is 1000.

  • In response: present if set by create/update request
  • In create request: optional
  • In update request: optional, any value provided will be appended to the existing list

ToolExitCode

Exit code from a tool execution.

JSON representation 
{
  "number": integer
}
Fields
number

integer

Tool execution exit code. A value of 0 means that the execution was successful.

  • In response: always set
  • In create/update request: always set

TestIssue

An issue detected occurring during a test execution.

JSON representation 
{
  "errorMessage": string,
  "stackTrace": {
    object (StackTrace)
  },
  "warning": {
    object (Any)
  },
  "severity": enum (Severity),
  "type": enum (Type),
  "category": enum (Category)
}
Fields
errorMessage

string

A brief human-readable message describing the issue. Required.
stackTrace
(deprecated)

object (StackTrace)

Deprecated in favor of stack trace fields inside specific warnings.
warning

object (Any)

Warning message with additional details of the issue. Should always be a message from com.google.devtools.toolresults.v1.warnings
severity

enum (Severity)

Severity of issue. Required.
type

enum (Type)

Type of issue. Required.
category

enum (Category)

Category of issue. Required.

Any

Any contains an arbitrary serialized protocol buffer message along with a URL that describes the type of the serialized message.

Protobuf library provides support to pack/unpack Any values in the form of utility functions or additional generated methods of the Any type.

Example 1: Pack and unpack a message in C++.

Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
  ...
}

Example 2: Pack and unpack a message in Java.

Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
  foo = any.unpack(Foo.class);
}

Example 3: Pack and unpack a message in Python.

foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
  any.Unpack(foo)
  ...

Example 4: Pack and unpack a message in Go

 foo := &pb.Foo{...}
 any, err := ptypes.MarshalAny(foo)
 ...
 foo := &pb.Foo{}
 if err := ptypes.UnmarshalAny(any, foo); err != nil {
   ...
 }

The pack methods provided by protobuf library will by default use 'type.googleapis.com/full.type.name' as the type URL and the unpack methods only use the fully qualified type name after the last '/' in the type URL, for example "foo.bar.com/x/y.z" will yield type name "y.z".

JSON

The JSON representation of an Any value uses the regular representation of the deserialized, embedded message, with an additional field @type which contains the type URL. Example:

package google.profile;
message Person {
  string first_name = 1;
  string last_name = 2;
}

{
  "@type": "type.googleapis.com/google.profile.Person",
  "firstName": <string>,
  "lastName": <string>
}

If the embedded message type is well-known and has a custom JSON representation, that representation will be embedded adding a field value which holds the custom JSON in addition to the @type field. Example (for message google.protobuf.Duration):

{
  "@type": "type.googleapis.com/google.protobuf.Duration",
  "value": "1.212s"
}
JSON representation 
{
  "typeUrl": string,
  "value": string
}
Fields
typeUrl

string

A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in path/google.protobuf.Duration). The name should be in a canonical form (e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme http, https, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows:

  • If no scheme is provided, https is assumed.
  • An HTTP GET on the URL must yield a google.protobuf.Type value in binary format, or produce an error.
  • Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be used with implementation specific semantics.
value

string (bytes format)

Must be a valid serialized protocol buffer of the above specified type.

A base64-encoded string.

Severity

Severities of issues.

Enums
unspecifiedSeverity Default unspecified severity. Do not use. For versioning only.
info Non critical issue, providing users with some info about the test run.
suggestion Non critical issue, providing users with some hints on improving their testing experience, e.g., suggesting to use Game Loops.
warning Potentially critical issue.
severe Critical issue.

Type

Types of issues.

Enums
unspecifiedType Default unspecified type. Do not use. For versioning only.
fatalException Issue is a fatal exception.
nativeCrash Issue is a native crash.
anr Issue is an ANR crash.
unusedRoboDirective Issue is an unused robo directive.
compatibleWithOrchestrator Issue is a suggestion to use orchestrator.
launcherActivityNotFound Issue with finding a launcher activity
startActivityNotFound Issue with resolving a user-provided intent to start an activity
incompleteRoboScriptExecution A Robo script was not fully executed.
completeRoboScriptExecution A Robo script was fully and successfully executed.
failedToInstall The APK failed to install.
nonSdkApiUsageViolation App accessed a non-sdk Api.
nonSdkApiUsageReport App accessed a non-sdk Api (new detailed report)
encounteredNonAndroidUiWidgetScreen Robo crawl encountered at least one screen with elements that are not Android UI widgets.
encounteredLoginScreen Robo crawl encountered at least one probable login screen.
performedGoogleLogin Robo signed in with Google.
iosException iOS App crashed with an exception.
iosCrash iOS App crashed without an exception (e.g. killed).
performedMonkeyActions Robo crawl involved performing some monkey actions.
usedRoboDirective Robo crawl used a Robo directive.
usedRoboIgnoreDirective Robo crawl used a Robo directive to ignore an UI element.
insufficientCoverage Robo did not crawl some potentially important parts of the app.
inAppPurchases Robo crawl involved some in-app purchases.
crashDialogError Crash dialog was detected during the test execution
uiElementsTooDeep UI element depth is greater than the threshold
blankScreen Blank screen is found in the Robo crawl
overlappingUiElements Overlapping UI elements are found in the Robo crawl
unityException An uncaught Unity exception was detected (these don't crash apps).
deviceOutOfMemory Device running out of memory was detected
logcatCollectionError Problems detected while collecting logcat
detectedAppSplashScreen Robo detected a splash screen provided by app (vs. Android OS splash screen).

Category

Categories of issues.

Enums
unspecifiedCategory Default unspecified category. Do not use. For versioning only.
common Issue is not specific to a particular test kind (e.g., a native crash).
robo Issue is specific to Robo run.

TestTiming

Testing timing break down to know phases.

JSON representation 
{
  "testProcessDuration": {
    object (Duration)
  }
}
Fields
testProcessDuration

object (Duration)

How long it took to run the test process.

  • In response: present if previously set.
  • In create/update request: optional

ToolExecutionStep

Generic tool step to be used for binaries we do not explicitly support. For example: running cp to copy artifacts from one location to another.

JSON representation 
{
  "toolExecution": {
    object (ToolExecution)
  }
}
Fields
toolExecution

object (ToolExecution)

A Tool execution.

  • In response: present if set by create/update request
  • In create/update request: optional

MultiStep

Details when multiple steps are run with the same configuration as a group.

JSON representation 
{
  "primaryStepId": string,
  "multistepNumber": integer,
  "primaryStep": {
    object (PrimaryStep)
  }
}
Fields
primaryStepId

string

Step Id of the primary (original) step, which might be this step.
multistepNumber

integer

Unique int given to each step. Ranges from 0(inclusive) to total number of steps(exclusive). The primary step is 0.
primaryStep

object (PrimaryStep)

Present if it is a primary (original) step.

PrimaryStep

Stores rollup test status of multiple steps that were run as a group and outcome of each individual step.

JSON representation 
{
  "rollUp": enum (OutcomeSummary),
  "individualOutcome": [
    {
      object (IndividualOutcome)
    }
  ]
}
Fields
rollUp

enum (OutcomeSummary)

Rollup test status of multiple steps that were run with the same configuration as a group.
individualOutcome[]

object (IndividualOutcome)

Step Id and outcome of each individual step.

IndividualOutcome

Step Id and outcome of each individual step that was run as a group with other steps with the same configuration.

JSON representation 
{
  "stepId": string,
  "outcomeSummary": enum (OutcomeSummary),
  "multistepNumber": integer,
  "runDuration": {
    object (Duration)
  }
}
Fields
stepId

string
outcomeSummary

enum (OutcomeSummary)
multistepNumber

integer

Unique int given to each step. Ranges from 0(inclusive) to total number of steps(exclusive). The primary step is 0.
runDuration

object (Duration)

How long it took for this step to run.

Methods

accessibilityClusters

Lists accessibility clusters for a given Step

May return any of the following canonical error codes:

  • PERMISSION_DENIED - if the user is not authorized to read project
  • INVALID_ARGUMENT - if the request is malformed
  • FAILED_PRECONDITION - if an argument in the request happens to be invalid; e.g.

create

 Creates a Step.

get

 Gets a Step.

getPerfMetricsSummary

 Retrieves a PerfMetricsSummary.

list

 Lists Steps for a given Execution.

patch

 Updates an existing Step with the supplied partial entity.

publishXunitXmlFiles

 Publish xml files to an existing Step.