With Firebase Test Lab for Android, you now have the option to test with virtual devices as part of the new virtual device beta program. Virtual devices are high-fidelity virtual simulations of specific Android Nexus devices. These devices have the highest availability for scheduling and are most appropriate for day-to-day development or continuous integration testing.
Why should you use virtual devices?
Virtual devices have the following advantages when compared with physical devices:
- High availability: You can run tests and get test results more quickly when testing with virtual devices. Because virtual devices are created on-demand, your tests start almost immediately, providing quick validation of your app. This makes virtual devices ideal for testing small updates to your app, or for regression testing.
- Longer test durations: Tests on physical devices are limited to a test duration of 30 minutes on each device. Virtual devices support a test duration of up to 60 minutes, which makes them suitable for running longer tests, and especially for measuring the performance of an app.
- Lower cost: Virtual devices are priced at $1 per hour for each virtual device used to test your app. Virtual devices are available as a beta offering until October 1st, 2016, and during this time you can test your app using virtual devices at no charge. Because they are low-cost, virtual devices are a good fit for daily testing using continuous integration systems, or before checking in code. To learn more, see Test Lab billing.
How to test your app with virtual devices
You can test your app with virtual devices the same way that you test it with physical devices. Simply select virtual devices when you select test dimensions to configure a test matrix:
Best practices for testing your app
Virtual devices increase your range of options when testing your app with Test Lab. We recommend the following approach to testing your app throughout your app development lifecycle:
- In Android Studio: When developing your app, use the Android Studio emulator or an attached physical device to examine each build for initial validation. If you have instrumentation tests, you can also run these tests from Android Studio on either physical or virtual devices provided by Test Lab.
- Use Test Lab with a generic low-resolution medium DPI (MDPI) virtual
device: You can run Robo test and instrumentation tests on your app more
quickly with the Test Lab generic low-resolution MDPI phone virtual
device than with other virtual devices. The MDPI virtual device is available
on API levels 23, 24 and 25. To use this device from the gcloud command-line
interface, use the
- Use Continuous Integration (CI) systems on each code change when working on shared projects: If you work on a large project, or if you contribute to projects that are shared using GitHub or a similar sites, we recommend you use Continuous Integration (CI) systems and test your apps on virtual devices each time that the CI system runs, or before each pull request. To learn more about using Test Lab with CI systems, see Using Test Lab for Android with Continuous Integration Systems.
- Before releasing significant app updates: We recommend using Test Lab to test your app on physical devices before you release app updates with significant changes in UI and functionality. This will help to ensure that your app runs well on a wide range of popular physical devices, and also ensures test coverage for any app functionality that relies on physical device features that are not simulated by virtual devices. To learn more about these features, see Known Limitations.
Some physical device features are not currently simulated by virtual devices, or are simulated with some limitations. The following table summarizes those features that are currently unavailable on virtual devices, or that are available with certain limitations:
|Non-x86 Application Binary Interfaces (ABIs)||Virtual devices only support the x86 ABI. If you are developing with the
Android NDK, then ensure you generate code for the x86 instruction set. To
learn more, see
Note: If some of the tests in your test matrix are marked Invalid, This could be because your app relies on native ARM code.
|Video encoding and decoding||Currently, only H264 video decoding is supported on virtual devices. Video
encoding is not supported.
Note: Videos of virtual device tests currently run at one frame per second, making them less smooth than videos of physical device tests.
|OpenGL ES 3.x||The OpenGL ES 3.x and higher graphics libraries are not available on virtual devices.|
|Camera HAL v3||To learn more about the Android Camera HAL v3.0, see Camera HAL v3 overview.|