Introducing solid and snappy Virtual Device Testing for Android

Rejoice, Android developers, we present you Virtual Device Testing for Android! 🎉

Rejoice, Android developers, we present you Virtual Device Testing for Android! 🎉

We started development as we received several reports of timed out emulator boots, unreliable test results with fake fails and hanging tests. Plus, earlier you had to run device tests one by one, so testing could be quite tiresome. 😫

Enter Virtual Device Testing for Android! From now on you can run Android emulators reliably and get conclusive test results presented in logs, video and screenshots on Bitrise. You can run tests parallelly on multiple devices which is much faster, so we'll spare you some time to do some more programming instead! (Or watching an episode of Silicon Valley. You decide.)

After testing with the Android Emulator you only got one html file heralding success or announcing failure and the test took about 20-30 minutes. From now on testing will take significantly less time, and in addition, the results will be more informative and comprehensive. Beside the usual html file, the result contains a video screen capture of the whole test, screenshots, an activity map, raw and stylized logs per device to overload you with information. It's quicker and easier to debug as you'll have detailed reports and you'll see what causes the fail and when and why (?!!!)

Use the shiny new features

Activate

You can activate this separately for each app on the app's Settings tab:

Add the step

Then you'll need to add the step called Virtual Device Testing for Android to the workflow where you want to run your emulator tests (that is all your testing workflows). We support both Instrumentation tests and Robo tests, the latter is the default.

If you want to run Instrumentation test, you’ll need to have the two APK paths. You can find out here how to make the separate test APK path available in an environment variable in addition to the other one.

Build & Test

If you add the step and start a build, you'll find a new tab on the Build page of the app, next to Logs and Apps & Artifacts.

See the awesome test results overview...

When the build is concluded, you'll see the list of devices you ran your tests on, this is basically the Dashboard for your tests.

... and the details per device

If you click on a device, you'll see the test results, the status of individual test cases, a video of the whole test, screenshots, an activity map, stylized (easy to read) logs, and a list of all the files generated.

Or download all the test result files

At the bottom of the test results, there are all the files that were generated during the test for you to download:

The tests run on Firebase Test Lab and we display the results we get from them. You have unlimited device minutes for free while this feature is in beta.

Robo or Instrumentation?

We support both Instrumentation tests and Robo tests, but it is only possible to have one of them in a workflow, so if you want both, you'll need set up two workflows.

Robo

Robo is the default, so you'll only need to add the Virtual Device Testing for Android step after the Gradle Runner step and this is it.

Instrumentation

For Instrumentation tests, you'll need change your primary workflow:

  1. Add an extra task assembleDebugAndroidTest in your gradle-runner's gradle task input, so it will be something like: assembleDebug assembleDebugAndroidTest.
  2. Add Virtual Device testing for Android step after your gradle-runner step.
    As a minimum, in Virtual Device testing for Android step set Test APK path inputs to your built debug apk (e.g.: ./app/build/outputs/apk/app-debug-androidTest-unaligned.apk) Make sure to configure only one section of the step, which matches your desired test type!
    (APK path is exported by the gradle-runner step, and it is set as the default for the step's input, so if you want to use a different APK then don't forget to change the step's input accordingly!)

We hope you'll like this as much as we do! Try it, test it, love it and you know, we loooove feedback, so ping us and share the love! 💖

No items found.

Explore more topics

App development

Best practices from engineers on how to use Bitrise to build better apps, faster.

Community

Meet other Bitrise engineers, technology experts, power users, partners and join our BUGs.

Company

All the updates about Bitrise events, sponsorships, employees, and more.

Insights

Mobile development, latest tech, industry insights, and interviews with experts.

Mobile DevOps

Learn why mobile development is unique and requires a set of unique practices.

Releases

Stay tuned for the last updates, new features, and product improvements.

Get the latest from Bitrise

Join other Mobile DevOps engineers who receive regular emails from Bitrise, filled with tips, news, and best practices.