fMBT - fMBT in Android UI testing

fMBT version 0.9 introduces a bunch of improvements in modelling, Android testing and debugging UI tests. In this blog I will demonstrate these and some earlier fMBT improvements with an Android UI testing example.

Setting up the environment

In order to use fMBT for Android testing, you will need three things: fMBT (v0.9 or later), Android SDK, and either an Android device or emulator. Here are the details.

In Android SDK, Platform tools (especially adb) is the minimum requirement.

If you use Android emulator, make sure to include a SD Card. A couple of megabytes is enough. If you wish to run the fMBT Android example, I recommend emulated Nexus S with Android 4.2.2.

If you use an Android device, you will need to enable USB debugging in Settings.

Your Android environment is ready when you can login to the device or emulator with adb shell.

First steps with fmbtandroid

fmbtandroid is a Python library for Android UI testing. Reading docs and trying it out is most convenient in Python shell. (I warmly recommend enabling history and tab completion or using an enhanced shell like IPython). Example:

$ python
>>> import fmbtandroid
>>> help(fmbtandroid)
>>> help(fmbtandroid.Device)

Instantiating fmbtandroid.Device connects to the first available Android device in adb devices list. After that you can send inputs and verify screen contents. Visual log helps inspecting has been done. Run the following example in Python.

import fmbtandroid

# Connect to a device or emulator.
d = fmbtandroid.Device()

# Start writing log to example.html
d.enableVisualLog("example.html")

# Take a screenshot for bitmap and OCR matching.
d.refreshScreenshot()

# Require starting from the homescreen.
assert d.verifyOcrText('Google'), "not in homescreen"

# Open application grid by tapping 50 % of max. X and 95 % of max. Y.
# The origo is in the top left corner of the display.
d.tap((.5, .95))

# Wait until "Browser" text appears on the display, then tap it.
assert d.waitOcrText('Browser'), "could not find 'Browser' in appgrid"
d.tapOcrText('Browser')

Now see the log: chromium example.html. When you open the first log item, you will see screenshots from the device with tapped coordinates and recognized texts highlighted in them, and timestamps for all calls and returns. When this kind of sequences are executed by fMBT during a test run, log entries are grouped under names of AAL/Python code blocks. In this example, all log entries are under "undefined" block, as fMBT did not generate this test.

In general, it is perfectly fine to use fmbtandroid for Android testing in without fMBT, too. Logging is the only feature in it fmbtandroid that has a built-in support for fMBT test steps, but even that can be overriden.

Taking reference bitmaps for swipeBitmap, tapBitmap, verifyBitmap and waitBitmap is pretty convenient with screenshot tools like Shutter. You can grab bitmaps either directly from emulator's screen, or from the device by executing in shell:

# Save screenshot from the device (or emulator) to /tmp
python -c 'import fmbtandroid; fmbtandroid.Device().refreshScreenshot().save("/tmp/screenshot.png")'

# Display the screenshot
display /tmp/screenshot.png &

# Capture a region from the screenshot to a bitmap
shutter -s -exit_after_capture -o webpage-logo.png

Generating tests

In Android UI test example fMBT generates tests from test steps defined in phone.aal. It is written in AAL/Python language.

You can fetch the test and open it in the editor as follows:

git clone https://github.com/askervin/fmbt-androidtest
cd fmbt-androidtest
fmbt-editor phone.aal regressiontest.conf

Generated test can be seen in the Test tab (F6). It is generated from phone.aal according to test configuration in regressiontest.conf. The configuration aims to cover all possible input pairs (perm(2)). You can change this, for instance, to cover five unique input sequences from launching the phone application to returning to the homescreen: replace coverage = ... line in regressiontest.conf with

coverage  = uinputs(from 'i:launch.*' to 'i:return.*')
pass      = coverage(5)

Tip: you can view measured coverage by right-clicking the Test tab and choosing Coverage in the menu.

Running the test

Run the test in fmbt-androidtest example:

  1. Start Nexus-S emulator in Android SDK.
    • Make sure adb shell connects to it.
    • Make sure Nexus-S is in the homescreen and there are no calls in progress.
  2. Execute fmbt regressiontest.conf in the fmbt-androidtest directory.

Once the test run is over, run chromium devicelog.html to see the details in each executed test step and verified states.

Project: 

Comments

Antti Kervinen's picture

Update: new platforms and fmbt-scripter

fMBT GUI testing support has improved a lot since writing this blog. Most significantly,

  • fmbtandroid-like GUI test interface supports testing Tizen, Windows, X11and anything that runs a VNC server. See wiki for more information.
  • fmbt-scripter helps writing and debugging Python test code, and it makes selecting bitmaps easy from any platform supported by fMBT's GUI test interface. As an example, move text cursor on "clock.png" in test script, click the "Select clock.png" button and select the bitmap from the screenshot on the left.

fmbt-scripter and Nexus-s

 

Pages