Ape: Automated Testing of Android Applications with Abstraction Refinement

Download our model-based automated testing tool Ape.

Install

Files inside ape-bin.zip:

1
2
3
4
5
ape-bin/
ape-bin/ape.jar
ape-bin/ape.py
ape-bin/install.py
ape-bin/README.md

Just copy the ape.jar to the phone.

1
adb push ape.jar /sdcard/

Run

We provide a python script (i.e., ape.py) to facilitate running ape on Android platform.

The following command starts to run Ape to test the Calculator on a real device connected via adb.

1
./ape.py -p com.google.android.calculator --running-minutes 100 --ape sata

Check the ape.py if you want to run Ape with an emulator. You should at least remove the -d options for adb.

Options:

You can also specify the total amount of Monkey events. In this mode, Ape will stop by default once there is a crash.

1
./ape.py -p com.google.android.calculator --ape sata 1000

Known Bugs

  1. Memory Bloat:
    • Ape runs in the phone with limited memory for each process. Hence it may run out of memory as now we simply keep a large number of Strings and XML document objects in the memory.

Visualization

We provide a tool to visualize the model.

  1. Ape writes several js files into the output folder for visualization.
    • Check the tail of the output message to locate the output folder in the phone.
  2. adb pull /sdcard/your-output-folder to your local directory.
  3. Copy the following files into the local output directory
  4. Open the copied vis.html in your browser.
    • You may need to wait for a certain amount of time to let the browser render the model.

Examples:

Now we can check the timeline.

Configuration

Ape is under developing now. There is no stable documentation right now.

Note

The documentation below is out-of-date now. Updated documentation will come later.

Put the following content into /sdcard/ape.properties to configure Ape.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
ape.WebViewActionThreshold = 30
ape.activityStableRestartThreshold = 200
ape.alwaysIgnoreWebViewAction = true
ape.avoidEditText = false
ape.checkHomogeneous = false
ape.checkUnsaturatedTrivialState = true
ape.defaultAlpha = 0.2
ape.defaultBeta = 0.8
ape.defaultEpsilon = 0.05
ape.defaultGUIThrottle = 200
ape.defaultGamma = 0.8
ape.defaultMaxDepthEarlyStage = 3
ape.defaultMaxLengthEarlyStage = 2
ape.extraBasePriority = 3
ape.fallbackToGraphTransition = true
ape.fillTransitionsByHistory = true
ape.flushImagesThreshold = 10
ape.generateRandomTextInput = true
ape.graphStableRestartThreshold = 100
ape.ignoreEmpty = true
ape.ignoreEmptyWebViewWidget = true
ape.ignoreEmptyWidget = false
ape.ignoreOutOfBounds = true
ape.ignoreOutOfBoundsWidget = true
ape.imageWriterCount = 3
ape.includeFocusable = true
ape.invalidBackCircleThreshold = 3
ape.maxAppendThrottle = 3000
ape.nopActionThrottle = 1000
ape.onlyAddedActions = true
ape.randomPickFromStringList = false
ape.simpleGreedyEarlyStage = false
ape.sizeOfGuiTreeBuffer = 5
ape.startActionThrottle = 0
ape.stateStableCheckWelcomeThreshold = 10
ape.stateStableRestartThreshold = 50
ape.stopWhenEqual = true
ape.takeScreenshot = true
ape.takeScreenshotForEveryStep = true
ape.takeScreenshotForNewState = true
ape.throttleForUnvisitedAction = 500
ape.throttlePerActivityTransition = 100
ape.throttlePerTrivialState = 1000
ape.throttlePerWeakEdge = 500
ape.trivialStateActionThreshold = 1
ape.trivialStateUnsaturationActionThreshold = 3
ape.trivialStateUnsaturationThreshold = 3
ape.trivialStateWidgetThreshold = 3
ape.useComplexSPathChildrenThreshold = 5
ape.useComplexSPathDescendantActionThreshold = 20
ape.useDynamicSPath = true
ape.useShortID = false
ape.useShortestPathForEarlyStage = false
ape.useSimpleSPath = false
ape.useSingleScroll = false

Trap Detection

Ape will force the app to be restarted if it detects that it stays in a particular activity/state for a certain number of steps. In addition, Ape will also restart the app if the graph is not updated for a specific steps.

Check the following options.

  1. ape.graphStableRestartThreshold
  2. ape.stateStableRestartThreshold
  3. ape.activityStableRestartThreshold

WebView

The uiautomator dump now support dumping contents into WebView. But Ape ignores widgets inside a WebView by default.

Check the following options.

  1. ape.alwaysIgnoreWebViewAction = true
    • This option will make Ape ignore any WebView.
  2. ape.WebViewActionThreshold = 30
    • This option will make Ape to ignore any WebView that has more than 30 interactive widgets.

XPathlet

Ape now supports configuring widgets-specified behaviors via XPath.

Put the following json into the file /sdcard/ape.xpath. This will disable actions on any widget that has no text.

1
2
3
4
[{
    "xpath": "//*[@text='']",
    "actions": []
}]

Note

Use https://jsonlint.com/ to validate your json first.

The field actions is an array of action names.

1
2
3
4
5
6
7
NOP
CLICK
LONG_CLICK
SCROLL_TOP_DOWN
SCROLL_BOTTOM_UP
SCROLL_LEFT_RIGHT
SCROLL_RIGHT_LEFT

Screenshots

1
2
3
4
ape.takeScreenshot = true
ape.takeScreenshotForEveryStep = true
ape.takeScreenshotForNewState = true
ape.imageWriterCount = 3

Input Text

Ape decides to do text input by checking the following three conditions.

  1. Specified by XPath

    1
    2
    3
    4
    [{
        "xpath": "//*[@text='Title']",
        "text": "I am a Title"
    }]
    

    An example of testing the Google Keep app can be found here.

    This feature depends on the ADB Keyboard. You must build and install a ADBKeyboard first.

    Warning

    ADB Keyboard cannot properly handle imeOptions. To avoid unnecessary crashes, we now only do text input but disable IME actions. More information can be found at https://developer.android.com/reference/android/view/inputmethod/EditorInfo.html.

  2. ape.randomPickFromStringList = true

    • Randomly pick a line from a text file located at /sdcard/ape.strings.
  3. ape.generateRandomTextInput = true
    • Randomly generate a string by regex [0-9a-z]{0,32}.

Note

Only one type of input action can be triggered.

Acknowledgments

We thank the following experts for their insightful comments on Ape.