API level 27 (Android 8.1/Oreo) doesn't report a frame metric called
"deadline missed", but the JankTest library expects to be able to find
data for that metric. If it is not found, an assertion is reported and
the run fails.
Given Android Oreo is reasonably old at this point, set the minimum SDK
version to 28 so we can gather all the metrics we need.
Also bump the targetSdkVersion to make it clear what version was used
for putting this code together.
I noticed we were sometimes getting stale object exceptions. To help prevent
that situation from happening, pass around resource id's for the objects and
only fetch an instance of the objects whenever we need to click/scroll.
This means the time between fetching the object and using it is short, and
hopefully the object won't go stale or get recycled in that brief moment,
preventing the stale exceptions.
On tablets we have a landscape view by default, so we need to account for that
in our tests. On smaller phones, the landscape mode doesn't show the article
preview.
On both tablets and small screen phones, we need to be mindful of scrolling
too fast and missing an element we're looking for, either because a bigger
screen means we scroll faster or because a smaller screen means only a few
items are shown at one time.
Modify the JetNews workload to use the new jank testing classes. This allows
simplification of the code and also makes it more straightforward to implement
new workloads that also need to gather frame metrics.
This patch adds the JetNews jank-testing workload. This is accomplished
through the uiauto helper library and the jank test classes.
This workload requires a JetNews app APK to be available. We plan to
make it available through the workload-automation-assets repo.
At the end of the run, users should end up with a json file containing all
the frame/jank metrics.
There are 3 parameters for this workload:
- tests: Specifies which of the 3 available tests to run (default is to run
all of them)
- flingspeed: The speed of the fling interactions.
- repeat: How many times each of the selected tests is to be executed in a
single measuring session.
When speedometer is running on a Chrome package on Android 14, a pop-up
window was showing on the screen. Chrome preferences file is modified
to dismiss the window.
Bromite is a fork of Chromium that's easily available for Android. Apart
from small changes it works the same as Chromium and works with this
speedometer workload. Add it to the 'package_names' list to allow using
it as an option.
https://www.bromite.org/
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Update rt-app binaries to the latest version of the "lisa" branch in
douglas-raillard-arm GitHub fork. This tracks the upstream master branch
with a number of critical patches required notably to work with uclamp.
Set the view parameter so that the FPS instrument can collect frame data
from the workload.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Rework the uibenchjanktests workload to allow specifying a list of
subtests. The activity will be re-launched for each provided subtest. If
none are specified, all available tests will be run in alphabetical order.
The workload output will now include metrics with their respective test
names as classifiers.
Add a 'full' parameter to revert back to the old default 'full run'
behaviour with restarts between subtests.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Add a workload for the Dr Arm demo app. Includes functionality for
automatically pulling the ADPF FPS report file from the target if one
was generated by the app.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Add support for Geekbench 6 as a workload on Android.
This commit adds 6.*.* as a valid version for the Geekbench workload and
updates the UIAuto apk accordingly.
It also refactors the update_result function seeing as the one
originally used for GB4 can now be used for 4, 5 and 6 and so it makes
more sense to treat it as a 'generic' update_result function. The
functionality should stay the same.
Backwards compatibility with GB2 & GB3 should be maintained.
The non corporate version of geekbench5 didn't work although the code
had everything needed, except for a number of tiny required tweaks:
1. Add '5' in the supported versions in __init__.py
2. Fix the name of the android package in__init__.py and
UiAutomation.java
3. Improve handling of minorVersion to fix potential exception when we
don't specify the minorVersion number in the yaml file. Launching
geekbench5 works fine when it's the only one installed. But if you
have multiple versions, then using the version string in the yaml
agenda didn't like specifying '5' as the version and threw exception
out of bound because we assume '5.X' as input. No reason I'm aware of
to force support for a specific version of geekbench5. So keep it
relaxed until we know for sure it breaks with a specific version.
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Which works as the corporate version except for a different in package
name and a set of Fixed Time Tests that don't exist on free version.
Only support v 4.X and v5.X as that's what's available.
Note there's a clash with glbenchmark package name. glbenchmark is an
ancient version provided by the same developers but was superseded by
gfxbench. The version checks in both workloads should ensure we get the
right one in the unlikely case both are installed.
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
The test was hanging for a long time waiting for RUN text. Checking for
description first prevents that.
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
The code for reading the results is almost the same as for Antutu 8,
but it needs to be adjusted to account for a slightly different set
of benchmarks.
At least on my device, Antutu 9 takes over 10 minutes to run, so increase
the timeout to 20 minutes.
The older versions of the plugin caused problems building with newer
NDK versions due to a lack of MIPS support.
This also required upgrading to a version of Gradle that knows about
the Google Maven repository.
We need to make those attributes class-attributes, to make sure they are still
defined in subsequent jobs. We still access them through 'self', however.
On the latest version of android (currently Q) for applications that
are designed to run on older versions of android, an additional
screen asking to confirm the required permissions can popup.
Enable confirming of the granted permissions.
Resourceid and classes have been modified so update the
workload to handle these cases.
Additionally on some devices regex matches appear to fail
so workaround to match separately.
Add support for newer version of the apk.
Also add support for differing screen sizes, on larger devices
the direction of swipe to change slide differs, perform both
horizontal and vertical swipes to satisfy both layouts.
