On some devices the score string obtained can contain extra characters.
Only use the numerical values from the score when converting, otherwise
if not found set the result to 'NaN'.
By clearing the application data each time the workload is run this
forces the required assets to be re-installed each time. As the
workload is not affected by persistent state do not perform the
clearing.
While attempting to run vellamo on Android Q, a popup warning with
the message, "This app was built for an older version of Android and may not
work properly. Try checking for updates, or contact the developer." would
appear, causing the workload to halt.
Close the popup warning before dismissing EULA and executing the remaining
steps to run vellamo.
Tested with vellamo apk version 3.2.4.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
Previously the LogcatRunMonitor left the logcat process running in the
background causing issues with concurrent accesses. Now ensure the thread
terminates correctly.
For workloads that support Linux targets do not use
`uninstall_executable` as this is not available, instead use `uninstall` as
other targets should be able to determine the appropriate uninstallation
method.
Previously the workload assumed that all the scores were visible on a
single screen however for devices with smaller displays the results need
to scrolled.
A new popup appears when running pcmark on android Q that complains
about the app being built for an older version of android.
Since this popup will be temporary, the fix has to make sure not to
break in the future when this popup disappears or when the test is ran
on a compatible version of android.
To achieve this, we attempt to dismiss the popup and if we timeout we
silently carry on with the test assuming no popup will appear.
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Creating a new workload to execute the following tests on GFXBench.
* Car Chase
* Car Chase Offscreen
* Manhattan 3.1
* 1080p Manhattan 3.1 Offscreen
* 1440p Manhattan 3.1 Offscreen
* Tessellation
* Tessellation Offscreen
Leftover code from WA2 meant that logcat was cleared on the device by
the workload directly instead of using devlib, this caused issues if logcat was
still being cleared from other areas of the code.
On some devices running Android 8.1 the start benchmark button was
failing to be clicked, this is a workaround to click on the coordinates
of the button instead of the UiObject iteslf.
- Add copyright headers to files that did not already have them
- Update the year to the last year the file was modified in files with
existing headers.
Mongoperf currently has a non-ASCII character in the file that causes a failure.
This commit rewords the description to remove the incriminating character.
Add a workload that runs mongoperf benchmark that measures I/O
performance on a MongoDB server. This workload assumes that mongoperf
is already installed.
Lengthen timeout for gemm and conv tests to be the same as for sparse
test. While the former two usually take a lot less time, their execution
time will vary significantly depending on the target and the runtime
environment (e.g. cpu frequencies might be forced to lowest values).
urllib2 does not exist in Python 3, and its methods have been moved into
urllib.request. Use future library to create aliases that work across
both 2 and 3.
Add a workload for lat_mem_rd and bw_mem tests of LMbench benchmark, a suite of
portable ANSI/C microbenchmarks for UNIX/POSIX.
This is a port of corresponding workload from WA2.
Certain devices present a night time reading popup alert when opening the books workload.
This can throw off the workload so I've added a function to check for this and dismiss if neccessary.
As part of resolving a resource, record its MD5 hash in the output
metadata. In order to enable this, resource resolution is now done via
the context, rather than directly via the ResourceResolver (in order not
to introduce a context dependency in the Gui object, context now
emulates the resolver interface).
Update workloads that allow for specifying which cpus to be ran on to
all use the same interface while maintaining backwards compatibility with
the existing configuration options.
stress-ng will stress test a computer system in various selectable
ways. It was designed to exercise various physical subsystems of a
computer as well as the various operating system kernel interfaces.
The rt-app workload parses the rt-app JSON file in order to override
some options (specifically duration). Previously, if the JSON was
syntactically incorrect, an uninformative ValueError was raised. Now
we raise a ConfigError with appropriate message prompting the user to
fix the file.
Port sysbench workload from WA2.
sysbench is a modular, cross-platform and multi-threaded benchmark tool
for evaluating OS parameters that are important for a system running a
database under intensive load.
Port manual workload from WA2. This workload allows using WA to
configure and instrument the target, while letting the user perform
arbitrary actions in place of running a fixed workload.
since Linaro's modification have been merged in upstream version,
we can remove this comment
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Ensure the workload uses the passed working directory for attaching
images instead of a hard coded name and allow for finding of the attach
button by description as well as name.
Not all benchmarks are available on all phones and will report "Not
supported" To allow for this update the regexes and if not a valid score
set any unsupported results to 'NaN'
We remove the requirement for rt-app to run on a rooted device as this
does not appear to be required. We make sure that we run the workload
from a directory where we have write permissions by first changing the
directory before executing the workload (in_directory argument of
invoke).
In the case that the device is rooted, we run rt-app as
root. Otherwise, we simply run the workload.
By default rt-app writes to stderr, not to stdout. Therefore, if we
only sample the stdout, we get no output from the workload itself.
With this change, we redirect stderr to stdout to allow us to get the
workload output.
Set target_binary as a class, rather than instance, attribute. This
happens only only once per run, and setting it as instance attribute the
first time, makes it unavailable for subsequent instances of the same
workload.
Always prepend the format for the file into the downloaded filepath to
ensure matching is correctly performed. Previously the ogg file did not
specify this causing it to be re-downloaded for each run.
In devlib the default working directory has changed causing the workload
to look in the wrong directory, now the workload should use the passed
working directory and extract the relevant name from there.
To improve reproducibility of tests, it could help to ensure that the
same screen orientation is used on media playback.
This adds a new workload parameter which allows to ensure a LANDSCAPE
mode, which can be useful to better represent video playback scenarios.
When not enforced by the agenda, we enforce a PORTRAIT orientation.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Fix output when running for a specified number of loops (rather than
duration) and adding delay to threads. Previously, the output for the
last thread was not printed due to a missing check for a negative
duration.
Remove wa.framework.plugin.Artifact and associated references. The name
of the class clashes with the class from output and can potentially
cause confusion.
The original intention for this was to be an "expected artifact
descriptor" of sorts that plugins can specify for validation purposes,
but that functionality was never implemented. Given that the framework
has undergone significant changes since this was implemented, it's not
clear that this is the best way to go about the original goal.
Therefore remove this for now.
To reflect that the current implementation of ReventWorkload uses an APK
file on the device the name has been changed appropriately and the
relevant workloads have been updated.
The report_results parameter has been removed as the parsing functionality
is now performed by the `uxperf` processor therefore to report the
results of the application launch time ensure the processor has been
enabled.
Do not concatenate start/stop markers to the action string with a '_',
delimit them with a space instead. This will make easier to parse the
message afterwards.
Seem I cherry-picked the geekbench workload from my WIP branch
without testing properly, and missed a fixup commit. Here's the
content of that commit, which fixes some missing updates from WA2 ->
WA3 APIs.
