Previously when retrieving apks only it's name would be used to choose
an apk. Now the native code reported by the apk is used to determine
the correct version to run for the specific device. It tries to
match the primary abi of device with native code before falling back to
using a compatible apk.
When using the check_abi parameter it no longer relies on naming convention
and only allows apks with native code supporting a devices primary abi to be
used.
Updated the relevant documentation.
Added property to linux device to return list of supported abis
to be consistent with android devices. Currently only returns a
list containing the primary abi.
Fixed new gamemode not being set for compatibility mode.
Fixed input buffer not being flushed causing replay to fail.
Fixed infinte loop if revent recording did not end correctly.
Updated revent binaries.
Tested by running Mate 8 on the lowest cluster at the lowest available frequency
On rooted devices, skip attempting to pull log files from device. This allows
the benchmark to run to completion without failure.
GB log files are stored in /data/data/com.primatelabs.geekbench/files and
not accessible without root. On Chromebooks ( which have no adb root
rights) it is possible to copy from this folder manually after the test run.
Moved broadcast to super. Mandatory and Default are XOR
Added a longer wait for sync to finish. Increases reliability on certain phones
Changed recipient to not mandatory and a default set
Wait for sync when launching gmail from the sharing feature
Fix: cornercase where image viewer already points to working directory. Refactored code due to duplication
Added new function to BaseUiAutomation class to find a folder in the gallery
Changed termination signal to interrupt signal to prevent code exiting too early.
Added exit handler to ensure revent exits correctly as previously was
crashing and therefore not running final code.
Fixed error in writing input event where half of timestamp seconds was missing.
Fixed typo in documentation for revent file structure.
When collecting ftrace events, the instrument will insert frequencies
into the trace to make it possible reconstruct power states when there
were no frequency transitions during the measured period.
The format in which frequencies are inserted is different in devlib.
Since post-processing scripts may be run on traces collected by devlib
as well as WA, it needs to support both formats.
Fixed a bug, where "ui" runtime parameter was being set regardless of
the underlying platforms for "freq" iterations, causing the to fail on
non-ChomeOS devices.
Executables were not previously automatically downloaded from a
provided remote_assets_url. The http_getter is now called to attempt to
find and download executable files.
Ensure the device log file is deleted between runs and that the host file is not kept as an
artefact. Both can lead to errors appending data from the next iteration/run.
produces a different results output.
- Add UiAutomator and results parsing support for GB4
- Remove the version checking in init.py and rely instead on the ApkWorkload version
checking. The required version in the agenda is checked against that on the host or target
Tested with GB3.4.1 and GB4.0.0 on a Galaxy S7. Older variants of the APK were not available.
Added cornercase of accidentially triggering a card popup
New version includes a new dialogue and name for search. Added support for this
If device has more than one account associated, a new popup appears. Handle that with an optional parameter listing specific account to select, if ommitted the first account in the list is selected
- Corrected the depency for opencv bindings. Even though the library is
imported as "cv2", the package containing it is called
"opencv-python".
- pylint: ignore no-member warnings; cv2 pulls it's memebers dynamically
from the underlying libopencv, so pylint can't see them.
Due to inaccuracies in revent playback, the resultant state does not
always match the templates precisely causing state detection to fail.
To help this images are now scaled to different sizes before being
matched to the templates to compensate for slight variations in size.
When using 'check_exit_code' and 'as_root' options for adb_shell with
a command containing single quotes, the provided command was escaped
twice which has now been avoided.
- Updated the revent parser to handle the new revent format.
- Updated the revent parser to expose device info and recording
metatdata.
- The parser can now be used in streaming and non-streaming mode (stream
events from the file as they are being accessed, or read them all in
at once).
- -g option added to "record" command to expose the gamepad recording
mode.
- Added support for gamepad recording. This type of recording contains
only the events from a gamepad device (which is automatically
identified). The details of this devices are collected and stored as
part of the recording. On playback, uinput is used to create a virtual
gamepad and replay into it.
- A "mode" field has been added to the recording format to help
distinguish between the normal and gamepad recording types.
- A field for the total number of input events has been added before the
start of the event stream (after the device description).
- The structure of revent code has undergone a major overhaul to improve
maintainability and robustness.
- More detailed "info" command output.
- Updated Makefile to support debug/production builds.
- Corrected code highlighting and phase names in documentation
- Fixed check_states paremeter not being honoured
- Moved state dependencies check to happen earlier in execution
and to be a user facing error
In order to use state detection with revent, a screen capture
is required for comparison. Added an argument to revent record command to
automatically record a screen capture after the recording is
complete.
- renamed the capture method from "capture_view_hierachy" to
"capture_ui_hierarchy" to fix typo and make more descriptive.
- Change the file extension of the cature file to ".uix" because this is
the extension uiautomatorviewer looks for.
It is possible that the command executed by get_pids_of() will return
None (in cases where there are no running processes with the specified
name and the grep call didn't find anything). If that happens, then the
subsequent call to split() failed (as that is not a method of None). To
avoid this, substitute an empty string instead.
It looks like on recent systems, adb has started to correctly forward
stderr from the target device to stderr on the host (wereas in the past,
it got output to stdout on the host). This commit makes sure that
stderr gets correctly forwarded to the coller in cases where return code
checking was not enabled.
Adds support for passing -d option to "pm install", which allows
installing an APK when a newer version of the package is already present
on the device.
Capitalization for the "LET'S ROLL" button text was incorrect in the
automation code. This has not caused issues up to this point, but it
seems the button is no longer being found in the latest AOSP. This
commit corrects the capitalization.
Conform to code style of other workloads.
Changed search strings to be a list. Allows user the ability to search for 1 or more search terms.
Renamed Reader to AdobeReader. Removed sleep and now detect and wait for searchicon
Image order not guaranteed. Changed workflow to use subfolders to guarantee ordering and have same action be performed on the same image. As a result, swipe gestures are no longer applicable
Colour change was using clicks. Now uses drags. This is more reliable across devices
Previously if trying to downgrade an app using the `-r` option of
`adb install` the `INSTALL_FAILED_VERSION_DOWNGRADE` error would occur.
The app is now uninstalled first to prevent this.
Since there has been an update to the skype app, there was now a
update prompt preventing test from running. Now checks for the
prompt on older versions of skype and dismisses it if present.
Since initilize only runs once per workload, we can't set variables required
for each iteration in initilize. Revent workload was setting its file paths
in initilize, now moved to setup.
APK Resolution is now handled a bit differently to try maximise the likelyhood
of a workload running.
Like before `force_install` will always try to install the host version, if it
is not present or is not a correct version, it will error.
`check_apk` has changed so that when it is `True` it will prefer to use the host
side APK. If it is not there, or not a suitable version and/or abi and the target
already has a correct version of the app, the target app will be used. When it is
to `False` WA will prefer the target version of the app so long as it is a valid
version, if it is not then it will fallback to the host side APK.
In a recent commit workload versions were changed to match their APK versions.
This commit adds the old versions to the allowed versions and automatically
maps them onto the new values.
Constraints and allowed values of Extension Parameters will now be check
when the Parameter value is set, rather than when validating the
extension. Mandatory status of a Parameter is still checked during
valudation.
When running the stand-alone cpustates script and specifying a timeline
file (which causes the corresponding reporter to be enabled), a
timeline report is generated in addition to the usual cpustates and
parallelism reports.
Up to this point, the main() of the stand-alone script was expecting
exactly two reports and so it crashing when running with the timeline
option. This commit fixes this case.
Super works by having a iterator of parent classes to ensure each is called
once and only once. WA calls some parent methods in classes with multiple
inheritenceconditionally so calls them directly instread. This breaks super
which ends up calling some methods multiple times.
To work around this until workloads are reworked to use composition rather than
inheritance, all classes that that subclass `Workload` directly no longer use
`super`.
Implement new method to enforce that a valid apk version is used to run
the workload. Based on a min and max range of apk versions tested
during development.
- Device subclasses should provide their own implementation.
Default behaviour is to raise a `NotImplementedError`
- Workload subclasses can set `requires_network` to `True` and
network connectivity check will be performed during `setup()`
New method to force a re-index of the mediaserver cache for the
specified directory. Used in workloads that require external media files
as dependencies.
It has been observed on some devices that the FINE_LOCATION premissions is required
for antutu to run without asking for permissions at run time but this was not listed
in the APK manifest. This caused issues on devices were only the permissions in the
manifest can be granted. This commit sliences any error when trying to set this permission
as well as only trying only on Android 6+
CalledProcessErrorWithStderr is a subclass of CalledProcessError that
also takes stderr output. Both that and normal output must be passed as
keyword arguments. They were being passed as keyword arguments inside
_scp() of SshConnection, causing cryptic errors to appear.
Additionally, "output" was not being properly poped off before invoking
super's init.
Add method to automatically check against a dictionary of known package
versions that don't work well with AndroidUiAutoBenchmark workloads.
Raises an exception if found.
- Added statedetect.py in utils which is a standalone module that
contains all the methods needed for state detection
- Modified the setup() and run() methods of the GameWorkload class
in common/android/workload.py to have a parameter that enables
state checks and run the check after setup and run if requested.
State detection uses the template matching method available in
OpenCV to determine the state of the workload by detecting
predefined unique elements on a screenshot from the device.
Dump window hierarchy view from uiautomator to a file when WA fails
during execution. Note: the xml file are pre-formatted after dump.
Implementation specific to android.device.
Issue: On some devices, _grant_requested_permissions may throw an
error when trying to grant permissions that were already granted
at install time using 'adb install -g'.
Fix: Surround permission grant method for API 23+ with try/except
Issue: Currently, _grant_requested_permissions skips all lines after
the first line not starting with 'android.permission'. This causes
a problem for apps where some required permissions appear after a
non-matching line e.g. Google Slides.
Fix: Don't break on non-matching line until end of section is reached
The uxperf result processor now provides warnings for unmatched UX_PERF
markers when running the fps instrument. Previously unmatched markers
resulted in an exception being thrown. Includes additional logging for
debugging purposes.
Add a general check that pings a IP address rather than checking the
status of dumpsys wifi due to the fact that not all devices are
connected via wifi. Intended for workloads that require an internet
connection to operate.
Add a new parameter to fps instrument to specify the time period between
calls to ``dumpsys SurfaceFlinger --latency`` in seconds when collecting
frame data. A lower value improves the granularity of timings when
recording actions for UX Performance metrics.
