Super works by having a iterator of parent classes to ensure each is called
once and only once. WA calls some parent methods in classes with multiple
inheritenceconditionally so calls them directly instread. This breaks super
which ends up calling some methods multiple times.
To work around this until workloads are reworked to use composition rather than
inheritance, all classes that that subclass `Workload` directly no longer use
`super`.
Implement new method to enforce that a valid apk version is used to run
the workload. Based on a min and max range of apk versions tested
during development.
- Device subclasses should provide their own implementation.
Default behaviour is to raise a `NotImplementedError`
- Workload subclasses can set `requires_network` to `True` and
network connectivity check will be performed during `setup()`
New method to force a re-index of the mediaserver cache for the
specified directory. Used in workloads that require external media files
as dependencies.
Add method to automatically check against a dictionary of known package
versions that don't work well with AndroidUiAutoBenchmark workloads.
Raises an exception if found.
- Added statedetect.py in utils which is a standalone module that
contains all the methods needed for state detection
- Modified the setup() and run() methods of the GameWorkload class
in common/android/workload.py to have a parameter that enables
state checks and run the check after setup and run if requested.
State detection uses the template matching method available in
OpenCV to determine the state of the workload by detecting
predefined unique elements on a screenshot from the device.
Dump window hierarchy view from uiautomator to a file when WA fails
during execution. Note: the xml file are pre-formatted after dump.
Implementation specific to android.device.
Issue: On some devices, _grant_requested_permissions may throw an
error when trying to grant permissions that were already granted
at install time using 'adb install -g'.
Fix: Surround permission grant method for API 23+ with try/except
Issue: Currently, _grant_requested_permissions skips all lines after
the first line not starting with 'android.permission'. This causes
a problem for apps where some required permissions appear after a
non-matching line e.g. Google Slides.
Fix: Don't break on non-matching line until end of section is reached
Add a general check that pings a IP address rather than checking the
status of dumpsys wifi due to the fact that not all devices are
connected via wifi. Intended for workloads that require an internet
connection to operate.
- Implement a new Marker API in BaseUiAutomation so workload can
generate start and end markers with string name. Outputs to logcat.
- Document the Marker output log format in the WA documentation
- Create a results processor to take existing instrument fps logs and
parse them based on the workload markers. Produce per-action fps
metrics.
- Add simple timing results based on the workload markers
- Check the APK's versionName property against the workload's
expected version if specified
- If workload specifies check_abi param, try to get APK from
ABI-specific path on the host
- Add variant_name param to APK resource-getter for backwards
compatibility of dex2oat and peacekeeper
Issue: For certain installation errors, it is possible for WA to
incorrectly report that an APK was reinstalled while it actually
wasn't, leading to bugs later on in the run.
Fix:
- Add the '-r' flag to adb install when reinstalling, to make sure
APK is replaced.
- Add '-g' flag for API 23 and higher, to grant all permissions that
would otherwise be requested at runtime (similar to pre-API 23)
- device.is_rooted check was being perfromed inside validate() which
gets invoked before the device is connected. This worked for most
Android devices, because connections is a no-op for them, however
failed for Linux targets. The check is now performed inside
initialize().
- Added _is_ready() check inside is_rooted to catch similar problems
quicker in the future.
Moved ``working_directory`` parameter to BaseLinuxDevice.
Changed the default ``binaries_directory`` for AndroidDevice to allow
WA binaries to be easily separated.
Previously the WA device name was used when searching for revent files.
Since most were `generic_android` this made it difficult to keep revent
files for multiple android devices. Now it the device model is used instead.
If a file with the device model is not found it will fall back to the WA
device name.
kick_off has been changed to behave the same as AndroidDevice.
Said changes caused kill all to fail on rooted devices. Killall will now
behave in the same way as kick_off, if specifically told to (or not to)
run as root it will. Otherwise it will run as root if the device is rooted
These are now gathered via `getprop` rather than trying to parse the
build.prop file directly.
This fixes issues with build.prop files that have imports.
Added splitVersion and compareVersions functions allow versions strings
like "3.2.045" to be compared.
Also fixed the build script to now copy to the correct folder
Some characters would be interpreted by the shell thus breaking the
command. Adding quotes around the parameters solved this.
N.B Space still needs to be replaced.
Due to the previous commits, this command no longer works properly.
It turns out there is an issue with using multiple levels of escaping.
It seems that bash handles the backslashes and single quotes separately
incorrectly processing our escaping. To get around this we are writing the
sqlite command to a shell script file and running that.
This seems to be the only case in WA at the moment that requires this,
if more show up/when WA moves to devlib it should use the devlib shutil
mechanism.
If uninstall fails, "pm clear" should be called to make sure that the
next time the app is launched it starts from a known state (which would
normally be ensured by the uninstall).
It's possible that there is already a version of an app on target that
differs form the version of the apk on the host. In such cases, WA will
usually try to uninstall the target version and install the host
version.
It's possible that the uninstall may fail. If that happens, it will be
reported as a warning but workload exectuion will proceed with the
target version. In this case, apk_version would have already been set to
that of the host apk. This change ensures that the APK version is
correctly set to the target version (the one that actually ran).
