To reflect that the current implementation of ReventWorkload uses an APK
file on the device the name has been changed appropriately and the
relevant workloads have been updated.
Changes from checking if there are deployable assets to checking for
discovered assets. This allows for automated pushing of assets without
invoking the resource getters, for example pre discovered files.
Mandate that either a package name is specified or the owner defines
supported package names. This prevents any apk file in the correct
location incorrectly being resolved.
Some workloads can insert timestamped markers into the logs if this
parameter is enabled in their automation. These markers can then be
used to look at measurements for specific parts of the execution.
These two lists are updated by workloads as assets are discovered and
deployed. As these are defined as class attributes in the base Workload
class, the same two list instances are shared between all classes. This
results in subsequent workloads re-deploying assets from the previous
workloads, and may lead to mismatches with deployable_assets, which is
overwritten on per-workload basis.
Make these into instance attriutes to prevent these issues.
Upon clearing data of a previously installed application the runtime
permissions can also be cleared, to avoid having to reinstall the
application all available permissions are explicitly granted to the
required application.
Allows for automatically pushing a set of arbitrary assets to the device
during initialization and adds a parameter to specify whether the assets should
be removed again at the end of the run.
Some workloads, such as Geekbench, may phone home and report data about the
device they are running on. This poses a risk for users that are testing on
unreleased or otherwise confidential devices - perhaps they use a standard
agenda to run a large battery of tests, in which case they may forget to disable
these dangerous workloads.
This provides a mechanism to prevent running those workloads from running by
setting allow_phone_home=False in the user configuration.
- Add "activity" property to PackageManager exposing the underlying
apk_info attribute.
- Add "view" class attribute to ApkWorkload, initialized to None
- If "view" is not set for a particular workload, set a default based on
the package and activity of the underlying workload.
- do not attempt to replay setup if a setup recording has not been
provided (perviously, setup was mandatory).
- update the apk initialization to the correct method inside the record
command.
Rename "package" parameter to "package_name" (to be cosnsitent with
package_names class attribute). "package" is now a property containting
the name of the actual package used after the APK is resolved.
Previously any apk found would be selected even if the abi did not match
the device, now an apk is only selected if it is compatible with the
device. Additionally the 'exact_abi' parameter has been added to allow only
selecting an apk if a devices primary abi's native code is present in
the apk, or there is no native code.
Due to the fact that uiauto apk files built on different machines will
be signed with different keys, adb will fail to overwrite a previous
version even when set to replace. This commit now will uninstall the
previous uiauto apk file if present before attempting to install the new
version.
Previously initialize and finalize were being called for each iteration
of each workload at the start/end of the run which is incorrect
behaviour. To prevent this, each iteration of a workload now shares a
single instance of the workload combined with the 'once_per_instance' decorator
to ensure that the methods are only invoked once per set of workload
runs.
To support runs that require different versions of an application
the discovery/installation process should be performed at the beginning
of each iteration and therefore has been moved into setup.
In order to be able to pass the package name of the app under test to
the instrumented test, the generation of the uiautomator commands needed
to be postponed until after the apk resolution had been performed.
The starting activity is not always listed in an apk or required to start
an application to its default state, for example youtube. Therefore now the
starting activity will only be used if one is available.
In order to pass arguments to the instrumented test via the command line
they need to be encoded, this commit utilises a ParamDict to ensure
parameters are encoded correctly before passing them as arguments.
This commit removes the UI elements from ApkUIWorkload to provide a base
ApkWorkload class in order to support workloads that do not require a GUI.
It also remove the `@once` decorator as apk resolution may need to be
run more than once per run.
Previously apk resolution would fail if the application apk was not present on
device as this was used to retrieve information e.g. the package name. To work
around this, the resolution process now supports searching the device for either
a list of package names retrieved from the workload or as a parameter passed by
the user at runtime and if found on the device will be pulled to the host to
perform its analysis.
The new method of using uiautomation 2 is by using instrumented apk files rather
than JAR files. This commit updates the base workload has been updated to
install/uninstall the new uiautomation APK files and to invoke the relevant
instrumentation.
Added a new ReventWorkload as a base for revent based workloads.
Updated ReventGui to use `revent_recorder` and now uses `target.model`
instead of `taget.name`.
Added a workload type to handle workloads that have both an APK with an
application and associated automation JAR. Added benchmarkpi
implementation using using the new workload.
- Workload's update_result stage has now been broken up into two parts:
extract_results and update_output. This is to allow the assistant to
pull output from the target in between the two stages.
- Updated assistant implementations for Linux and Android targets from
the exisiting code.
- Extended target descriptor code to handle assistants and their
parameters as well.
- Updated the target manager to actually make use of the assistants.
Changing the way target descriptions work from a static mapping to
something that is dynamically generated and is extensible via plugins.
Also moving core target implementation stuff under "framework".
