sascha/workload-automation
1
0
Fork 0
mirror of https://github.com/ARM-software/workload-automation.git synced 2025-01-31 10:11:17 +00:00
Code Issues Releases Wiki Activity

ApkWorkload: Reworked APK Resolution.

Browse Source 
APK Resolution is now handled a bit differently to try maximise the likelyhood
of a workload running.

Like before `force_install` will always try to install the host version, if it
is not present or is not a correct version, it will error.

`check_apk` has changed so that when it is `True` it will prefer to use the host
side APK. If it is not there, or not a suitable version and/or abi and the target
already has a correct version of the app, the target app will be used. When it is
to `False` WA will prefer the target version of the app so long as it is a valid
version, if it is not then it will fallback to the host side APK.
This commit is contained in:
Sebastian Goscik 2016-09-07 15:26:33 +01:00
parent 1a23bd03a2
commit 486ade6499
3 changed files with 195 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
doc/source/apk_workloads.rst Normal file
View File

@ -0,0 +1,57 @@
.. _apk_workload_settings:
APK Workloads
=============
APK resolution
--------------
WA has various resource getters that can be configured to locate APK files but for most people APK files
should be kept in the ``$WA_HOME/dependencies/SOME_WORKLOAD/`` directory. (by default
``~/.workload_automation/dependencies/SOME_WORKLOAD/``). The ``WA_HOME`` enviroment variable can be used
to chnage the location of this folder. The APK files need to be put into the corresponding directories
for the workload they belong to. The name of the file can be anything but as explained below may need
to contain certain peices of information.
All ApkWorkloads have parameters that affect the way in which APK files are resolved, ``check_abi``,
``force_install`` and ``check_apk``. Their exact behaviours are outlined below.
.. confval:: check_abi
If this setting is enabled WA's resource resolvers will look for the devices ABI within the file name
e.g. ``calculator_arm65.apk``. By default this setting is disabled since most apks will work across all
devices. You may wish to enable this feature when working with devices that support multiple ABI's (like
64-bit devices that can run 32-bit APK files) and are specifically trying to test one or the other.
.. confval:: force_install
If this setting is enabled WA will *always* use the APK file on the host, and re-install it on every
iteration. If there is no APK on the host that is a suitable version and/or ABI for the workload WA
will error when ``force_install`` is enabled.
.. confval:: check_apk
This parameter is used to specify a preference over host or target versions of the app. When set to
``True`` WA will prefer the host side version of the APK. It will check if the host has the APK and
if the host APK meets the version requirements of the workload. If does and the target already has
same version nothing will be done, other wise it will overwrite the targets app with the host version.
If the hosts is missing the APK or it does not meet version requirements WA will fall back to the app
on the target if it has the app and it is of a suitable version. When this parameter is set to
``false`` WA will prefer to use the version already on the target if it meets the workloads version
requirements. If it does not it will fall back to search the host for the correct version. In both modes
if neither the host nor target have a suitable version, WA will error and not run the workload.
Some workloads will also feature the follow parameters which will alter the way their APK files are resolved.
.. confval:: version
This parameter is used to specify which version of uiautomation for the workload is used. In some workloads
e.g. ``geekbench`` multiple versions with drastically different UI's are supported. When a workload uses a
version it is required for the APK file to contain the uiautomation version in the file name. In the case
of antutu the file names could be: ``geekbench_2.apk`` or ``geekbench_3.apk``.
.. confval:: variant_name
Some workloads use variants of APK files, this is usually the case with web browser APK files, these work
in exactly the same way as the version, the variant of the apk

1
doc/source/index.rst
View File

@ -118,6 +118,7 @@ and detailed descriptions of how WA functions under the hood.
additional_topics
daq_device_setup
revent
apk_workloads
contributing
API Reference

210
wlauto/common/android/workload.py
View File

@ -25,7 +25,7 @@ from wlauto.core.workload import Workload
from wlauto.core.resource import NO_ONE
from wlauto.common.android.resources import ApkFile
from wlauto.common.resources import ExtensionAsset, Executable, File
from wlauto.exceptions import WorkloadError, ResourceError, ConfigError, DeviceError
from wlauto.exceptions import WorkloadError, ResourceError, DeviceError
from wlauto.utils.android import ApkInfo, ANDROID_NORMAL_PERMISSIONS, UNSUPPORTED_PACKAGES
from wlauto.utils.types import boolean
from wlauto.utils.revent import ReventParser
@ -35,12 +35,12 @@ import wlauto.common.android.resources
DELAY = 5
# Due to the way `super` works you have to call it at every level but WA executes some
# methods conditionally and so has to do them directly via the class, this breaks super
# methods conditionally and so has to call them directly via the class, this breaks super
# and causes it to run things mutiple times ect. As a work around for this untill workloads
# are reworked everything that subclasses workload calls parent methods explicitly
class UiAutomatorWorkload(Workload):
"""
Base class for all workloads that rely on a UI Automator JAR file.
@ -173,13 +173,16 @@ class ApkWorkload(Workload):
description='Timeout for the installation of the apk.'),
Parameter('check_apk', kind=boolean, default=True,
description='''
Discover the APK for this workload on the host, and check that
the version matches the one on device (if already installed).
When set to True the APK file on the host will be prefered if
it is a valid version and ABI, if not it will fall back to the
version on the targer. When set to False the target version is
prefered.
'''),
Parameter('force_install', kind=boolean, default=False,
description='''
Always re-install the APK, even if matching version is found already installed
on the device. Runs ``adb install -r`` to ensure existing APK is replaced.
on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When
this is set, check_apk is ignored.
'''),
Parameter('uninstall_apk', kind=boolean, default=False,
description='If ``True``, will uninstall workload\'s APK as part of teardown.'),
@ -199,88 +202,149 @@ class ApkWorkload(Workload):
def setup(self, context):
Workload.setup(self, context)
# Get APK for the correct version and device ABI
# Get target version
target_version = self.device.get_installed_package_version(self.package)
if target_version:
target_version = LooseVersion(target_version)
self.logger.debug("Found version '{}' on target device".format(target_version))
# Get host version
self.apk_file = context.resolver.get(ApkFile(self, self.device.abi),
version=getattr(self, 'version', None),
check_abi=getattr(self, 'check_abi', False),
variant_name=getattr(self, 'variant_name', None),
strict=self.check_apk)
# Validate the APK
if self.check_apk:
if not self.apk_file:
raise WorkloadError('No APK file found for workload {}.'.format(self.name))
else:
strict=False)
host_version = None
if self.apk_file is not None:
host_version = ApkInfo(self.apk_file).version_name
if host_version:
host_version = LooseVersion(host_version)
self.logger.debug("Found version '{}' on host".format(host_version))
# Error if apk was not found anywhere
if target_version is None and host_version is None:
msg = "Could not find APK for '{}' on the host or target device"
raise ResourceError(msg.format(self.name))
# Ensure the apk is setup on the device
if self.force_install:
raise ConfigError('force_install cannot be "True" when check_apk is set to "False".')
self.force_install_apk(context, host_version, target_version)
elif self.check_apk:
self.prefer_host_apk(context, host_version, target_version)
else:
self.prefer_target_apk(context, host_version, target_version)
self.initialize_package(context)
# Check the APK version against the min and max versions compatible
# with the workload before launching the package. Note: must be called
# after initialize_package() to get self.apk_version.
if self.check_apk:
self.check_apk_version()
self.reset(context)
self.apk_version = self.device.get_installed_package_version(self.package)
context.add_classifiers(apk_version=self.apk_version)
if self.launch_main:
self.launch_package() # launch default activity without intent data
self.device.execute('am kill-all') # kill all *background* activities
self.device.clear_logcat()
def initialize_package(self, context):
installed_version = self.device.get_installed_package_version(self.package)
if self.check_apk:
self.initialize_with_host_apk(context, installed_version)
else:
if not installed_version:
message = '''{} not found on the device and check_apk is set to "False"
so host version was not checked.'''
raise WorkloadError(message.format(self.package))
message = 'Version {} installed on device; skipping host APK check.'
self.logger.debug(message.format(installed_version))
self.reset(context)
self.apk_version = installed_version
context.add_classifiers(apk_version=self.apk_version)
def force_install_apk(self, context, host_version, target_version):
if host_version is None:
raise ResourceError("force_install is 'True' but could not find APK on the host")
try:
self.validate_version(host_version)
except ResourceError as e:
msg = "force_install is 'True' but the host version is invalid:\n\t{}"
raise ResourceError(msg.format(str(e)))
self.install_apk(context, replace=(target_version is not None))
def initialize_with_host_apk(self, context, installed_version):
host_version = ApkInfo(self.apk_file).version_name
if installed_version != host_version:
if installed_version:
message = '{} host version: {}, device version: {}; re-installing...'
self.logger.debug(message.format(os.path.basename(self.apk_file),
host_version, installed_version))
def prefer_host_apk(self, context, host_version, target_version):
msg = "check_apk is 'True' "
if host_version is None:
try:
self.validate_version(target_version)
except ResourceError as e:
msg += "but the APK was not found on the host and the target version is invalid:\n\t{}"
raise ResourceError(msg.format(str(e)))
else:
message = '{} host version: {}, not found on device; installing...'
self.logger.debug(message.format(os.path.basename(self.apk_file),
host_version))
self.force_install = True # pylint: disable=attribute-defined-outside-init
msg += "but the APK was not found on the host, using target version"
self.logger.debug(msg)
return
try:
self.validate_version(host_version)
except ResourceError as e1:
msg += "but the host APK version is invalid:\n\t{}\n"
if target_version is None:
msg += "The target does not have the app either"
raise ResourceError(msg.format(str(e1)))
try:
self.validate_version(target_version)
except ResourceError as e2:
msg += "The target version is also invalid:\n\t{}"
raise ResourceError(msg.format(str(e1), str(e2)))
else:
message = '{} version {} found on both device and host.'
self.logger.debug(message.format(os.path.basename(self.apk_file),
host_version))
if self.force_install:
if installed_version:
self.device.uninstall(self.package)
# It's possible that the uninstall above fails, which might result in a warning
# and/or failure during installation. However execution should proceed, so need
# to make sure that the right apk_vesion is reported in the end.
if self.install_apk(context):
self.apk_version = host_version
else:
self.apk_version = installed_version
else:
self.apk_version = installed_version
self.reset(context)
msg += "using the target version instead"
self.logger.debug(msg.format(str(e1)))
else: # Host version is valid
if target_version is not None and target_version == host_version:
msg += " and a matching version is alread on the device, doing nothing"
self.logger.debug(msg)
return
msg += " and the host version is not on the target, installing APK"
self.logger.debug(msg)
self.install_apk(context, replace=(target_version is not None))
def check_apk_version(self):
if self.min_apk_version:
if LooseVersion(self.apk_version) < LooseVersion(self.min_apk_version):
message = "APK version not supported. Minimum version required: {}"
raise WorkloadError(message.format(self.min_apk_version))
def prefer_target_apk(self, context, host_version, target_version):
msg = "check_apk is 'False' "
if target_version is None:
try:
self.validate_version(host_version)
except ResourceError as e:
msg += "but the app was not found on the target and the host version is invalid:\n\t{}"
raise ResourceError(msg.format(str(e)))
else:
msg += "and the app was not found on the target, using host version"
self.logger.debug(msg)
self.install_apk(context)
return
try:
self.validate_version(target_version)
except ResourceError as e1:
msg += "but the target app version is invalid:\n\t{}\n"
if host_version is None:
msg += "The host does not have the APK either"
raise ResourceError(msg.format(str(e1)))
try:
self.validate_version(host_version)
except ResourceError as e2:
msg += "The host version is also invalid:\n\t{}"
raise ResourceError(msg.format(str(e1), str(e2)))
else:
msg += "Using the host APK instead"
self.logger.debug(msg.format(str(e1)))
self.install_apk(context, replace=True)
else:
msg += "and a valid version of the app is already on the target, using target app"
self.logger.debug(msg)
if self.max_apk_version:
if LooseVersion(self.apk_version) > LooseVersion(self.max_apk_version):
message = "APK version not supported. Maximum version supported: {}"
raise WorkloadError(message.format(self.max_apk_version))
def validate_version(self, version):
min_apk_version = getattr(self, 'min_apk_version', None)
max_apk_version = getattr(self, 'max_apk_version', None)
if min_apk_version is not None and max_apk_version is not None:
if version < LooseVersion(min_apk_version) or \
version > LooseVersion(max_apk_version):
msg = "version '{}' not supported. " \
"Minimum version required: '{}', Maximum version known to work: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
elif min_apk_version is not None:
if version < LooseVersion(min_apk_version):
msg = "version '{}' not supported. " \
"Minimum version required: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
elif max_apk_version is not None:
if version > LooseVersion(max_apk_version):
msg = "version '{}' not supported. " \
"Maximum version known to work: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
def launch_package(self):
if not self.activity:
@ -302,9 +366,9 @@ class ApkWorkload(Workload):
if self.device.get_sdk_version() >= 23:
self._grant_requested_permissions()
def install_apk(self, context):
def install_apk(self, context, replace=False):
success = False
output = self.device.install(self.apk_file, self.install_timeout, replace=self.force_install)
output = self.device.install(self.apk_file, self.install_timeout, replace=replace)
if 'Failure' in output:
if 'ALREADY_EXISTS' in output:
self.logger.warn('Using already installed APK (did not unistall properly?)')