sascha/workload-automation
1
0
Fork 0
mirror of https://github.com/ARM-software/workload-automation.git synced 2025-02-07 13:41:24 +00:00
Code Issues Releases Wiki Activity

ApkWorkload: Reworked APK Resolution.

Browse Source 
APK Resolution is now handled a bit differently to try maximise the likelyhood
of a workload running.

Like before `force_install` will always try to install the host version, if it
is not present or is not a correct version, it will error.

`check_apk` has changed so that when it is `True` it will prefer to use the host
side APK. If it is not there, or not a suitable version and/or abi and the target
already has a correct version of the app, the target app will be used. When it is
to `False` WA will prefer the target version of the app so long as it is a valid
version, if it is not then it will fallback to the host side APK.
This commit is contained in:
Sebastian Goscik 2016-09-07 15:26:33 +01:00
parent 1a23bd03a2
commit 486ade6499
3 changed files with 195 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
doc/source/apk_workloads.rst Normal file
View File

@ -0,0 +1,57 @@
.. _apk_workload_settings:
APK Workloads
=============
APK resolution
--------------
WA has various resource getters that can be configured to locate APK files but for most people APK files
should be kept in the ``$WA_HOME/dependencies/SOME_WORKLOAD/`` directory. (by default
``~/.workload_automation/dependencies/SOME_WORKLOAD/``). The ``WA_HOME`` enviroment variable can be used
to chnage the location of this folder. The APK files need to be put into the corresponding directories
for the workload they belong to. The name of the file can be anything but as explained below may need
to contain certain peices of information.
All ApkWorkloads have parameters that affect the way in which APK files are resolved, ``check_abi``,
``force_install`` and ``check_apk``. Their exact behaviours are outlined below.
.. confval:: check_abi
If this setting is enabled WA's resource resolvers will look for the devices ABI within the file name
e.g. ``calculator_arm65.apk``. By default this setting is disabled since most apks will work across all
devices. You may wish to enable this feature when working with devices that support multiple ABI's (like
64-bit devices that can run 32-bit APK files) and are specifically trying to test one or the other.
.. confval:: force_install
If this setting is enabled WA will *always* use the APK file on the host, and re-install it on every
iteration. If there is no APK on the host that is a suitable version and/or ABI for the workload WA
will error when ``force_install`` is enabled.
.. confval:: check_apk
This parameter is used to specify a preference over host or target versions of the app. When set to
``True`` WA will prefer the host side version of the APK. It will check if the host has the APK and
if the host APK meets the version requirements of the workload. If does and the target already has
same version nothing will be done, other wise it will overwrite the targets app with the host version.
If the hosts is missing the APK or it does not meet version requirements WA will fall back to the app
on the target if it has the app and it is of a suitable version. When this parameter is set to
``false`` WA will prefer to use the version already on the target if it meets the workloads version
requirements. If it does not it will fall back to search the host for the correct version. In both modes
if neither the host nor target have a suitable version, WA will error and not run the workload.
Some workloads will also feature the follow parameters which will alter the way their APK files are resolved.
.. confval:: version
This parameter is used to specify which version of uiautomation for the workload is used. In some workloads
e.g. ``geekbench`` multiple versions with drastically different UI's are supported. When a workload uses a
version it is required for the APK file to contain the uiautomation version in the file name. In the case
of antutu the file names could be: ``geekbench_2.apk`` or ``geekbench_3.apk``.
.. confval:: variant_name
Some workloads use variants of APK files, this is usually the case with web browser APK files, these work
in exactly the same way as the version, the variant of the apk

1
doc/source/index.rst
View File

@ -118,6 +118,7 @@ and detailed descriptions of how WA functions under the hood.
additional_topics additional_topics
daq_device_setup daq_device_setup
revent revent
apk_workloads
contributing contributing
API Reference API Reference

210
wlauto/common/android/workload.py
View File

@ -25,7 +25,7 @@ from wlauto.core.workload import Workload
from wlauto.core.resource import NO_ONE from wlauto.core.resource import NO_ONE
from wlauto.common.android.resources import ApkFile from wlauto.common.android.resources import ApkFile
from wlauto.common.resources import ExtensionAsset, Executable, File from wlauto.common.resources import ExtensionAsset, Executable, File
from wlauto.exceptions import WorkloadError, ResourceError, ConfigError, DeviceError from wlauto.exceptions import WorkloadError, ResourceError, DeviceError
from wlauto.utils.android import ApkInfo, ANDROID_NORMAL_PERMISSIONS, UNSUPPORTED_PACKAGES from wlauto.utils.android import ApkInfo, ANDROID_NORMAL_PERMISSIONS, UNSUPPORTED_PACKAGES
from wlauto.utils.types import boolean from wlauto.utils.types import boolean
from wlauto.utils.revent import ReventParser from wlauto.utils.revent import ReventParser
@ -35,12 +35,12 @@ import wlauto.common.android.resources
DELAY = 5 DELAY = 5
# Due to the way `super` works you have to call it at every level but WA executes some # Due to the way `super` works you have to call it at every level but WA executes some
# methods conditionally and so has to do them directly via the class, this breaks super # methods conditionally and so has to call them directly via the class, this breaks super
# and causes it to run things mutiple times ect. As a work around for this untill workloads # and causes it to run things mutiple times ect. As a work around for this untill workloads
# are reworked everything that subclasses workload calls parent methods explicitly # are reworked everything that subclasses workload calls parent methods explicitly
class UiAutomatorWorkload(Workload): class UiAutomatorWorkload(Workload):
""" """
Base class for all workloads that rely on a UI Automator JAR file. Base class for all workloads that rely on a UI Automator JAR file.
@ -173,13 +173,16 @@ class ApkWorkload(Workload):
description='Timeout for the installation of the apk.'), description='Timeout for the installation of the apk.'),
Parameter('check_apk', kind=boolean, default=True, Parameter('check_apk', kind=boolean, default=True,
description=''' description='''
Discover the APK for this workload on the host, and check that When set to True the APK file on the host will be prefered if
the version matches the one on device (if already installed). it is a valid version and ABI, if not it will fall back to the
version on the targer. When set to False the target version is
prefered.
'''), '''),
Parameter('force_install', kind=boolean, default=False, Parameter('force_install', kind=boolean, default=False,
description=''' description='''
Always re-install the APK, even if matching version is found already installed Always re-install the APK, even if matching version is found already installed
on the device. Runs ``adb install -r`` to ensure existing APK is replaced. on the device. Runs ``adb install -r`` to ensure existing APK is replaced. When
this is set, check_apk is ignored.
'''), '''),
Parameter('uninstall_apk', kind=boolean, default=False, Parameter('uninstall_apk', kind=boolean, default=False,
description='If ``True``, will uninstall workload\'s APK as part of teardown.'), description='If ``True``, will uninstall workload\'s APK as part of teardown.'),
@ -199,88 +202,149 @@ class ApkWorkload(Workload):
def setup(self, context): def setup(self, context):
Workload.setup(self, context) Workload.setup(self, context)
# Get APK for the correct version and device ABI
# Get target version
target_version = self.device.get_installed_package_version(self.package)
if target_version:
target_version = LooseVersion(target_version)
self.logger.debug("Found version '{}' on target device".format(target_version))
# Get host version
self.apk_file = context.resolver.get(ApkFile(self, self.device.abi), self.apk_file = context.resolver.get(ApkFile(self, self.device.abi),
version=getattr(self, 'version', None), version=getattr(self, 'version', None),
check_abi=getattr(self, 'check_abi', False), check_abi=getattr(self, 'check_abi', False),
variant_name=getattr(self, 'variant_name', None), variant_name=getattr(self, 'variant_name', None),
strict=self.check_apk) strict=False)
# Validate the APK host_version = None
if self.check_apk: if self.apk_file is not None:
if not self.apk_file: host_version = ApkInfo(self.apk_file).version_name
raise WorkloadError('No APK file found for workload {}.'.format(self.name)) if host_version:
host_version = LooseVersion(host_version)
self.logger.debug("Found version '{}' on host".format(host_version))
# Error if apk was not found anywhere
if target_version is None and host_version is None:
msg = "Could not find APK for '{}' on the host or target device"
raise ResourceError(msg.format(self.name))
# Ensure the apk is setup on the device
if self.force_install:
self.force_install_apk(context, host_version, target_version)
elif self.check_apk:
self.prefer_host_apk(context, host_version, target_version)
else: else:
if self.force_install: self.prefer_target_apk(context, host_version, target_version)
raise ConfigError('force_install cannot be "True" when check_apk is set to "False".')
self.initialize_package(context) self.reset(context)
self.apk_version = self.device.get_installed_package_version(self.package)
# Check the APK version against the min and max versions compatible context.add_classifiers(apk_version=self.apk_version)
# with the workload before launching the package. Note: must be called
# after initialize_package() to get self.apk_version.
if self.check_apk:
self.check_apk_version()
if self.launch_main: if self.launch_main:
self.launch_package() # launch default activity without intent data self.launch_package() # launch default activity without intent data
self.device.execute('am kill-all') # kill all *background* activities self.device.execute('am kill-all') # kill all *background* activities
self.device.clear_logcat() self.device.clear_logcat()
def initialize_package(self, context): def force_install_apk(self, context, host_version, target_version):
installed_version = self.device.get_installed_package_version(self.package) if host_version is None:
if self.check_apk: raise ResourceError("force_install is 'True' but could not find APK on the host")
self.initialize_with_host_apk(context, installed_version) try:
else: self.validate_version(host_version)
if not installed_version: except ResourceError as e:
message = '''{} not found on the device and check_apk is set to "False" msg = "force_install is 'True' but the host version is invalid:\n\t{}"
so host version was not checked.''' raise ResourceError(msg.format(str(e)))
raise WorkloadError(message.format(self.package)) self.install_apk(context, replace=(target_version is not None))
message = 'Version {} installed on device; skipping host APK check.'
self.logger.debug(message.format(installed_version))
self.reset(context)
self.apk_version = installed_version
context.add_classifiers(apk_version=self.apk_version)
def initialize_with_host_apk(self, context, installed_version): def prefer_host_apk(self, context, host_version, target_version):
host_version = ApkInfo(self.apk_file).version_name msg = "check_apk is 'True' "
if installed_version != host_version: if host_version is None:
if installed_version: try:
message = '{} host version: {}, device version: {}; re-installing...' self.validate_version(target_version)
self.logger.debug(message.format(os.path.basename(self.apk_file), except ResourceError as e:
host_version, installed_version)) msg += "but the APK was not found on the host and the target version is invalid:\n\t{}"
raise ResourceError(msg.format(str(e)))
else: else:
message = '{} host version: {}, not found on device; installing...' msg += "but the APK was not found on the host, using target version"
self.logger.debug(message.format(os.path.basename(self.apk_file), self.logger.debug(msg)
host_version)) return
self.force_install = True # pylint: disable=attribute-defined-outside-init try:
else: self.validate_version(host_version)
message = '{} version {} found on both device and host.' except ResourceError as e1:
self.logger.debug(message.format(os.path.basename(self.apk_file), msg += "but the host APK version is invalid:\n\t{}\n"
host_version)) if target_version is None:
if self.force_install: msg += "The target does not have the app either"
if installed_version: raise ResourceError(msg.format(str(e1)))
self.device.uninstall(self.package) try:
# It's possible that the uninstall above fails, which might result in a warning self.validate_version(target_version)
# and/or failure during installation. However execution should proceed, so need except ResourceError as e2:
# to make sure that the right apk_vesion is reported in the end. msg += "The target version is also invalid:\n\t{}"
if self.install_apk(context): raise ResourceError(msg.format(str(e1), str(e2)))
self.apk_version = host_version
else: else:
self.apk_version = installed_version msg += "using the target version instead"
self.logger.debug(msg.format(str(e1)))
else: # Host version is valid
if target_version is not None and target_version == host_version:
msg += " and a matching version is alread on the device, doing nothing"
self.logger.debug(msg)
return
msg += " and the host version is not on the target, installing APK"
self.logger.debug(msg)
self.install_apk(context, replace=(target_version is not None))
def prefer_target_apk(self, context, host_version, target_version):
msg = "check_apk is 'False' "
if target_version is None:
try:
self.validate_version(host_version)
except ResourceError as e:
msg += "but the app was not found on the target and the host version is invalid:\n\t{}"
raise ResourceError(msg.format(str(e)))
else:
msg += "and the app was not found on the target, using host version"
self.logger.debug(msg)
self.install_apk(context)
return
try:
self.validate_version(target_version)
except ResourceError as e1:
msg += "but the target app version is invalid:\n\t{}\n"
if host_version is None:
msg += "The host does not have the APK either"
raise ResourceError(msg.format(str(e1)))
try:
self.validate_version(host_version)
except ResourceError as e2:
msg += "The host version is also invalid:\n\t{}"
raise ResourceError(msg.format(str(e1), str(e2)))
else:
msg += "Using the host APK instead"
self.logger.debug(msg.format(str(e1)))
self.install_apk(context, replace=True)
else: else:
self.apk_version = installed_version msg += "and a valid version of the app is already on the target, using target app"
self.reset(context) self.logger.debug(msg)
def check_apk_version(self): def validate_version(self, version):
if self.min_apk_version: min_apk_version = getattr(self, 'min_apk_version', None)
if LooseVersion(self.apk_version) < LooseVersion(self.min_apk_version): max_apk_version = getattr(self, 'max_apk_version', None)
message = "APK version not supported. Minimum version required: {}"
raise WorkloadError(message.format(self.min_apk_version))
if self.max_apk_version: if min_apk_version is not None and max_apk_version is not None:
if LooseVersion(self.apk_version) > LooseVersion(self.max_apk_version): if version < LooseVersion(min_apk_version) or \
message = "APK version not supported. Maximum version supported: {}" version > LooseVersion(max_apk_version):
raise WorkloadError(message.format(self.max_apk_version)) msg = "version '{}' not supported. " \
"Minimum version required: '{}', Maximum version known to work: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
elif min_apk_version is not None:
if version < LooseVersion(min_apk_version):
msg = "version '{}' not supported. " \
"Minimum version required: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
elif max_apk_version is not None:
if version > LooseVersion(max_apk_version):
msg = "version '{}' not supported. " \
"Maximum version known to work: '{}'"
raise ResourceError(msg.format(version, min_apk_version))
def launch_package(self): def launch_package(self):
if not self.activity: if not self.activity:
@ -302,9 +366,9 @@ class ApkWorkload(Workload):
if self.device.get_sdk_version() >= 23: if self.device.get_sdk_version() >= 23:
self._grant_requested_permissions() self._grant_requested_permissions()
def install_apk(self, context): def install_apk(self, context, replace=False):
success = False success = False
output = self.device.install(self.apk_file, self.install_timeout, replace=self.force_install) output = self.device.install(self.apk_file, self.install_timeout, replace=replace)
if 'Failure' in output: if 'Failure' in output:
if 'ALREADY_EXISTS' in output: if 'ALREADY_EXISTS' in output:
self.logger.warn('Using already installed APK (did not unistall properly?)') self.logger.warn('Using already installed APK (did not unistall properly?)')