sascha/workload-automation
1
0
Fork 0
mirror of https://github.com/ARM-software/workload-automation.git synced 2025-01-19 04:21:17 +00:00
Code Issues Releases Wiki Activity
workload-automation/doc/source/writing_extensions.rst

1032 lines
43 KiB
ReStructuredText
Raw Blame History

.. _writing_plugins:
==================
Writing Plugins
==================
Workload Automation offers several plugin points (or plugin types).The most
interesting of these are
:workloads: These are the tasks that get executed and measured on the device. These
can be benchmarks, high-level use cases, or pretty much anything else.
:devices: These are interfaces to the physical devices (development boards or end-user
devices, such as smartphones) that use cases run on. Typically each model of a
physical device would require its own interface class (though some functionality
may be reused by subclassing from an existing base).
:instruments: Instruments allow collecting additional data from workload execution (e.g.
system traces). Instruments are not specific to a particular Workload. Instruments
can hook into any stage of workload execution.
:result processors: These are used to format the results of workload execution once they have been
collected. Depending on the callback used, these will run either after each
iteration or at the end of the run, after all of the results have been
collected.
You create an plugin by subclassing the appropriate base class, defining
appropriate methods and attributes, and putting the .py file with the class into
an appropriate subdirectory under ``~/.workload_automation`` (there is one for
each plugin type).
Plugin Basics
================
This sub-section covers things common to implementing plugins of all types.
It is recommended you familiarize yourself with the information here before
proceeding onto guidance for specific plugin types.
To create an plugin, you basically subclass an appropriate base class and them
implement the appropriate methods
The Context
-----------
The majority of methods in plugins accept a context argument. This is an
instance of :class:`wlauto.core.execution.ExecutionContext`. If contains
of information about current state of execution of WA and keeps track of things
like which workload is currently running and the current iteration.
Notable attributes of the context are
context.spec
the current workload specification being executed. This is an
instance of :class:`wlauto.core.configuration.WorkloadRunSpec`
and defines the workload and the parameters under which it is
being executed.
context.workload
``Workload`` object that is currently being executed.
context.current_iteration
The current iteration of the spec that is being executed. Note that this
is the iteration for that spec, i.e. the number of times that spec has
been run, *not* the total number of all iterations have been executed so
far.
context.result
This is the result object for the current iteration. This is an instance
of :class:`wlauto.core.result.IterationResult`. It contains the status
of the iteration as well as the metrics and artifacts generated by the
workload and enable instrumentation.
context.device
The device interface object that can be used to interact with the
device. Note that workloads and instruments have their own device
attribute and they should be using that instead.
In addition to these, context also defines a few useful paths (see below).
Paths
-----
You should avoid using hard-coded absolute paths in your plugins whenever
possible, as they make your code too dependent on a particular environment and
may mean having to make adjustments when moving to new (host and/or device)
platforms. To help avoid hard-coded absolute paths, WA automation defines
a number of standard locations. You should strive to define your paths relative
to one of those.
On the host
~~~~~~~~~~~
Host paths are available through the context object, which is passed to most
plugin methods.
context.run_output_directory
This is the top-level output directory for all WA results (by default,
this will be "wa_output" in the directory in which WA was invoked.
context.output_directory
This is the output directory for the current iteration. This will an
iteration-specific subdirectory under the main results location. If
there is no current iteration (e.g. when processing overall run results)
this will point to the same location as ``root_output_directory``.
context.host_working_directory
This an addition location that may be used by plugins to store
non-iteration specific intermediate files (e.g. configuration).
Additionally, the global ``wlauto.settings`` object exposes on other location:
settings.dependency_directory
this is the root directory for all plugin dependencies (e.g. media
files, assets etc) that are not included within the plugin itself.
As per Python best practice, it is recommended that methods and values in
``os.path`` standard library module are used for host path manipulation.
On the device
~~~~~~~~~~~~~
Workloads and instruments have a ``device`` attribute, which is an interface to
the device used by WA. It defines the following location:
device.working_directory
This is the directory for all WA-related files on the device. All files
deployed to the device should be pushed to somewhere under this location
(the only exception being executables installed with ``device.install``
method).
Since there could be a mismatch between path notation used by the host and the
device, the ``os.path`` modules should *not* be used for on-device path
manipulation. Instead device has an equipment module exposed through
``device.path`` attribute. This has all the same attributes and behaves the
same way as ``os.path``, but is guaranteed to produce valid paths for the device,
irrespective of the host's path notation. For example:
.. code:: python
result_file = self.device.path.join(self.device.working_directory, "result.txt")
self.command = "{} -a -b -c {}".format(target_binary, result_file)
.. note:: result processors, unlike workloads and instruments, do not have their
own device attribute; however they can access the device through the
context.
Deploying executables to a device
---------------------------------
Some devices may have certain restrictions on where executable binaries may be
placed and how they should be invoked. To ensure your plugin works with as
wide a range of devices as possible, you should use WA APIs for deploying and
invoking executables on a device, as outlined below.
As with other resources (see :ref:`resources`) , host-side paths to the exectuable
binary to be deployed should be obtained via the resource resolver. A special
resource type, ``Executable`` is used to identify a binary to be deployed.
This is simiar to the regular ``File`` resource, however it takes an additional
parameter that specifies the ABI for which executable was compiled.
In order for the binary to be obtained in this way, it must be stored in one of
the locations scanned by the resource resolver in a directry structure
``<root>/bin/<abi>/<binary>`` (where ``root`` is the base resource location to
be searched, e.g. ``~/.workload_automation/depencencies/<plugin name>``, and
``<abi>`` is the ABI for which the exectuable has been compiled, as returned by
``self.device.abi``).
Once the path to the host-side binary has been obtained, it may be deployed using
one of two methods of a ``Device`` instace -- ``install`` or ``install_if_needed``.
The latter will check a version of that binary has been perviously deployed by
WA and will not try to re-install.
.. code:: python
from wlauto import Executable
host_binary = context.resolver.get(Executable(self, self.device.abi, 'some_binary'))
target_binary = self.device.install_if_needed(host_binary)
.. note:: Please also note that the check is done based solely on the binary name.
For more information please see: :func:`wlauto.common.linux.BaseLinuxDevice.install_if_needed`
Both of the above methods will return the path to the installed binary on the
device. The executable should be invoked *only* via that path; do **not** assume
that it will be in ``PATH`` on the target (or that the executable with the same
name in ``PATH`` is the version deployed by WA.
.. code:: python
self.command = "{} -a -b -c".format(target_binary)
self.device.execute(self.command)
Parameters
----------
All plugins can be parameterized. Parameters are specified using
``parameters`` class attribute. This should be a list of
:class:`wlauto.core.Parameter` instances. The following attributes can be
specified on parameter creation:
name
This is the only mandatory argument. The name will be used to create a
corresponding attribute in the plugin instance, so it must be a valid
Python identifier.
kind
This is the type of the value of the parameter. This could be an
callable. Normally this should be a standard Python type, e.g. ``int`
or ``float``, or one the types defined in :mod:`wlauto.utils.types`.
If not explicitly specified, this will default to ``str``.
.. note:: Irrespective of the ``kind`` specified, ``None`` is always a
valid value for a parameter. If you don't want to allow
``None``, then set ``mandatory`` (see below) to ``True``.
allowed_values
A list of the only allowed values for this parameter.
.. note:: For composite types, such as ``list_of_strings`` or
``list_of_ints`` in :mod:`wlauto.utils.types`, each element of
the value will be checked against ``allowed_values`` rather
than the composite value itself.
default
The default value to be used for this parameter if one has not been
specified by the user. Defaults to ``None``.
mandatory
A ``bool`` indicating whether this parameter is mandatory. Setting this
to ``True`` will make ``None`` an illegal value for the parameter.
Defaults to ``False``.
.. note:: Specifying a ``default`` will mean that this parameter will,
effectively, be ignored (unless the user sets the param to ``None``).
.. note:: Mandatory parameters are *bad*. If at all possible, you should
strive to provide a sensible ``default`` or to make do without
the parameter. Only when the param is absolutely necessary,
and there really is no sensible default that could be given
(e.g. something like login credentials), should you consider
making it mandatory.
constraint
This is an additional constraint to be enforced on the parameter beyond
its type or fixed allowed values set. This should be a predicate (a function
that takes a single argument -- the user-supplied value -- and returns
a ``bool`` indicating whether the constraint has been satisfied).
override
A parameter name must be unique not only within an plugin but also
with that plugin's class hierarchy. If you try to declare a parameter
with the same name as already exists, you will get an error. If you do
want to override a parameter from further up in the inheritance
hierarchy, you can indicate that by setting ``override`` attribute to
``True``.
When overriding, you do not need to specify every other attribute of the
parameter, just the ones you what to override. Values for the rest will
be taken from the parameter in the base class.
Validation and cross-parameter constraints
------------------------------------------
An plugin will get validated at some point after constructions. When exactly
this occurs depends on the plugin type, but it *will* be validated before it
is used.
You can implement ``validate`` method in your plugin (that takes no arguments
beyond the ``self``) to perform any additions *internal* validation in your
plugin. By "internal", I mean that you cannot make assumptions about the
surrounding environment (e.g. that the device has been initialized).
The contract for ``validate`` method is that it should raise an exception
(either ``wlauto.exceptions.ConfigError`` or plugin-specific exception type -- see
further on this page) if some validation condition has not, and cannot, been met.
If the method returns without raising an exception, then the plugin is in a
valid internal state.
Note that ``validate`` can be used not only to verify, but also to impose a
valid internal state. In particular, this where cross-parameter constraints can
be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
another parameter, there is no way to express that declaratively when specifying
the parameters. In that case the dependent attribute should be left unspecified
on creation and should instead be set inside ``validate``.
Logging
-------
Every plugin class has it's own logger that you can access through
``self.logger`` inside the plugin's methods. Generally, a :class:`Device` will log
everything it is doing, so you shouldn't need to add much additional logging in
your expansion's. But you might what to log additional information, e.g.
what settings your plugin is using, what it is doing on the host, etc.
Operations on the host will not normally be logged, so your plugin should
definitely log what it is doing on the host. One situation in particular where
you should add logging is before doing something that might take a significant amount
of time, such as downloading a file.
Documenting
-----------
All plugins and their parameter should be documented. For plugins
themselves, this is done through ``description`` class attribute. The convention
for an plugin description is that the first paragraph should be a short
summary description of what the plugin does and why one would want to use it
(among other things, this will get extracted and used by ``wa list`` command).
Subsequent paragraphs (separated by blank lines) can then provide a more
detailed description, including any limitations and setup instructions.
For parameters, the description is passed as an argument on creation. Please
note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
parameter, they do not need to be explicitly mentioned in the description (wa
documentation utilities will automatically pull those). If the ``default`` is set
in ``validate`` or additional cross-parameter constraints exist, this *should*
be documented in the parameter description.
Both plugins and their parameters should be documented using reStructureText
markup (standard markup for Python documentation). See:
http://docutils.sourceforge.net/rst.html
Aside from that, it is up to you how you document your plugin. You should try
to provide enough information so that someone unfamiliar with your plugin is
able to use it, e.g. you should document all settings and parameters your
plugin expects (including what the valid value are).
Error Notification
------------------
When you detect an error condition, you should raise an appropriate exception to
notify the user. The exception would typically be :class:`ConfigError` or
(depending the type of the plugin)
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`ResultProcessorError`.
All these errors are defined in :mod:`wlauto.exception` module.
:class:`ConfigError` should be raised where there is a problem in configuration
specified by the user (either through the agenda or config files). These errors
are meant to be resolvable by simple adjustments to the configuration (and the
error message should suggest what adjustments need to be made. For all other
errors, such as missing dependencies, mis-configured environment, problems
performing operations, etc., the plugin type-specific exceptions should be
used.
If the plugin itself is capable of recovering from the error and carrying
on, it may make more sense to log an ERROR or WARNING level message using the
plugin's logger and to continue operation.
Utils
-----
Workload Automation defines a number of utilities collected under
:mod:`wlauto.utils` subpackage. These utilities were created to help with the
implementation of the framework itself, but may be also be useful when
implementing plugins.
Adding a Workload
=================
.. note:: You can use ``wa create workload [name]`` script to generate a new workload
structure for you. This script can also create the boilerplate for
UI automation, if your workload needs it. See ``wa create -h`` for more
details.
New workloads can be added by subclassing :class:`wlauto.core.workload.Workload`
The Workload class defines the following interface::
class Workload(Plugin):
name = None
def init_resources(self, context):
pass
def validate(self):
pass
def initialize(self, context):
pass
def setup(self, context):
pass
def setup(self, context):
pass
def run(self, context):
pass
def update_result(self, context):
pass
def teardown(self, context):
pass
def finalize(self, context):
pass
.. note:: Please see :doc:`conventions` section for notes on how to interpret
this.
The interface should be implemented as follows
:name: This identifies the workload (e.g. it used to specify it in the
agenda_.
:init_resources: This method may be optionally override to implement dynamic
resource discovery for the workload. This method executes
early on, before the device has been initialized, so it
should only be used to initialize resources that do not
depend on the device to resolve. This method is executed
once per run for each workload instance.
:validate: This method can be used to validate any assumptions your workload
makes about the environment (e.g. that required files are
present, environment variables are set, etc) and should raise
a :class:`wlauto.exceptions.WorkloadError` if that is not the
case. The base class implementation only makes sure sure that
the name attribute has been set.
:initialize: This method will be executed exactly once per run (no matter
how many instances of the workload there are). It will run
after the device has been initialized, so it may be used to
perform device-dependent initialization that does not need to
be repeated on each iteration (e.g. as installing executables
required by the workload on the device).
:setup: Everything that needs to be in place for workload execution should
be done in this method. This includes copying files to the device,
starting up an application, configuring communications channels,
etc.
:run: This method should perform the actual task that is being measured.
When this method exits, the task is assumed to be complete.
.. note:: Instrumentation is kicked off just before calling this
method and is disabled right after, so everything in this
method is being measured. Therefore this method should
contain the least code possible to perform the operations
you are interested in measuring. Specifically, things like
installing or starting applications, processing results, or
copying files to/from the device should be done elsewhere if
possible.
:update_result: This method gets invoked after the task execution has
finished and should be used to extract metrics and add them
to the result (see below).
:teardown: This could be used to perform any cleanup you may wish to do,
e.g. Uninstalling applications, deleting file on the device, etc.
:finalize: This is the complement to ``initialize``. This will be executed
exactly once at the end of the run. This should be used to
perform any final clean up (e.g. uninstalling binaries installed
in the ``initialize``).
.. _agenda: agenda.html
Workload methods (except for ``validate``) take a single argument that is a
:class:`wlauto.core.execution.ExecutionContext` instance. This object keeps
track of the current execution state (such as the current workload, iteration
number, etc), and contains, among other things, a
:class:`wlauto.core.workload.WorkloadResult` instance that should be populated
from the ``update_result`` method with the results of the execution. ::
# ...
def update_result(self, context):
# ...
context.result.add_metric('energy', 23.6, 'Joules', lower_is_better=True)
# ...
Example
-------
This example shows a simple workload that times how long it takes to compress a
file of a particular size on the device.
.. note:: This is intended as an example of how to implement the Workload
interface. The methodology used to perform the actual measurement is
not necessarily sound, and this Workload should not be used to collect
real measurements.
.. code-block:: python
import os
from wlauto import Workload, Parameter
class ZiptestWorkload(Workload):
name = 'ziptest'
description = '''
Times how long it takes to gzip a file of a particular size on a device.
This workload was created for illustration purposes only. It should not be
used to collect actual measurements.
'''
parameters = [
Parameter('file_size', kind=int, default=2000000,
description='Size of the file (in bytes) to be gzipped.')
]
def setup(self, context):
# Generate a file of the specified size containing random garbage.
host_infile = os.path.join(context.output_directory, 'infile')
command = 'openssl rand -base64 {} > {}'.format(self.file_size, host_infile)
os.system(command)
# Set up on-device paths
devpath = self.device.path # os.path equivalent for the device
self.device_infile = devpath.join(self.device.working_directory, 'infile')
self.device_outfile = devpath.join(self.device.working_directory, 'outfile')
# Push the file to the device
self.device.push(host_infile, self.device_infile)
def run(self, context):
self.device.execute('cd {} && (time gzip {}) &>> {}'.format(self.device.working_directory,
self.device_infile,
self.device_outfile))
def update_result(self, context):
# Pull the results file to the host
host_outfile = os.path.join(context.output_directory, 'outfile')
self.device.pull(self.device_outfile, host_outfile)
# Extract metrics form the file's contents and update the result
# with them.
content = iter(open(host_outfile).read().strip().split())
for value, metric in zip(content, content):
mins, secs = map(float, value[:-1].split('m'))
context.result.add_metric(metric, secs + 60 * mins)
def teardown(self, context):
# Clean up on-device file.
self.device.remove(self.device_infile)
self.device.remove(self.device_outfile)
.. _GameWorkload:
Adding revent-dependent Workload:
---------------------------------
:class:`wlauto.common.game.GameWorkload` is the base class for all the workloads
that depend on :ref:`revent_files_creation` files. It implements all the methods
needed to push the files to the device and run them. New GameWorkload can be
added by subclassing :class:`wlauto.common.game.GameWorkload`:
The GameWorkload class defines the following interface::
class GameWorkload(Workload):
name = None
package = None
activity = None
The interface should be implemented as follows
:name: This identifies the workload (e.g. it used to specify it in the
agenda_.
:package: This is the name of the '.apk' package without its file plugin.
:activity: The name of the main activity that runs the package.
Example:
--------
This example shows a simple GameWorkload that plays a game.
.. code-block:: python
from wlauto.common.game import GameWorkload
class MyGame(GameWorkload):
name = 'mygame'
package = 'com.mylogo.mygame'
activity = 'myActivity.myGame'
Convention for Naming revent Files for :class:`wlauto.common.game.GameWorkload`
-------------------------------------------------------------------------------
There is a convention for naming revent files which you should follow if you
want to record your own revent files. Each revent file must start with the
device name(case sensitive) then followed by a dot '.' then the stage name
then '.revent'. All your custom revent files should reside at
'~/.workload_automation/dependencies/WORKLOAD NAME/'. These are the current
supported stages:
:setup: This stage is where the game is loaded. It is a good place to
record revent here to modify the game settings and get it ready
to start.
:run: This stage is where the game actually starts. This will allow for
more accurate results if the revent file for this stage only
records the game being played.
For instance, to add a custom revent files for a device named mydevice and
a workload name mygame, you create a new directory called mygame in
'~/.workload_automation/dependencies/'. Then you add the revent files for
the stages you want in ~/.workload_automation/dependencies/mygame/::
mydevice.setup.revent
mydevice.run.revent
Any revent file in the dependencies will always overwrite the revent file in the
workload directory. So it is possible for example to just provide one revent for
setup in the dependencies and use the run.revent that is in the workload directory.
Adding an Instrument
====================
Instruments can be used to collect additional measurements during workload
execution (e.g. collect power readings). An instrument can hook into almost any
stage of workload execution. A typical instrument would implement a subset of
the following interface::
class Instrument(Plugin):
name = None
description = None
parameters = [
]
def initialize(self, context):
pass
def setup(self, context):
pass
def start(self, context):
pass
def stop(self, context):
pass
def update_result(self, context):
pass
def teardown(self, context):
pass
def finalize(self, context):
pass
This is similar to a Workload, except all methods are optional. In addition to
the workload-like methods, instruments can define a number of other methods that
will get invoked at various points during run execution. The most useful of
which is perhaps ``initialize`` that gets invoked after the device has been
initialised for the first time, and can be used to perform one-time setup (e.g.
copying files to the device -- there is no point in doing that for each
iteration). The full list of available methods can be found in
:ref:`Signals Documentation <instrumentation_method_map>`.
Prioritization
--------------
Callbacks (e.g. ``setup()`` methods) for all instrumentation get executed at the
same point during workload execution, one after another. The order in which the
callbacks get invoked should be considered arbitrary and should not be relied
on (e.g. you cannot expect that just because instrument A is listed before
instrument B in the config, instrument A's callbacks will run first).
In some cases (e.g. in ``start()`` and ``stop()`` methods), it is important to
ensure that a particular instrument's callbacks run a closely as possible to the
workload's invocations in order to maintain accuracy of readings; or,
conversely, that a callback is executed after the others, because it takes a
long time and may throw off the accuracy of other instrumentation. You can do
this by prepending ``fast_`` or ``slow_`` to your callbacks' names. For
example::
class PreciseInstrument(Instument):
# ...
def fast_start(self, context):
pass
def fast_stop(self, context):
pass
# ...
``PreciseInstrument`` will be started after all other instrumentation (i.e.
*just* before the workload runs), and it will stopped before all other
instrumentation (i.e. *just* after the workload runs). It is also possible to
use ``very_fast_`` and ``very_slow_`` prefixes when you want to be really
sure that your callback will be the last/first to run.
If more than one active instrument have specified fast (or slow) callbacks, then
their execution order with respect to each other is not guaranteed. In general,
having a lot of instrumentation enabled is going to necessarily affect the
readings. The best way to ensure accuracy of measurements is to minimize the
number of active instruments (perhaps doing several identical runs with
different instruments enabled).
Example
-------
Below is a simple instrument that measures the execution time of a workload::
class ExecutionTimeInstrument(Instrument):
"""
Measure how long it took to execute the run() methods of a Workload.
"""
name = 'execution_time'
def initialize(self, context):
self.start_time = None
self.end_time = None
def fast_start(self, context):
self.start_time = time.time()
def fast_stop(self, context):
self.end_time = time.time()
def update_result(self, context):
execution_time = self.end_time - self.start_time
context.result.add_metric('execution_time', execution_time, 'seconds')
Adding a Result Processor
=========================
A result processor is responsible for processing the results. This may
involve formatting and writing them to a file, uploading them to a database,
generating plots, etc. WA comes with a few result processors that output
results in a few common formats (such as csv or JSON).
You can add your own result processors by creating a Python file in
``~/.workload_automation/result_processors`` with a class that derives from
:class:`wlauto.core.result.ResultProcessor`, which has the following interface::
class ResultProcessor(Plugin):
name = None
description = None
parameters = [
]
def initialize(self, context):
pass
def process_iteration_result(self, result, context):
pass
def export_iteration_result(self, result, context):
pass
def process_run_result(self, result, context):
pass
def export_run_result(self, result, context):
pass
def finalize(self, context):
pass
The method names should be fairly self-explanatory. The difference between
"process" and "export" methods is that export methods will be invoke after
process methods for all result processors have been generated. Process methods
may generated additional artifacts (metrics, files, etc), while export methods
should not -- the should only handle existing results (upload them to a
database, archive on a filer, etc).
The result object passed to iteration methods is an instance of
:class:`wlauto.core.result.IterationResult`, the result object passed to run
methods is an instance of :class:`wlauto.core.result.RunResult`. Please refer to
their API documentation for details.
Example
-------
Here is an example result processor that formats the results as a column-aligned
table::
import os
from wlauto import ResultProcessor
from wlauto.utils.misc import write_table
class Table(ResultProcessor):
name = 'table'
description = 'Gerates a text file containing a column-aligned table with run results.'
def process_run_result(self, result, context):
rows = []
for iteration_result in result.iteration_results:
for metric in iteration_result.metrics:
rows.append([metric.name, str(metric.value), metric.units or '',
metric.lower_is_better and '-' or '+'])
outfile = os.path.join(context.output_directory, 'table.txt')
with open(outfile, 'w') as wfh:
write_table(rows, wfh)
Adding a Resource Getter
========================
A resource getter is a new plugin type added in version 2.1.3. A resource
getter implement a method of acquiring resources of a particular type (such as
APK files or additional workload assets). Resource getters are invoked in
priority order until one returns the desired resource.
If you want WA to look for resources somewhere it doesn't by default (e.g. you
have a repository of APK files), you can implement a getter for the resource and
register it with a higher priority than the standard WA getters, so that it gets
invoked first.
Instances of a resource getter should implement the following interface::
class ResourceGetter(Plugin):
name = None
resource_type = None
priority = GetterPriority.environment
def get(self, resource, **kwargs):
raise NotImplementedError()
The getter should define a name (as with all plugins), a resource
type, which should be a string, e.g. ``'jar'``, and a priority (see `Getter
Prioritization`_ below). In addition, ``get`` method should be implemented. The
first argument is an instance of :class:`wlauto.core.resource.Resource`
representing the resource that should be obtained. Additional keyword
arguments may be used by the invoker to provide additional information about
the resource. This method should return an instance of the resource that
has been discovered (what "instance" means depends on the resource, e.g. it
could be a file path), or ``None`` if this getter was unable to discover
that resource.
Getter Prioritization
---------------------
A priority is an integer with higher numeric values indicating a higher
priority. The following standard priority aliases are defined for getters:
:cached: The cached version of the resource. Look here first. This priority also implies
that the resource at this location is a "cache" and is not the only version of the
resource, so it may be cleared without losing access to the resource.
:preferred: Take this resource in favour of the environment resource.
:environment: Found somewhere under ~/.workload_automation/ or equivalent, or
from environment variables, external configuration files, etc.
These will override resource supplied with the package.
:package: Resource provided with the package.
:remote: Resource will be downloaded from a remote location (such as an HTTP server
or a samba share). Try this only if no other getter was successful.
These priorities are defined as class members of
:class:`wlauto.core.resource.GetterPriority`, e.g. ``GetterPriority.cached``.
Most getters in WA will be registered with either ``environment`` or
``package`` priorities. So if you want your getter to override the default, it
should typically be registered as ``preferred``.
You don't have to stick to standard priority levels (though you should, unless
there is a good reason). Any integer is a valid priority. The standard priorities
range from -20 to 20 in increments of 10.
Example
-------
The following is an implementation of a getter for a workload APK file that
looks for the file under
``~/.workload_automation/dependencies/<workload_name>``::
import os
import glob
from wlauto import ResourceGetter, GetterPriority, settings
from wlauto.exceptions import ResourceError
class EnvironmentApkGetter(ResourceGetter):
name = 'environment_apk'
resource_type = 'apk'
priority = GetterPriority.environment
def get(self, resource):
resource_dir = _d(os.path.join(settings.dependency_directory, resource.owner.name))
version = kwargs.get('version')
found_files = glob.glob(os.path.join(resource_dir, '*.apk'))
if version:
found_files = [ff for ff in found_files if version.lower() in ff.lower()]
if len(found_files) == 1:
return found_files[0]
elif not found_files:
return None
else:
raise ResourceError('More than one .apk found in {} for {}.'.format(resource_dir,
resource.owner.name))
.. _adding_a_device:
Adding a Device
===============
At the moment, only Android devices are supported. Most of the functionality for
interacting with a device is implemented in
:class:`wlauto.common.AndroidDevice` and is exposed through ``generic_android``
device interface, which should suffice for most purposes. The most common area
where custom functionality may need to be implemented is during device
initialization. Usually, once the device gets to the Android home screen, it's
just like any other Android device (modulo things like differences between
Android versions).
If your device doesn't not work with ``generic_device`` interface and you need
to write a custom interface to handle it, you would do that by subclassing
``AndroidDevice`` and then just overriding the methods you need. Typically you
will want to override one or more of the following:
reset
Trigger a device reboot. The default implementation just sends ``adb
reboot`` to the device. If this command does not work, an alternative
implementation may need to be provided.
hard_reset
This is a harsher reset that involves cutting the power to a device
(e.g. holding down power button or removing battery from a phone). The
default implementation is a no-op that just sets some internal flags. If
you're dealing with unreliable prototype hardware that can crash and
become unresponsive, you may want to implement this in order for WA to
be able to recover automatically.
connect
When this method returns, adb connection to the device has been
established. This gets invoked after a reset. The default implementation
just waits for the device to appear in the adb list of connected
devices. If this is not enough (e.g. your device is connected via
Ethernet and requires an explicit ``adb connect`` call), you may wish to
override this to perform the necessary actions before invoking the
``AndroidDevice``\ s version.
init
This gets called once at the beginning of the run once the connection to
the device has been established. There is no default implementation.
It's there to allow whatever custom initialisation may need to be
performed for the device (setting properties, configuring services,
etc).
Please refer to the API documentation for :class:`wlauto.common.AndroidDevice`
for the full list of its methods and their functionality.
Other Plugin Types
=====================
In addition to plugin types covered above, there are few other, more
specialized ones. They will not be covered in as much detail. Most of them
expose relatively simple interfaces with only a couple of methods and it is
expected that if the need arises to extend them, the API-level documentation
that accompanies them, in addition to what has been outlined here, should
provide enough guidance.
:commands: This allows extending WA with additional sub-commands (to supplement
exiting ones outlined in the :ref:`invocation` section).
:modules: Modules are "plugins for plugins". They can be loaded by other
plugins to expand their functionality (for example, a flashing
module maybe loaded by a device in order to support flashing).
Packaging Your Plugins
=========================
If your have written a bunch of plugins, and you want to make it easy to
deploy them to new systems and/or to update them on existing systems, you can
wrap them in a Python package. You can use ``wa create package`` command to
generate appropriate boiler plate. This will create a ``setup.py`` and a
directory for your package that you can place your plugins into.
For example, if you have a workload inside ``my_workload.py`` and a result
processor in ``my_result_processor.py``, and you want to package them as
``my_wa_exts`` package, first run the create command ::
wa create package my_wa_exts
This will create a ``my_wa_exts`` directory which contains a
``my_wa_exts/setup.py`` and a subdirectory ``my_wa_exts/my_wa_exts`` which is
the package directory for your plugins (you can rename the top-level
``my_wa_exts`` directory to anything you like -- it's just a "container" for the
setup.py and the package directory). Once you have that, you can then copy your
plugins into the package directory, creating
``my_wa_exts/my_wa_exts/my_workload.py`` and
``my_wa_exts/my_wa_exts/my_result_processor.py``. If you have a lot of
plugins, you might want to organize them into subpackages, but only the
top-level package directory is created by default, and it is OK to have
everything in there.
.. note:: When discovering plugins thorugh this mechanism, WA traveries the
Python module/submodule tree, not the directory strucuter, therefore,
if you are going to create subdirectories under the top level dictory
created for you, it is important that your make sure they are valid
Python packages; i.e. each subdirectory must contain a __init__.py
(even if blank) in order for the code in that directory and its
subdirectories to be discoverable.
At this stage, you may want to edit ``params`` structure near the bottom of
the ``setup.py`` to add correct author, license and contact information (see
"Writing the Setup Script" section in standard Python documentation for
details). You may also want to add a README and/or a COPYING file at the same
level as the setup.py. Once you have the contents of your package sorted,
you can generate the package by running ::
cd my_wa_exts
python setup.py sdist
This will generate ``my_wa_exts/dist/my_wa_exts-0.0.1.tar.gz`` package which
can then be deployed on the target system with standard Python package
management tools, e.g. ::
sudo pip install my_wa_exts-0.0.1.tar.gz
As part of the installation process, the setup.py in the package, will write the
package's name into ``~/.workoad_automoation/packages``. This will tell WA that
the package contains plugin and it will load them next time it runs.
.. note:: There are no unistall hooks in ``setuputils``, so if you ever
uninstall your WA plugins package, you will have to manually remove
it from ``~/.workload_automation/packages`` otherwise WA will complain
abou a missing package next time you try to run it.
Reference in New Issue View Git Blame Copy Permalink