sascha/workload-automation
1
0
Fork 0
mirror of https://github.com/ARM-software/workload-automation.git synced 2025-01-31 10:11:17 +00:00
Code Issues Releases Wiki Activity

doc: Misc fixes

Browse Source 
Fix typos, formatting and irrelevant information.
This commit is contained in:
Marc Bonnici 2018-05-14 17:21:59 +01:00 committed by setrofim
parent f4dd1a1c84
commit 04fe3768af
19 changed files with 241 additions and 216 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
doc/source/changes.rst
View File

@ -16,8 +16,10 @@ changes moving from WA2 to WA3.
config file and will be performed upon first invocation of WA3. config file and will be performed upon first invocation of WA3.
- The "config" and "global" sections in an agenda are not interchangeable so can all be specified in a "config" section. - The "config" and "global" sections in an agenda are not interchangeable so can all be specified in a "config" section.
- "Results Processors" are now known as "Output Processors" and can now be ran offline. - "Results Processors" are now known as "Output Processors" and can now be ran offline.
- "Instrumentation" is now know as "Instruments" for more consistent naming. - "Instrumentation" is now known as "Instruments" for more consistent naming.
- "Both "Output Processor" and "Instrument" configuration has been merged into "Augmentations" (support for the old naming schemes have been retained for backwards compatibility) - "Both "Output Processor" and "Instrument" configuration have been merged
into "Augmentations" (support for the old naming schemes have been
retained for backwards compatibility)
- New features: - New features:

6
doc/source/developer_reference.rst
View File

@ -13,6 +13,12 @@ Developer Reference
.. include:: developer_reference/output_processing_api.rst .. include:: developer_reference/output_processing_api.rst
.. include:: developer_reference/writing_plugins.rst .. include:: developer_reference/writing_plugins.rst
-----------------
.. include:: developer_reference/contributing.rst .. include:: developer_reference/contributing.rst
-----------------
.. include:: developer_reference/revent.rst .. include:: developer_reference/revent.rst

4
doc/source/developer_reference/contributing.rst
View File

@ -1,7 +1,7 @@
Contributing Code Contributing Code
================= =================
We welcome code contributions via GitHub pull requests.To help with We welcome code contributions via GitHub pull requests. To help with
maintainability of the code line we ask that the code uses a coding style maintainability of the code line we ask that the code uses a coding style
consistent with the rest of WA code. Briefly, it is consistent with the rest of WA code. Briefly, it is
@ -16,7 +16,7 @@ consistent with the rest of WA code. Briefly, it is
"stats" for "statistics", "config" for "configuration", etc are OK). Do "stats" for "statistics", "config" for "configuration", etc are OK). Do
*not* use Hungarian notation (so prefer ``birth_date`` over ``dtBirth``). *not* use Hungarian notation (so prefer ``birth_date`` over ``dtBirth``).
New extensions should also follow implementation guidelines specified in New extensions should also follow implementation guidelines specified in the
:ref:`writing-plugins` section of the documentation. :ref:`writing-plugins` section of the documentation.
We ask that the following checks are performed on the modified code prior to We ask that the following checks are performed on the modified code prior to

2
doc/source/developer_reference/output_processing_api.rst
View File

@ -21,7 +21,7 @@ directory we can initialize a ``RunOutput`` as follows:
From here we can retrieve different information about the run. For example if we From here we can retrieve various information about the run. For example if we
want to see what the overall status of the run was, along with the runtime want to see what the overall status of the run was, along with the runtime
parameters and the metrics recorded from the first job was we can do the following: parameters and the metrics recorded from the first job was we can do the following:

4
doc/source/developer_reference/revent.rst
View File

@ -26,8 +26,8 @@ Only the run stage is mandatory, the remaining stages will be replayed if a
recording is present otherwise no actions will be performed for that particular recording is present otherwise no actions will be performed for that particular
stage. stage.
For instance, to add a custom revent files for a device named mydevice and For instance, to add a custom revent files for a device named "mydevice" and
a workload name myworkload, you need to add the revent files to the directory a workload name "myworkload", you need to add the revent files to the directory
``/home/$WA_USER_HOME/dependencies/myworkload/revent_files`` creating it if ``/home/$WA_USER_HOME/dependencies/myworkload/revent_files`` creating it if
necessary. :: necessary. ::

77
doc/source/developer_reference/writing_plugins.rst
View File

@ -4,7 +4,7 @@
Writing Plugins Writing Plugins
================ ================
Workload Automation offers several plugin points (or plugin types).The most Workload Automation offers several plugin points (or plugin types). The most
interesting of these are interesting of these are
:workloads: These are the tasks that get executed and measured on the device. These :workloads: These are the tasks that get executed and measured on the device. These
@ -14,24 +14,24 @@ interesting of these are
physical device would require its own interface class (though some functionality physical device would require its own interface class (though some functionality
may be reused by subclassing from an existing base). may be reused by subclassing from an existing base).
:instruments: Instruments allow collecting additional data from workload execution (e.g. :instruments: Instruments allow collecting additional data from workload execution (e.g.
system traces). Instruments are not specific to a particular Workload. Instruments system traces). Instruments are not specific to a particular workload. Instruments
can hook into any stage of workload execution. can hook into any stage of workload execution.
:output processors: These are used to format the results of workload execution once they have been :output processors: These are used to format the results of workload execution once they have been
collected. Depending on the callback used, these will run either after each collected. Depending on the callback used, these will run either after each
iteration or at the end of the run, after all of the results have been iteration and/or at the end of the run, after all of the results have been
collected. collected.
You can create an plugin by subclassing the appropriate base class, defining You can create a plugin by subclassing the appropriate base class, defining
appropriate methods and attributes, and putting the .py file with the class into appropriate methods and attributes, and putting the .py file containing the
the "plugins" subdirectory under ``~/.workload_automation`` (or equivalent) class into the "plugins" subdirectory under ``~/.workload_automation`` (or
where it will be automatically picked up by WA. equivalent) where it will be automatically picked up by WA.
Plugin Basics Plugin Basics
---------------- --------------
This sub-section covers things common to implementing plugins of all types. It This sub-section covers things common to implementing plugins of all types. It
is recommended you familiarize yourself with the information here before is recommended you familiarize yourself with the information here before
proceeding onto guidance for specific plugin types. proceeding onto guidance for specific plugin types.
.. _context: .. _context:
@ -280,7 +280,8 @@ As with other resources, host-side paths to the executable binary to be deployed
should be obtained via the :ref:`resource resolver <resource-resolution>`. A should be obtained via the :ref:`resource resolver <resource-resolution>`. A
special resource type, ``Executable`` is used to identify a binary to be special resource type, ``Executable`` is used to identify a binary to be
deployed. This is similar to the regular ``File`` resource, however it takes an deployed. This is similar to the regular ``File`` resource, however it takes an
additional parameter that specifies the ABI for which executable was compiled. additional parameter that specifies the ABI for which the executable was
compiled for.
In order for the binary to be obtained in this way, it must be stored in one of In order for the binary to be obtained in this way, it must be stored in one of
the locations scanned by the resource resolver in a directory structure the locations scanned by the resource resolver in a directory structure
@ -304,7 +305,8 @@ binary has been previously deployed by WA and will not try to re-install.
.. note:: Please also note that the check is done based solely on the binary name. .. note:: Please also note that the check is done based solely on the binary name.
For more information please see: :func:`devlib.target.Target.install_if_needed` For more information please see the devlib
`documentation <http://devlib.readthedocs.io/en/latest/target.html#Target.install_if_needed>`_.
Both of the above methods will return the path to the installed binary on the Both of the above methods will return the path to the installed binary on the
target. The executable should be invoked *only* via that path; do **not** assume target. The executable should be invoked *only* via that path; do **not** assume
@ -312,7 +314,7 @@ that it will be in ``PATH`` on the target (or that the executable with the same
name in ``PATH`` is the version deployed by WA. name in ``PATH`` is the version deployed by WA.
For more information on how to implement this, please see the For more information on how to implement this, please see the
:ref:`how to guide <deploying-executables-example>` :ref:`how to guide <deploying-executables-example>`.
Deploying assets Deploying assets
@ -344,8 +346,8 @@ name
Python identifier. Python identifier.
kind kind
This is the type of the value of the parameter. This could be an This is the type of the value of the parameter. This must be an
callable. Normally this should be a standard Python type, e.g. ``int`` callable. Normally this should be a standard Python type, e.g. ``int``
or ``float``, or one the types defined in :mod:`wa.utils.types`. or ``float``, or one the types defined in :mod:`wa.utils.types`.
If not explicitly specified, this will default to ``str``. If not explicitly specified, this will default to ``str``.
@ -402,12 +404,12 @@ override
Validation and cross-parameter constraints Validation and cross-parameter constraints
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An plugin will get validated at some point after constructions. When exactly A plugin will get validated at some point after construction. When exactly
this occurs depends on the plugin type, but it *will* be validated before it this occurs depends on the plugin type, but it *will* be validated before it
is used. is used.
You can implement ``validate`` method in your plugin (that takes no arguments You can implement ``validate`` method in your plugin (that takes no arguments
beyond the ``self``) to perform any additions *internal* validation in your beyond the ``self``) to perform any additional *internal* validation in your
plugin. By "internal", I mean that you cannot make assumptions about the plugin. By "internal", I mean that you cannot make assumptions about the
surrounding environment (e.g. that the device has been initialized). surrounding environment (e.g. that the device has been initialized).
@ -476,7 +478,7 @@ notify the user. The exception would typically be :class:`ConfigError` or
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`. :class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
All these errors are defined in :mod:`wa.framework.exception` module. All these errors are defined in :mod:`wa.framework.exception` module.
:class:`ConfigError` should be raised where there is a problem in configuration A :class:`ConfigError` should be raised where there is a problem in configuration
specified by the user (either through the agenda or config files). These errors specified by the user (either through the agenda or config files). These errors
are meant to be resolvable by simple adjustments to the configuration (and the are meant to be resolvable by simple adjustments to the configuration (and the
error message should suggest what adjustments need to be made. For all other error message should suggest what adjustments need to be made. For all other
@ -606,9 +608,9 @@ Workload Interface
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
The workload interface should be implemented as follows: The workload interface should be implemented as follows:
:name: This identifies the workload (e.g. it used to specify it in the :name: This identifies the workload (e.g. it is used to specify the workload
:ref:`agenda`. in the :ref:`agenda <agenda>`).
:init_resources: This method may be optionally override to implement dynamic :init_resources: This method may be optionally overridden to implement dynamic
resource discovery for the workload. This method executes resource discovery for the workload. This method executes
early on, before the device has been initialized, so it early on, before the device has been initialized, so it
should only be used to initialize resources that do not should only be used to initialize resources that do not
@ -661,7 +663,7 @@ track of the current execution state (such as the current workload, iteration
number, etc), and contains, among other things, a number, etc), and contains, among other things, a
:class:`wa.framework.output.JobOutput` instance that should be populated from :class:`wa.framework.output.JobOutput` instance that should be populated from
the ``update_output`` method with the results of the execution. For more the ``update_output`` method with the results of the execution. For more
information please see `the context`_ documentation.:: information please see `the context`_ documentation. ::
# ... # ...
@ -697,7 +699,7 @@ The revent workload classes define the following interfaces::
The interface should be implemented as follows The interface should be implemented as follows
:name: This identifies the workload (e.g. it used to specify it in the :name: This identifies the workload (e.g. it used to specify it in the
:ref:`agenda`. :ref:`agenda <agenda>`.
:package_names: This is a list of the android application apk packages names that :package_names: This is a list of the android application apk packages names that
are required to run the workload. are required to run the workload.
@ -714,11 +716,11 @@ Automation, the methods of the new instrument will be found automatically and
hooked up to the supported signals. Once a signal is broadcasted, the hooked up to the supported signals. Once a signal is broadcasted, the
corresponding registered method is invoked. corresponding registered method is invoked.
Each method in Instrument must take two arguments, which are self and context. Each method in ``Instrument`` must take two arguments, which are ``self`` and
Supported method and their corresponding signals can be found in the ``context``. Supported methods and their corresponding signals can be found in
:ref:`Signals Documentation <instruments_method_map>` To make implementations the :ref:`Signals Documentation <instruments_method_map>`. To make
easier and common, the basic steps to add new instrument is similar to the steps implementations easier and common, the basic steps to add new instrument is
to add new workload and an example can be found similar to the steps to add new workload and an example can be found
:ref:`here <adding-an-instrument-example>`. :ref:`here <adding-an-instrument-example>`.
.. _instrument-api: .. _instrument-api:
@ -787,7 +789,7 @@ The full interface of WA instruments is shown below::
""" """
pass pass
This is similar to a Workload, except all methods are optional. In addition to This is similar to a ``Workload``, except all methods are optional. In addition to
the workload-like methods, instruments can define a number of other methods that the workload-like methods, instruments can define a number of other methods that
will get invoked at various points during run execution. The most useful of will get invoked at various points during run execution. The most useful of
which is perhaps ``initialize`` that gets invoked after the device has been which is perhaps ``initialize`` that gets invoked after the device has been
@ -861,11 +863,11 @@ Below is a simple instrument that measures the execution time of a workload::
self.start_time = None self.start_time = None
self.end_time = None self.end_time = None
@fast @very_fast
def start(self, context): def start(self, context):
self.start_time = time.time() self.start_time = time.time()
@fast @very_fast
def stop(self, context): def stop(self, context):
self.end_time = time.time() self.end_time = time.time()
@ -887,7 +889,7 @@ generating plots, etc. WA comes with a few output processors that output
results in a few common formats (such as csv or JSON). results in a few common formats (such as csv or JSON).
You can add your own output processors by creating a Python file in You can add your own output processors by creating a Python file in
``~/.workload_automation/result_processors`` with a class that derives from ``~/.workload_automation/plugins`` with a class that derives from
:class:`wa.OutputProcessor <wa.framework.processor.OutputProcessor>`, which has :class:`wa.OutputProcessor <wa.framework.processor.OutputProcessor>`, which has
the following interface:: the following interface::
@ -919,7 +921,7 @@ the following interface::
The method names should be fairly self-explanatory. The difference between The method names should be fairly self-explanatory. The difference between
"process" and "export" methods is that export methods will be invoke after "process" and "export" methods is that export methods will be invoked after
process methods for all output processors have been generated. Process methods process methods for all output processors have been generated. Process methods
may generated additional artifacts (metrics, files, etc), while export methods may generated additional artifacts (metrics, files, etc), while export methods
should not -- the should only handle existing results (upload them to a should not -- the should only handle existing results (upload them to a
@ -927,8 +929,7 @@ database, archive on a filer, etc).
The output object passed to job methods is an instance of The output object passed to job methods is an instance of
:class:`wa.framework.output.JobOutput`, the output object passed to run methods :class:`wa.framework.output.JobOutput`, the output object passed to run methods
is an instance of :class:`wa.RunOutput <wa.framework.output.RunOutput>`. Please is an instance of :class:`wa.RunOutput <wa.framework.output.RunOutput>`.
refer to their API documentation for details.
Adding a Resource Getter Adding a Resource Getter
@ -987,7 +988,7 @@ should typically be registered as ``preferred``.
You don't have to stick to standard priority levels (though you should, unless You don't have to stick to standard priority levels (though you should, unless
there is a good reason). Any integer is a valid priority. The standard priorities there is a good reason). Any integer is a valid priority. The standard priorities
range from 0 to 50 in increments of 10. range from 0 to 40 in increments of 10.
Example Example
^^^^^^^ ^^^^^^^
@ -1036,9 +1037,9 @@ The currently available platforms are:
The currently available targets from devlib are: The currently available targets from devlib are:
:linux: A device running a Linux based OS. :linux: A device running a Linux based OS.
:android: A device running the Android OS. :android: A device running Android OS.
:local: Used to run locally on a linux based host. :local: Used to run locally on a linux based host.
:chromeos: A device running chromeos, supporting an android container if available. :chromeos: A device running ChromeOS, supporting an android container if available.
For an example of adding you own customized version of an existing devlib target, For an example of adding you own customized version of an existing devlib target,
please see the how to section :ref:`Adding a Custom Target <adding-custom-target-example>`. please see the how to section :ref:`Adding a Custom Target <adding-custom-target-example>`.
@ -1113,7 +1114,7 @@ management tools, e.g. ::
sudo pip install my_wa_exts-0.0.1.tar.gz sudo pip install my_wa_exts-0.0.1.tar.gz
As part of the installation process, the setup.py in the package, will write the As part of the installation process, the setup.py in the package, will write the
package's name into ``~/.workoad_automoation/packages``. This will tell WA that package's name into ``~/.workoad_automation/packages``. This will tell WA that
the package contains plugin and it will load them next time it runs. the package contains plugin and it will load them next time it runs.
.. note:: There are no uninstall hooks in ``setuputils``, so if you ever .. note:: There are no uninstall hooks in ``setuputils``, so if you ever

4
doc/source/faq.rst
View File

@ -37,8 +37,8 @@ install the application onto the device or source the apk and place into
**Q:** I am trying to set a valid runtime parameters however I still receive the error ``"Unknown runtime parameter"`` **Q:** I am trying to set a valid runtime parameters however I still receive the error ``"Unknown runtime parameter"``
------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------
**A:** PLease ensure you have the corresponding module loaded on the device. **A:** Please ensure you have the corresponding module loaded on the device.
Please see :ref:`Runtime Parameters <runtime-parameters>` for the list of See :ref:`Runtime Parameters <runtime-parameters>` for the list of
runtime parameters and their containing modules, and the appropriate section in runtime parameters and their containing modules, and the appropriate section in
:ref:`setting up a device <setting-up-a-device>` for ensuring it is installed. :ref:`setting up a device <setting-up-a-device>` for ensuring it is installed.

57
doc/source/how_tos/developers/adding_plugins.rst
View File

@ -11,7 +11,7 @@ then the ``initialize`` method should be decorated with the ``@once``
method which is decorated instead. Please note if doing this then any installed method which is decorated instead. Please note if doing this then any installed
paths should be added as class attributes rather than instance variables. As a paths should be added as class attributes rather than instance variables. As a
general rule if binaries are installed as part of ``initialize`` then they general rule if binaries are installed as part of ``initialize`` then they
should be installed in the complementary ``finalize`` method. should be uninstalled in the complementary ``finalize`` method.
Part of an example workload demonstrating this is shown below: Part of an example workload demonstrating this is shown below:
@ -41,11 +41,11 @@ Part of an example workload demonstrating this is shown below:
Adding a Workload Examples Adding a Workload Examples
========================== ==========================
The easiest way to create a new workload is to use the create command. ``wa The easiest way to create a new workload is to use the
create workload <args>``. This will use predefined templates to create a :ref:`create <create-command>` command. ``wa create workload <args>``. This
workload based on the options that are supplied to be used as a starting point will use predefined templates to create a workload based on the options that are
for the workload. For more information on using the create workload command see supplied to be used as a starting point for the workload. For more information
``wa create workload -h`` on using the create workload command see ``wa create workload -h``
The first thing to decide is the type of workload you want to create depending The first thing to decide is the type of workload you want to create depending
on the OS you will be using and the aim of the workload. The are currently 6 on the OS you will be using and the aim of the workload. The are currently 6
@ -55,7 +55,7 @@ Once you have decided what type of workload you wish to choose this can be
specified with ``-k <workload_kind>`` followed by the workload name. This specified with ``-k <workload_kind>`` followed by the workload name. This
will automatically generate a workload in the your ``WA_CONFIG_DIR/plugins``. If will automatically generate a workload in the your ``WA_CONFIG_DIR/plugins``. If
you wish to specify a custom location this can be provided with ``-p you wish to specify a custom location this can be provided with ``-p
<directory>`` <path>``
Adding a Basic Workload Example Adding a Basic Workload Example
-------------------------------- --------------------------------
@ -68,10 +68,9 @@ This will generate a very basic workload with dummy methods for the workload
interface and it is left to the developer to add any required functionality to interface and it is left to the developer to add any required functionality to
the workload. the workload.
This example shows a simple workload that times how long it takes to compress a Not all the methods are required to be implemented, this example shows how a
file of a particular size on the device, not all the methods are required to be subset might be used to implement a simple workload that times how long it takes
implements however as many as possible have been used to demonstrate their to compress a file of a particular size on the device.
purpose.
.. note:: This is intended as an example of how to implement the Workload .. note:: This is intended as an example of how to implement the Workload
@ -92,7 +91,6 @@ purpose.
This workload was created for illustration purposes only. It should not be This workload was created for illustration purposes only. It should not be
used to collect actual measurements. used to collect actual measurements.
''' '''
parameters = [ parameters = [
@ -137,9 +135,9 @@ purpose.
This method is used to extract any results from the target for This method is used to extract any results from the target for
example we want to pull the file containing the timing information example we want to pull the file containing the timing information
that we will use to generate metrics for our workload and then we that we will use to generate metrics for our workload and then we
add this file as a artifact as a 'raw' file which once WA has add this file as an artifact with a 'raw' kind, which means once WA
finished processing it will allow it to decide whether to keep the has finished processing it will allow it to decide whether to keep
file or not. the file or not.
""" """
super(ZipTestWorkload, self).extract_results(context) super(ZipTestWorkload, self).extract_results(context)
# Pull the results file to the host # Pull the results file to the host
@ -149,9 +147,9 @@ purpose.
def update_output(self, context): def update_output(self, context):
""" """
In this method we can do any generation of metric that we wish to In this method we can do any generation of metrics that we wish to
for our workload. In this case we are going to simply convert the for our workload. In this case we are going to simply convert the
times reported to seconds and add them as 'metrics' to WA which can times reported into seconds and add them as 'metrics' to WA which can
then be displayed to the user along with any others in a format then be displayed to the user along with any others in a format
dependant on which output processors they have enabled for the run. dependant on which output processors they have enabled for the run.
""" """
@ -343,7 +341,8 @@ command::
This will generate a revent based workload you will end up with a very similar This will generate a revent based workload you will end up with a very similar
python file as to the one outlined in generating a :ref:`UiAutomator based python file as to the one outlined in generating a :ref:`UiAutomator based
workload <apkuiautomator-example>` except without the java automation files. workload <apkuiautomator-example>` however without the accompanying java
automation files.
The main difference between the two is that this workload will subclass The main difference between the two is that this workload will subclass
``ApkReventWorkload`` instead of ``ApkUiautomatorWorkload`` as shown below. ``ApkReventWorkload`` instead of ``ApkUiautomatorWorkload`` as shown below.
@ -367,7 +366,7 @@ The main difference between the two is that this workload will subclass
Adding an Instrument Example Adding an Instrument Example
============================= =============================
This is an example of how we would create a instrument which will trace device This is an example of how we would create a instrument which will trace device
errors. For more detailed information please see errors using a custom "trace" binary file. For more detailed information please see
:ref:`here <instrument-reference>`. The first thing to do is to subclass :ref:`here <instrument-reference>`. The first thing to do is to subclass
:class:`Instrument`, overwrite the variable name with what we want our instrument :class:`Instrument`, overwrite the variable name with what we want our instrument
to be called and locate our binary for our instrument. to be called and locate our binary for our instrument.
@ -388,7 +387,7 @@ We then declare and implement the required methods as detailed
:ref:`here <instrument-api>`. For the ``initialize`` method, we want to install :ref:`here <instrument-api>`. For the ``initialize`` method, we want to install
the executable file to the target so we can use the target's ``install`` the executable file to the target so we can use the target's ``install``
method which will try to copy the file to a location on the device that method which will try to copy the file to a location on the device that
supports execution, will change the file mode appropriately and return the supports execution, change the file mode appropriately and return the
file path on the target. :: file path on the target. ::
def initialize(self, context): def initialize(self, context):
@ -398,7 +397,7 @@ Then we implemented the start method, which will simply run the file to start
tracing. Supposing that the call to this binary requires some overhead to begin tracing. Supposing that the call to this binary requires some overhead to begin
collecting errors we might want to decorate the method with the ``@slow`` collecting errors we might want to decorate the method with the ``@slow``
decorator to try and reduce the impact on other running instruments. For more decorator to try and reduce the impact on other running instruments. For more
information on prioritization please see :ref:`here <prioritization>`:: information on prioritization please see :ref:`here <prioritization>`. ::
@slow @slow
def start(self, context): def start(self, context):
@ -406,7 +405,7 @@ information on prioritization please see :ref:`here <prioritization>`::
Lastly, we need to stop tracing once the workload stops and this happens in the Lastly, we need to stop tracing once the workload stops and this happens in the
stop method, assuming stopping the collection also require some overhead we have stop method, assuming stopping the collection also require some overhead we have
again decorated the method:: again decorated the method. ::
@slow @slow
def stop(self, context): def stop(self, context):
@ -425,9 +424,9 @@ example for trace data we will want to pull it to the device and add it as a
Once we have retrieved the data we can now do any further processing and add any Once we have retrieved the data we can now do any further processing and add any
relevant :ref:`Metrics <metrics>` to the :ref:`context <context>`. For this we relevant :ref:`Metrics <metrics>` to the :ref:`context <context>`. For this we
will use the the ``add_metric`` method and to add the instruments results to the will use the the ``add_metric`` method to add the results to the final output
final result for that workload. The method can be passed 4 params, which are the for that workload. The method can be passed 4 params, which are the metric
metric `key`, `value`, `unit` and `lower_is_better`. :: `key`, `value`, `unit` and `lower_is_better`. ::
def update_output(self, context): def update_output(self, context):
# parse the file if needs to be parsed, or add result directly to # parse the file if needs to be parsed, or add result directly to
@ -442,7 +441,7 @@ instruments and the code to clear these file goes in teardown method. ::
def teardown(self, context): def teardown(self, context):
self.target.remove(os.path.join(self.target.working_directory, 'trace.txt')) self.target.remove(os.path.join(self.target.working_directory, 'trace.txt'))
At the very end of the run we would want to uninstall the binary we deployed earlier :: At the very end of the run we would want to uninstall the binary we deployed earlier. ::
def finalize(self, context): def finalize(self, context):
self.target.uninstall(self.binary_name) self.target.uninstall(self.binary_name)
@ -499,7 +498,7 @@ Next we need to implement any relevant methods, (please see
available methods). In this case we only want to implement the available methods). In this case we only want to implement the
``export_run_output`` method as we are not generating any new artifacts and ``export_run_output`` method as we are not generating any new artifacts and
we only care about the overall output rather than the individual job we only care about the overall output rather than the individual job
outputs. And the implementation is very simple, it just loops through all outputs. The implementation is very simple, it just loops through all
the available metrics for all the available jobs and adds them to a list the available metrics for all the available jobs and adds them to a list
which is written to file and then added as an :ref:`artifact <artifact>` to which is written to file and then added as an :ref:`artifact <artifact>` to
the :ref:`context <context>`. the :ref:`context <context>`.
@ -514,7 +513,7 @@ the :ref:`context <context>`.
class Table(OutputProcessor): class Table(OutputProcessor):
name = 'table' name = 'table'
description = 'Generates a text file containing a column-aligned table with run results.' description = 'Generates a text file containing a column-aligned table of run results.'
def export_run_output(self, output, target_info): def export_run_output(self, output, target_info):
rows = [] rows = []
@ -527,7 +526,7 @@ the :ref:`context <context>`.
outfile = output.get_path('table.txt') outfile = output.get_path('table.txt')
with open(outfile, 'w') as wfh: with open(outfile, 'w') as wfh:
write_table(rows, wfh) write_table(rows, wfh)
output.add_artifact('results_table', 'table.txt, 'export') output.add_artifact('results_table', 'table.txt', 'export')
.. _adding-custom-target-example: .. _adding-custom-target-example:

18
doc/source/how_tos/users/agenda.rst
View File

@ -347,8 +347,8 @@ Classifiers
------------ ------------
Classifiers can be used in 2 distinct ways, the first use is being supplied in Classifiers can be used in 2 distinct ways, the first use is being supplied in
an agenda as a set of key-value pairs which can be used to help identify sub- an agenda as a set of key-value pairs which can be used to help identify sub-tests
tests of a run, for example if you have multiple sections in your agenda running of a run, for example if you have multiple sections in your agenda running
your workloads at different frequencies you might want to set a classifier your workloads at different frequencies you might want to set a classifier
specifying which frequencies are being used. These can then be utilized later, specifying which frequencies are being used. These can then be utilized later,
for example with the ``csv`` :ref:`output processor <output-processors>` with for example with the ``csv`` :ref:`output processor <output-processors>` with
@ -384,9 +384,9 @@ An example agenda is shown here:
The other way that they can used is by being automatically added by some The other way that they can used is by being automatically added by some
workloads to identify their results metrics and artifacts. For example some workloads to identify their results metrics and artifacts. For example some
workloads perform multiple tests with the same execution run and therefore will workloads perform multiple tests with the same execution run and therefore will
use metrics to differentiate between them, For example the ``recentfling`` use metrics to differentiate between them, e.g. the ``recentfling`` workload
workload will use classifiers to distinguish between which loop a particular will use classifiers to distinguish between which loop a particular result is
result is for or whether it is an average across all loops ran. for or whether it is an average across all loops ran.
The output from the agenda above will produce a csv file similar to what is The output from the agenda above will produce a csv file similar to what is
shown below. Some columns have been omitted for clarity however as can been seen shown below. Some columns have been omitted for clarity however as can been seen
@ -500,7 +500,7 @@ turn override global settings.
Augmentationts Augmentations
-------------- --------------
Augmentations are plugins that augment the execution of workload jobs with Augmentations are plugins that augment the execution of workload jobs with
@ -509,7 +509,7 @@ metrics and/or artifacts, such as traces or logs. There are two types of
augmentations: augmentations:
Instruments Instruments
These "instrument" a WA run in order to change it's behavior (e.g. These "instrument" a WA run in order to change it's behaviour (e.g.
introducing delays between successive job executions), or collect introducing delays between successive job executions), or collect
additional measurements (e.g. energy usage). Some instruments may depend additional measurements (e.g. energy usage). Some instruments may depend
on particular features being enabled on the target (e.g. cpufreq), or on particular features being enabled on the target (e.g. cpufreq), or
@ -677,9 +677,11 @@ Each workload will be run in two configurations: once, to collect energy
measurements, and once to collect thermal data and kernel trace. Trace can give measurements, and once to collect thermal data and kernel trace. Trace can give
insight into why a workload is using more or less energy than expected, but it insight into why a workload is using more or less energy than expected, but it
can be relatively intrusive and might impact absolute energy and performance can be relatively intrusive and might impact absolute energy and performance
metrics, which is why it is collected separately. classifiers_ are used to metrics, which is why it is collected separately. Classifiers_ are used to
separate metrics from the two configurations in the results. separate metrics from the two configurations in the results.
.. _other-agenda-configuration:
Other Configuration Other Configuration
------------------- -------------------

42
doc/source/how_tos/users/device_setup.rst
View File

@ -20,12 +20,11 @@ Android
General Device Setup General Device Setup
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
You can specify the device interface by setting ``device`` setting in You can specify the device interface by setting ``device`` setting in a
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by ``config`` file or section. Available interfaces can be viewed by running ``wa
running ``wa list targets`` command. If you don't see your specific platform list targets`` command. If you don't see your specific platform listed (which is
listed (which is likely unless you're using one of the Arm-supplied platforms), then likely unless you're using one of the Arm-supplied platforms), then you should
you should use ``generic_android`` interface (this is set in the config by use ``generic_android`` interface (this is what is used by the default config).
default).
.. code-block:: yaml .. code-block:: yaml
@ -38,7 +37,8 @@ common parameters you might want to change are outlined below.
.. confval:: device .. confval:: device
If you have multiple Android devices connected to the host machine, you will If you have multiple Android devices connected to the host machine, you will
need to set this to indicate to WA which device you want it to use. need to set this to indicate to WA which device you want it to use. The will
be the adb name the is displayed when running ``adb devices``
.. confval:: working_directory .. confval:: working_directory
@ -81,7 +81,7 @@ A typical ``device_config`` inside ``config.yaml`` may look something like
device_config: device_config:
device: 0123456789ABCDEF device: 0123456789ABCDEF
# ... # ...
or a more specific config could be be or a more specific config could be be
@ -172,12 +172,11 @@ Linux
General Device Setup General Device Setup
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
You can specify the device interface by setting ``device`` setting in You can specify the device interface by setting ``device`` setting in a
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by ``config`` file or section. Available interfaces can be viewed by running
running ``wa list targets`` command. If you don't see your specific platform ``wa list targets`` command. If you don't see your specific platform listed
listed (which is likely unless you're using one of the Arm-supplied platforms), then (which is likely unless you're using one of the Arm-supplied platforms), then
you should use ``generic_linux`` interface (this is set in the config by you should use ``generic_linux`` interface.
default).
.. code-block:: yaml .. code-block:: yaml
@ -239,25 +238,24 @@ Chrome OS
General Device Setup General Device Setup
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
You can specify the device interface by setting ``device`` setting in You can specify the device interface by setting ``device`` setting in a
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by ``config`` file or section. Available interfaces can be viewed by
running ``wa list targets`` command. If you don't see your specific platform running ``wa list targets`` command. If you don't see your specific platform
listed (which is likely unless you're using one of the Arm-supplied platforms), then listed (which is likely unless you're using one of the Arm-supplied platforms), then
you should use ``generic_chromeos`` interface (this is set in the config by you should use ``generic_chromeos`` interface.
default).
.. code-block:: yaml .. code-block:: yaml
device: generic_chromeos device: generic_chromeos
The device interface may be configured through ``device_config`` setting, who's The device interface may be configured through ``device_config`` setting, who's
value is a ``dict`` mapping setting names to their values. The chrome os target value is a ``dict`` mapping setting names to their values. The ChromeOS target
is essentially the same as a linux device and requires a similar setup, however is essentially the same as a linux device and requires a similar setup, however
it also optionally supports connecting to an android container running on the it also optionally supports connecting to an android container running on the
device which will be automatically deted if present. If the device supports device which will be automatically detected if present. If the device supports
android applications then the android configuration is also supported. In order android applications then the android configuration is also supported. In order
to support this then WA will open 2 connections to the device, one via SSH to to support this WA will open 2 connections to the device, one via SSH to
the main ChomeOS OS and another via ADB to the android container where a limited the main OS and another via ADB to the android container where a limited
subset of functionality can be performed. subset of functionality can be performed.
In order to distinguish between the two connections some of the android specific In order to distinguish between the two connections some of the android specific

2
doc/source/how_tos/users/revent.rst
View File

@ -94,7 +94,7 @@ command with an example output shown below::
INFO Recording(s) are available at: '$WA_USER_DIRECTORY/dependencies/angrybirds_rio/revent_files' INFO Recording(s) are available at: '$WA_USER_DIRECTORY/dependencies/angrybirds_rio/revent_files'
Once you have made your desired recordings, you can either manually playback Once you have made your desired recordings, you can either manually playback
individual recordings using the :ref:`reply <replay-command>` command or, with individual recordings using the :ref:`replay <replay-command>` command or, with
the recordings in the appropriate dependencies location, simply run the workload the recordings in the appropriate dependencies location, simply run the workload
using the :ref:`run <run-command>` command and then all the available recordings will be using the :ref:`run <run-command>` command and then all the available recordings will be
played back automatically. played back automatically.

20
doc/source/installation.rst
View File

@ -227,7 +227,7 @@ which contains addional information about how to build and to use the file.
Some WA plugins have additional dependencies that need to be Some WA plugins have additional dependencies that need to be
satisfied before they can be used. Not all of these can be provided with WA and satisfied before they can be used. Not all of these can be provided with WA and
so will need to be supplied by the user. They should be placed into so will need to be supplied by the user. They should be placed into
``~/.workload_uatomation/dependencies/<extension name>`` so that WA can find ``~/.workload_automation/dependencies/<extension name>`` so that WA can find
them (you may need to create the directory if it doesn't already exist). You them (you may need to create the directory if it doesn't already exist). You
only need to provide the dependencies for workloads you want to use. only need to provide the dependencies for workloads you want to use.
@ -237,16 +237,16 @@ APK Files
--------- ---------
APKs are application packages used by Android. These are necessary to install on APKs are application packages used by Android. These are necessary to install on
a device when running an :ref:`ApkWorkload <apk-workload>` or derivative. PLease a device when running an :ref:`ApkWorkload <apk-workload>` or derivative. Please
see the workload description using the :ref:`show command <show-command>` to see see the workload description using the :ref:`show <show-command>` command to see
which version of the apk the UI automation has been tested with and place the which version of the apk the UI automation has been tested with and place the
apk in the corresponding Automation may also work with other versions apk in the corresponding workloads dependency folder. Automation may also work
(especially if it's only a minor or revision difference -- major version with other versions (especially if it's only a minor or revision difference --
differences are more likely to contain incompatible UI changes) but this has not major version differences are more likely to contain incompatible UI changes)
been tested. As a general rule we do not guarantee support for the latest but this has not been tested. As a general rule we do not guarantee support for
version of an app and they are updated on as needed basis. We do however attempt the latest version of an app and they are updated on an as needed basis. We do
to support backwards compatibility with previous major releases however beyond however attempt to support backwards compatibility with previous major releases
this support will likely be dropped. however beyond this support will likely be dropped.
Gaming Workloads Gaming Workloads

106
doc/source/migration_guide.rst
View File

@ -15,12 +15,12 @@ Configuration
Default configuration file change Default configuration file change
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Instead of the standard ``config.py`` file located at ``$WA_USER_HOME/config.py` Instead of the standard ``config.py`` file located at
WA not use a ``confg.yaml`` files which is written in the YAML format instead of ``$WA_USER_HOME/config.py`` WA now uses a ``confg.yaml`` file (at the same
python. Additionally upon first invocation WA3 will automatically try and detect location) which is written in the YAML format instead of python. Additionally
whether a WA2 config file is present and convert it to use the new WA3 format. upon first invocation WA3 will automatically try and detect whether a WA2 config
During this process any know parameter name changes should be detected and file is present and convert it to use the new WA3 format. During this process
updated accordingly. any known parameter name changes should be detected and updated accordingly.
Plugin Changes Plugin Changes
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
@ -78,7 +78,7 @@ the new parameter names should be preferred:
- The workload parameter :confval:`check_apk` has been renamed to - The workload parameter :confval:`check_apk` has been renamed to
:confval:`prefer_host_package` to be more explicit in it's functionality to indicated :confval:`prefer_host_package` to be more explicit in it's functionality to indicated
whether a package on the target for the host should have priority when whether a package on the target or the host should have priority when
searching for a suitable package. searching for a suitable package.
Individual workload parameters have been attempted to be standardized for the Individual workload parameters have been attempted to be standardized for the
@ -110,7 +110,7 @@ moved into a ``__failed`` subdirectory to allow for quicker analysis.
Output API Output API
~~~~~~~~~~ ~~~~~~~~~~
There is now an Output API which can be used to more easily post process the There is now an Output API which can be used to more easily post process the
output from a workload. For more information please see the output from a run. For more information please see the
:ref:`Output API <output-api>` documentation. :ref:`Output API <output-api>` documentation.
@ -155,15 +155,15 @@ Workloads
Python Workload Structure Python Workload Structure
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``update_results`` method has been split out into 2 stages. There is now - The ``update_results`` method has been split out into 2 stages. There is now
``extract_results`` and ``update_output`` which should be used for extracting ``extract_results`` and ``update_output`` which should be used for extracting
any results from the target back to the host system and to update the output any results from the target back to the host system and to update the output
with any metrics or artefacts for the specific workload iteration respectively. with any metrics or artefacts for the specific workload iteration respectively.
WA now features :ref:`decorators <decorators>` which can be used to allow for more efficient - WA now features :ref:`decorators <decorators>` which can be used to allow for more efficient
binary deployment and that they are only installed to the device once per run. For binary deployment and that they are only installed to the device once per run. For
more information of implementing this please see more information of implementing this please see
:ref:`deploying executables to a target <deploying-executables>`. :ref:`deploying executables to a target <deploying-executables>`.
APK Functionality APK Functionality
@ -176,27 +176,25 @@ available as the apk attribute of the workload. This means that for example
UiAutomator Java Structure UiAutomator Java Structure
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
Instead of a single ``runUiAutomation`` method to perform all of the UiAutomation, Instead of a single ``runUiAutomation`` method to perform all of the UiAutomation,
the structure has been refactored into 4 methods that can optionally be overridden. the structure has been refactored into 5 methods that can optionally be overridden.
The available methods are ``initialize``, ``setup``, ``runWorkload``, ``extactResults`` The available methods are ``initialize``, ``setup``, ``runWorkload``, ``extactResults``
and ``teardown`` to better mimic the different stages in the python workload. and ``teardown`` to better mimic the different stages in the python workload.
``initialize`` should have the ``@Before`` tag attached to the method which will cause it - ``initialize`` should be used to retrieve
to be ran before each of the stages of the workload. This method should be used to retrieve and set any relevant parameters required during the workload.
and set any relevant parameters required during the workload. - ``setup`` should be used to perform any setup required for the workload, for
example dismissing popups or configuring and required settings.
- ``runWorkload`` should be used to perform the actual measurable work of the workload.
- ``extractResults`` should be used to extract any relevant results from the
target after the workload has been completed.
- ``teardown`` should be used to perform any final clean up of the workload on the target.
The remaining method all have the ``@Test`` tag attached to the method to indicate that this .. note:: The ``initialize`` method should have the ``@Before`` tag attached
is a test stage that should be called at the appropriate time. to the method which will cause it to be ran before each of the stages of
the workload. The remaining method should all have the ``@Test`` tag
``setup`` should be used to perform any setup required for the workload, for attached to the method to indicate that this is a test stage that should be
example dismissing popups or configuring and required settings. called at the appropriate time.
``runWorkload`` should be used to perform the actual measurable work of the workload.
``extractResults`` should be used to extract any relevant results from the
target after the workload has been completed.
``teardown`` should be used to perform any final clean up of the workload on the target.
GUI Functionality GUI Functionality
^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
@ -208,28 +206,26 @@ you wish to pass parameters to a UiAuotmator workload you will now need to use
Attributes Attributes
^^^^^^^^^^ ^^^^^^^^^^
The ``device`` attribute of the workload is now a devlib ``target``. Some of the - The old ``package`` attribute has been replaced by ``package_names`` which
command names remain the same, however there will be differences. The API can be expects a list of strings which allows for multiple package names to be
found here: http://devlib.readthedocs.io/en/latest/target.html however some of specified if required. It is also no longer required to explicitly state the
the more common changes can be found below: launch-able activity, this will be automatically discovered from the apk so this
workload attribute can be removed.
- The ``device`` attribute of the workload is now a devlib ``target``. Some of the
command names remain the same, however there will be differences. The API can be
found at http://devlib.readthedocs.io/en/latest/target.html however some of
the more common changes can be found below:
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
| Original Method | New Method | | Original Method | New Method |
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
|``self.device.pull_file(file)`` | ``self.target.pull(file)`` | |``self.device.pull_file(file)`` | ``self.target.pull(file)`` |
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
|``self.device.push_file(file)`` | ``self.target.push(file)`` | |``self.device.push_file(file)`` | ``self.target.push(file)`` |
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
|``self.device.install_executable(file)`` | ``self.target.install(file)`` | |``self.device.install_executable(file)`` | ``self.target.install(file)`` |
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
|``self.device.execute(cmd, background=True)`` | ``self.target.background(cmd)``| |``self.device.execute(cmd, background=True)`` | ``self.target.background(cmd)``|
+----------------------------------------------+---------------------------------+ +----------------------------------------------+---------------------------------+
The old ``package`` attribute has been replaced by ``package_names`` which
expects a list of strings which allows for multiple package names to be
specified if required. It is also no longer required to explicitly state the
launch-able activity, this will be automatically discovered from the apk so this
workload attribute can be removed.

19
doc/source/user_guide.rst
View File

@ -67,7 +67,7 @@ List Command
============ ============
In order to get started with using WA we first we need to find In order to get started with using WA we first we need to find
out what is available to use. In order to do this we can use the "list" out what is available to use. In order to do this we can use the :ref:`list <list-command>`
command followed by the type of plugin that you wish to see. command followed by the type of plugin that you wish to see.
For example to see what workloads are available along with a short description For example to see what workloads are available along with a short description
@ -97,7 +97,9 @@ Which will give an output in the format of:
The same syntax can be used to display ``commands``, The same syntax can be used to display ``commands``,
``energy_instrument_backends``, ``instruments``, ``output_processors``, ``energy_instrument_backends``, ``instruments``, ``output_processors``,
``resource_getters`` and ``targets``. Alternatively please see the ``resource_getters``, ``targets``. Once you have found the plugin you are
looking for you can use the :ref:`show <show-command>` command to display more
detailed information. Alternatively please see the
:ref:`Plugin Reference <plugin-reference>` for an online version. :ref:`Plugin Reference <plugin-reference>` for an online version.
Show Command Show Command
@ -141,13 +143,14 @@ configuration file which will in turn overwrite the default configuration file.
Android Android
------- -------
By default, the device is set to 'generic_android'. WA is configured to work By default, the device WA will use is set to 'generic_android'. WA is configured
with a generic Android device through ``adb``. If you only have one device listed to work with a generic Android device through ``adb``. If you only have one
when you execute ``adb devices``, and your device has a standard Android device listed when you execute ``adb devices``, and your device has a standard
configuration, then no extra configuration is required. Android configuration, then no extra configuration is required.
However, if your device is connected via network, you will have to manually execute However, if your device is connected via network, you will have to manually
``adb connect <device ip>`` so that it appears in the device listing. execute ``adb connect <device ip>`` (or specify this in your
:ref:`agenda <agenda>`) so that it appears in the device listing.
If you have multiple devices connected, you will need to tell WA which one you If you have multiple devices connected, you will need to tell WA which one you
want it to use. You can do that by setting ``device`` in the device_config section. want it to use. You can do that by setting ``device`` in the device_config section.

9
doc/source/user_reference.rst
View File

@ -12,5 +12,14 @@ User Reference
--------------------------------------------------------------- ---------------------------------------------------------------
.. include:: user_reference/configuration.rst .. include:: user_reference/configuration.rst
-------------------
.. include:: user_reference/invocation.rst .. include:: user_reference/invocation.rst
-------------------
.. include:: user_reference/output_directory.rst .. include:: user_reference/output_directory.rst

40
doc/source/user_reference/invocation.rst
View File

@ -16,16 +16,18 @@ Individual sub-commands are discussed in detail below.
Run Run
--- ---
The most common sub-command you will use is ``run``. This will run specified The most common sub-command you will use is ``run``. This will run the specified
workload(s) and process resulting output. This takes a single mandatory workload(s) and process its resulting output. This takes a single mandatory
argument that specifies what you want WA to run. This could be either a argument which specifies what you want WA to run. This could be either a workload
workload name, or a path to an "agenda" file that allows to specify multiple name, or a path to an agenda" file that allows to specify multiple workloads as
workloads as well as a lot additional configuration (see :ref:`agenda` well as a lot additional configuration (see :ref:`agenda` section for details).
section for details). Executing :: Executing ::
wa run -h wa run -h
Will display help for this subcommand that will look something like this:: Will display help for this subcommand that will look something like this:
.. code-block:: none
usage: wa run [-h] [-c CONFIG] [-v] [--version] [-d DIR] [-f] [-i ID] usage: wa run [-h] [-c CONFIG] [-v] [--version] [-d DIR] [-f] [-i ID]
[--disable INSTRUMENT] [--disable INSTRUMENT]
@ -130,7 +132,7 @@ will produce something like: ::
from the website: from the website:
The whole idea behind this application is to use the same Pi calculation The whole idea behind this application is to use the same Pi calculation
algorithm on every Android Device and check how fast that proccess is. algorithm on every Android Device and check how fast that process is.
Better calculation times, conclude to faster Android devices. This way you Better calculation times, conclude to faster Android devices. This way you
can also check how lightweight your custom made Android build is. Or not. can also check how lightweight your custom made Android build is. Or not.
@ -185,9 +187,9 @@ will produce something like: ::
markers_enabled : boolean markers_enabled : boolean
If set to ``True``, workloads will insert markers into logs If set to ``True``, workloads will insert markers into logs
at various points during execution. These markes may be used at various points during execution. These markers may be used
by other plugins or post-processing scripts to provide by other plugins or post-processing scripts to provide
measurments or statistics for specific parts of the workload measurements or statistics for specific parts of the workload
execution. execution.
.. note:: You can also use this command to view global settings by using ``wa show settings`` .. note:: You can also use this command to view global settings by using ``wa show settings``
@ -202,9 +204,8 @@ This aids in the creation of new WA-related objects for example agendas and work
For more detailed information on creating workloads please see the For more detailed information on creating workloads please see the
:ref:`adding a workload <adding-a-workload>` section for more details. :ref:`adding a workload <adding-a-workload>` section for more details.
agendas:
As an example to create an agenda that will run the dhrystone and memcpy workloads As an example to create an agenda that will run the dhrystone and memcpy workloads
that will use the status and hwmon augumentations, run each test 3 times and save that will use the status and hwmon augmentations, run each test 3 times and save
into the file ``my_agenda.yaml`` the following command can be used:: into the file ``my_agenda.yaml`` the following command can be used::
wa create agenda dhrystone memcpy status hwmon -i 3 -o my_agenda.yaml wa create agenda dhrystone memcpy status hwmon -i 3 -o my_agenda.yaml
@ -271,7 +272,7 @@ using the following command::
Record Record
------ ------
This command simiplifies the process of recording revent files. It will This command simplifies the process of recording revent files. It will
automatically deploy revent and has options to automatically open apps and automatically deploy revent and has options to automatically open apps and
record specified stages of a workload. Revent allows you to record raw inputs record specified stages of a workload. Revent allows you to record raw inputs
such as screen swipes or button presses. This can be useful for recording inputs such as screen swipes or button presses. This can be useful for recording inputs
@ -285,14 +286,14 @@ devices it is obtained from ``/proc/device-tree/model``. - suffix is used by WA
to determine which part of the app execution the recording is for, currently to determine which part of the app execution the recording is for, currently
these are either ``setup``, ``run``, ``extract_results`` or ``teardown``. All these are either ``setup``, ``run``, ``extract_results`` or ``teardown``. All
stages except ``run`` are optional for playback and to specify which stages stages except ``run`` are optional for playback and to specify which stages
shoule be recorded the ``-s``, ``-r``, ``-e`` or ``-t`` arguments respectively, should be recorded the ``-s``, ``-r``, ``-e`` or ``-t`` arguments respectively,
or optionally ``-a`` to indicate all stages should be recorded. or optionally ``-a`` to indicate all stages should be recorded.
The full set of options for this command are:: The full set of options for this command are::
usage: wa record [-h] [-c CONFIG] [-v] [--version] [-d DEVICE] [-o FILE] [-s] usage: wa record [-h] [-c CONFIG] [-v] [--version] [-d DEVICE] [-o FILE] [-s]
[-e] [-t] [-a] [-C] [-p PACKAGE | -w WORKLOAD] [-r] [-e] [-t] [-a] [-C] [-p PACKAGE | -w WORKLOAD]
optional arguments: optional arguments:
-h, --help show this help message and exit -h, --help show this help message and exit
@ -307,8 +308,8 @@ The full set of options for this command are::
-o FILE, --output FILE -o FILE, --output FILE
Specify the output file Specify the output file
-s, --setup Record a recording for setup stage -s, --setup Record a recording for setup stage
-e, --extract_results -r, --run Record a recording for run stage
Record a recording for extract_results stage -e, --extract_results Record a recording for extract_results stage
-t, --teardown Record a recording for teardown stage -t, --teardown Record a recording for teardown stage
-a, --all Record recordings for available stages -a, --all Record recordings for available stages
-C, --clear Clear app cache before launching it -C, --clear Clear app cache before launching it
@ -324,8 +325,9 @@ For more information please see :ref:`Revent Recording <revent-recording>`.
Replay Replay
------ ------
Along side ``record`` wa also has a command to playback a single recorded revent file. Alongside ``record`` wa also has a command to playback a single recorded revent
It behaves similar to the ``record`` command taking a subset of the same options allowing you to automatically launch a package on the device :: file. It behaves similar to the ``record`` command taking a subset of the same
options allowing you to automatically launch a package on the device ::
usage: wa replay [-h] [-c CONFIG] [-v] [--debug] [--version] [-p PACKAGE] [-C] usage: wa replay [-h] [-c CONFIG] [-v] [--debug] [--version] [-p PACKAGE] [-C]
revent revent

37
doc/source/user_reference/runtime_parameters.rst
View File

@ -14,22 +14,27 @@ Example
^^^^^^^ ^^^^^^^
Say we want to perform an experiment on an Android big.LITTLE devices to compare Say we want to perform an experiment on an Android big.LITTLE devices to compare
the power consumption between the big and LITTLE clusters running the dhrystone the power consumption between the big and LITTLE clusters running the dhrystone
workload. Assuming we have additional instrumentation active for this device and benchmarkpi workloads. Assuming we have additional instrumentation active
that we can measure the power the device is consuming, to reduce external for this device that can measure the power the device is consuming, to reduce
factors we want to ensure that the device has its screen turned off and that it external factors we want to ensure that the device is in airplane mode turned on
is in airplane mode turned on for both tests. We will then run 2 :ref:`sections for all our tests and the screen is off only for our dhrystone run. We will then
<sections>` will each enable a single cluster on the device, set the cores to their run 2 :ref:`sections <sections>` will each enable a single cluster on the
maximum frequency and disable all available idle states. device, set the cores to their maximum frequency and disable all available idle
states.
.. code-block:: yaml .. code-block:: yaml
config:
runtime_parameters:
airplane_mode: true
#.. #..
workloads: workloads:
- name: dhrystone - name: dhrystone
iterations: 1 iterations: 1
runtime_parameters: runtime_parameters:
airplane_mode: true
screen_on: false screen_on: false
- name: benchmarkpi
iterations: 1
sections: sections:
- id: LITTLES - id: LITTLES
runtime_parameters: runtime_parameters:
@ -167,8 +172,9 @@ CPUIdle
^^^^^^^ ^^^^^^^
:idle_states: A ``string`` or list of strings which can be used to specify what :idle_states: A ``string`` or list of strings which can be used to specify what
idles states should be enabled for all cores if there are common frequencies idles states should be enabled for all cores if there are common
available. 'all' and 'none' are also valid entries as a shorthand idle states available. 'all' and 'none' are also valid entries as a
shorthand
:<core_name>_idle_states: A ``string`` or list of strings which can be used to :<core_name>_idle_states: A ``string`` or list of strings which can be used to
specify what idles states should be enabled for cores of a particular type specify what idles states should be enabled for cores of a particular type
@ -200,7 +206,7 @@ Android Specific Runtime Parameters
entries are ``NATURAL``, ``LEFT``, ``INVERTED``, ``RIGHT``. entries are ``NATURAL``, ``LEFT``, ``INVERTED``, ``RIGHT``.
:screen_on: A ``boolean`` to specify whether the devices screen should be :screen_on: A ``boolean`` to specify whether the devices screen should be
turned on. Defaults to ``true``. turned on. Defaults to ``True``.
.. _setting-sysfiles: .. _setting-sysfiles:
@ -209,18 +215,19 @@ Setting Sysfiles
In order to perform additional configuration of a target the ``sysfile_values`` In order to perform additional configuration of a target the ``sysfile_values``
runtime parameter can be used. The value for this parameter is a mapping (an runtime parameter can be used. The value for this parameter is a mapping (an
associative array, in YAML) of file paths onto values that should be written associative array, in YAML) of file paths onto values that should be written
into those files. sysfile_values is the only runtime parameter that is available into those files. ``sysfile_values`` is the only runtime parameter that is
for any (Linux) device. Other runtime parameters will depend on the specifics of available for any (Linux) device. Other runtime parameters will depend on the
the device used (e.g. its CPU cores configuration) as detailed above. specifics of the device used (e.g. its CPU cores configuration) as detailed
above.
.. note:: By default WA will attempt to verify that the sysfile value was .. note:: By default WA will attempt to verify that the any sysfile values were
written correctly by reading the node back and comparing the two values. If written correctly by reading the node back and comparing the two values. If
you do not wish this check to happen, for example the node you are writing to you do not wish this check to happen, for example the node you are writing to
is write only, you can append an ``!`` to the file path to disable this is write only, you can append an ``!`` to the file path to disable this
verification. verification.
For example the following configuration could be used to enable and verify that cpu0 For example the following configuration could be used to enable and verify that cpu0
is online, however will not attempt to check that it's governor have been set to is online, however will not attempt to check that its governor have been set to
userspace:: userspace::
- name: dhrystone - name: dhrystone

2
wa/framework/configuration/core.py
View File

@ -652,7 +652,7 @@ class RunConfiguration(Configuration):
The maximum number of times failed jobs will be retried before The maximum number of times failed jobs will be retried before
giving up. If not set. giving up. If not set.
.. note:: this number does not include the original attempt .. note:: This number does not include the original attempt
''', ''',
), ),
ConfigurationPoint( ConfigurationPoint(

2
wa/workloads/geekbench/__init__.py
View File

@ -37,8 +37,8 @@ class Geekbench(ApkUiautoWorkload):
Geekbench scores are calibrated against a baseline score of 1,000 (which is Geekbench scores are calibrated against a baseline score of 1,000 (which is
the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are
better, with double the score indicating double the performance. better, with double the score indicating double the performance.
The benchmarks fall into one of four categories:
The benchmarks fall into one of four categories:
- integer performance. - integer performance.
- floating point performance. - floating point performance.
- memory performance. - memory performance.