mirror of
https://github.com/ARM-software/workload-automation.git
synced 2025-01-31 02:01:16 +00:00
doc: Misc fixes
Fix typos, formatting and irrelevant information.
This commit is contained in:
Marc Bonnici
setrofim
parent
f4dd1a1c84
commit
04fe3768af
@ -16,8 +16,10 @@ changes moving from WA2 to WA3.
|
||||
config file and will be performed upon first invocation of WA3.
|
||||
- The "config" and "global" sections in an agenda are not interchangeable so can all be specified in a "config" section.
|
||||
- "Results Processors" are now known as "Output Processors" and can now be ran offline.
|
||||
- "Instrumentation" is now know as "Instruments" for more consistent naming.
|
||||
- "Both "Output Processor" and "Instrument" configuration has been merged into "Augmentations" (support for the old naming schemes have been retained for backwards compatibility)
|
||||
- "Instrumentation" is now known as "Instruments" for more consistent naming.
|
||||
- "Both "Output Processor" and "Instrument" configuration have been merged
|
||||
into "Augmentations" (support for the old naming schemes have been
|
||||
retained for backwards compatibility)
|
||||
|
||||
|
||||
- New features:
|
||||
|
||||
@ -13,6 +13,12 @@ Developer Reference
|
||||
|
||||
.. include:: developer_reference/output_processing_api.rst
|
||||
.. include:: developer_reference/writing_plugins.rst
|
||||
|
||||
-----------------
|
||||
|
||||
.. include:: developer_reference/contributing.rst
|
||||
|
||||
-----------------
|
||||
|
||||
.. include:: developer_reference/revent.rst
|
||||
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
Contributing Code
|
||||
=================
|
||||
|
||||
We welcome code contributions via GitHub pull requests.To help with
|
||||
We welcome code contributions via GitHub pull requests. To help with
|
||||
maintainability of the code line we ask that the code uses a coding style
|
||||
consistent with the rest of WA code. Briefly, it is
|
||||
|
||||
@ -16,7 +16,7 @@ consistent with the rest of WA code. Briefly, it is
|
||||
"stats" for "statistics", "config" for "configuration", etc are OK). Do
|
||||
*not* use Hungarian notation (so prefer ``birth_date`` over ``dtBirth``).
|
||||
|
||||
New extensions should also follow implementation guidelines specified in
|
||||
New extensions should also follow implementation guidelines specified in the
|
||||
:ref:`writing-plugins` section of the documentation.
|
||||
|
||||
We ask that the following checks are performed on the modified code prior to
|
||||
|
||||
@ -21,7 +21,7 @@ directory we can initialize a ``RunOutput`` as follows:
|
||||
|
||||
|
||||
|
||||
From here we can retrieve different information about the run. For example if we
|
||||
From here we can retrieve various information about the run. For example if we
|
||||
want to see what the overall status of the run was, along with the runtime
|
||||
parameters and the metrics recorded from the first job was we can do the following:
|
||||
|
||||
|
||||
@ -26,8 +26,8 @@ Only the run stage is mandatory, the remaining stages will be replayed if a
|
||||
recording is present otherwise no actions will be performed for that particular
|
||||
stage.
|
||||
|
||||
For instance, to add a custom revent files for a device named mydevice and
|
||||
a workload name myworkload, you need to add the revent files to the directory
|
||||
For instance, to add a custom revent files for a device named "mydevice" and
|
||||
a workload name "myworkload", you need to add the revent files to the directory
|
||||
``/home/$WA_USER_HOME/dependencies/myworkload/revent_files`` creating it if
|
||||
necessary. ::
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
Writing Plugins
|
||||
================
|
||||
|
||||
Workload Automation offers several plugin points (or plugin types).The most
|
||||
Workload Automation offers several plugin points (or plugin types). The most
|
||||
interesting of these are
|
||||
|
||||
:workloads: These are the tasks that get executed and measured on the device. These
|
||||
@ -14,24 +14,24 @@ interesting of these are
|
||||
physical device would require its own interface class (though some functionality
|
||||
may be reused by subclassing from an existing base).
|
||||
:instruments: Instruments allow collecting additional data from workload execution (e.g.
|
||||
system traces). Instruments are not specific to a particular Workload. Instruments
|
||||
system traces). Instruments are not specific to a particular workload. Instruments
|
||||
can hook into any stage of workload execution.
|
||||
:output processors: These are used to format the results of workload execution once they have been
|
||||
collected. Depending on the callback used, these will run either after each
|
||||
iteration or at the end of the run, after all of the results have been
|
||||
iteration and/or at the end of the run, after all of the results have been
|
||||
collected.
|
||||
|
||||
You can create an plugin by subclassing the appropriate base class, defining
|
||||
appropriate methods and attributes, and putting the .py file with the class into
|
||||
the "plugins" subdirectory under ``~/.workload_automation`` (or equivalent)
|
||||
where it will be automatically picked up by WA.
|
||||
You can create a plugin by subclassing the appropriate base class, defining
|
||||
appropriate methods and attributes, and putting the .py file containing the
|
||||
class into the "plugins" subdirectory under ``~/.workload_automation`` (or
|
||||
equivalent) where it will be automatically picked up by WA.
|
||||
|
||||
|
||||
Plugin Basics
|
||||
----------------
|
||||
--------------
|
||||
|
||||
This sub-section covers things common to implementing plugins of all types. It
|
||||
is recommended you familiarize yourself with the information here before
|
||||
is recommended you familiarize yourself with the information here before
|
||||
proceeding onto guidance for specific plugin types.
|
||||
|
||||
.. _context:
|
||||
@ -280,7 +280,8 @@ As with other resources, host-side paths to the executable binary to be deployed
|
||||
should be obtained via the :ref:`resource resolver <resource-resolution>`. A
|
||||
special resource type, ``Executable`` is used to identify a binary to be
|
||||
deployed. This is similar to the regular ``File`` resource, however it takes an
|
||||
additional parameter that specifies the ABI for which executable was compiled.
|
||||
additional parameter that specifies the ABI for which the executable was
|
||||
compiled for.
|
||||
|
||||
In order for the binary to be obtained in this way, it must be stored in one of
|
||||
the locations scanned by the resource resolver in a directory structure
|
||||
@ -304,7 +305,8 @@ binary has been previously deployed by WA and will not try to re-install.
|
||||
|
||||
|
||||
.. note:: Please also note that the check is done based solely on the binary name.
|
||||
For more information please see: :func:`devlib.target.Target.install_if_needed`
|
||||
For more information please see the devlib
|
||||
`documentation <http://devlib.readthedocs.io/en/latest/target.html#Target.install_if_needed>`_.
|
||||
|
||||
Both of the above methods will return the path to the installed binary on the
|
||||
target. The executable should be invoked *only* via that path; do **not** assume
|
||||
@ -312,7 +314,7 @@ that it will be in ``PATH`` on the target (or that the executable with the same
|
||||
name in ``PATH`` is the version deployed by WA.
|
||||
|
||||
For more information on how to implement this, please see the
|
||||
:ref:`how to guide <deploying-executables-example>`
|
||||
:ref:`how to guide <deploying-executables-example>`.
|
||||
|
||||
|
||||
Deploying assets
|
||||
@ -344,8 +346,8 @@ name
|
||||
Python identifier.
|
||||
|
||||
kind
|
||||
This is the type of the value of the parameter. This could be an
|
||||
callable. Normally this should be a standard Python type, e.g. ``int``
|
||||
This is the type of the value of the parameter. This must be an
|
||||
callable. Normally this should be a standard Python type, e.g. ``int``
|
||||
or ``float``, or one the types defined in :mod:`wa.utils.types`.
|
||||
If not explicitly specified, this will default to ``str``.
|
||||
|
||||
@ -402,12 +404,12 @@ override
|
||||
Validation and cross-parameter constraints
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
An plugin will get validated at some point after constructions. When exactly
|
||||
A plugin will get validated at some point after construction. When exactly
|
||||
this occurs depends on the plugin type, but it *will* be validated before it
|
||||
is used.
|
||||
|
||||
You can implement ``validate`` method in your plugin (that takes no arguments
|
||||
beyond the ``self``) to perform any additions *internal* validation in your
|
||||
beyond the ``self``) to perform any additional *internal* validation in your
|
||||
plugin. By "internal", I mean that you cannot make assumptions about the
|
||||
surrounding environment (e.g. that the device has been initialized).
|
||||
|
||||
@ -476,7 +478,7 @@ notify the user. The exception would typically be :class:`ConfigError` or
|
||||
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
|
||||
All these errors are defined in :mod:`wa.framework.exception` module.
|
||||
|
||||
:class:`ConfigError` should be raised where there is a problem in configuration
|
||||
A :class:`ConfigError` should be raised where there is a problem in configuration
|
||||
specified by the user (either through the agenda or config files). These errors
|
||||
are meant to be resolvable by simple adjustments to the configuration (and the
|
||||
error message should suggest what adjustments need to be made. For all other
|
||||
@ -606,9 +608,9 @@ Workload Interface
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
The workload interface should be implemented as follows:
|
||||
|
||||
:name: This identifies the workload (e.g. it used to specify it in the
|
||||
:ref:`agenda`.
|
||||
:init_resources: This method may be optionally override to implement dynamic
|
||||
:name: This identifies the workload (e.g. it is used to specify the workload
|
||||
in the :ref:`agenda <agenda>`).
|
||||
:init_resources: This method may be optionally overridden to implement dynamic
|
||||
resource discovery for the workload. This method executes
|
||||
early on, before the device has been initialized, so it
|
||||
should only be used to initialize resources that do not
|
||||
@ -661,7 +663,7 @@ track of the current execution state (such as the current workload, iteration
|
||||
number, etc), and contains, among other things, a
|
||||
:class:`wa.framework.output.JobOutput` instance that should be populated from
|
||||
the ``update_output`` method with the results of the execution. For more
|
||||
information please see `the context`_ documentation.::
|
||||
information please see `the context`_ documentation. ::
|
||||
|
||||
# ...
|
||||
|
||||
@ -697,7 +699,7 @@ The revent workload classes define the following interfaces::
|
||||
The interface should be implemented as follows
|
||||
|
||||
:name: This identifies the workload (e.g. it used to specify it in the
|
||||
:ref:`agenda`.
|
||||
:ref:`agenda <agenda>`.
|
||||
:package_names: This is a list of the android application apk packages names that
|
||||
are required to run the workload.
|
||||
|
||||
@ -714,11 +716,11 @@ Automation, the methods of the new instrument will be found automatically and
|
||||
hooked up to the supported signals. Once a signal is broadcasted, the
|
||||
corresponding registered method is invoked.
|
||||
|
||||
Each method in Instrument must take two arguments, which are self and context.
|
||||
Supported method and their corresponding signals can be found in the
|
||||
:ref:`Signals Documentation <instruments_method_map>` To make implementations
|
||||
easier and common, the basic steps to add new instrument is similar to the steps
|
||||
to add new workload and an example can be found
|
||||
Each method in ``Instrument`` must take two arguments, which are ``self`` and
|
||||
``context``. Supported methods and their corresponding signals can be found in
|
||||
the :ref:`Signals Documentation <instruments_method_map>`. To make
|
||||
implementations easier and common, the basic steps to add new instrument is
|
||||
similar to the steps to add new workload and an example can be found
|
||||
:ref:`here <adding-an-instrument-example>`.
|
||||
|
||||
.. _instrument-api:
|
||||
@ -787,7 +789,7 @@ The full interface of WA instruments is shown below::
|
||||
"""
|
||||
pass
|
||||
|
||||
This is similar to a Workload, except all methods are optional. In addition to
|
||||
This is similar to a ``Workload``, except all methods are optional. In addition to
|
||||
the workload-like methods, instruments can define a number of other methods that
|
||||
will get invoked at various points during run execution. The most useful of
|
||||
which is perhaps ``initialize`` that gets invoked after the device has been
|
||||
@ -861,11 +863,11 @@ Below is a simple instrument that measures the execution time of a workload::
|
||||
self.start_time = None
|
||||
self.end_time = None
|
||||
|
||||
@fast
|
||||
@very_fast
|
||||
def start(self, context):
|
||||
self.start_time = time.time()
|
||||
|
||||
@fast
|
||||
@very_fast
|
||||
def stop(self, context):
|
||||
self.end_time = time.time()
|
||||
|
||||
@ -887,7 +889,7 @@ generating plots, etc. WA comes with a few output processors that output
|
||||
results in a few common formats (such as csv or JSON).
|
||||
|
||||
You can add your own output processors by creating a Python file in
|
||||
``~/.workload_automation/result_processors`` with a class that derives from
|
||||
``~/.workload_automation/plugins`` with a class that derives from
|
||||
:class:`wa.OutputProcessor <wa.framework.processor.OutputProcessor>`, which has
|
||||
the following interface::
|
||||
|
||||
@ -919,7 +921,7 @@ the following interface::
|
||||
|
||||
|
||||
The method names should be fairly self-explanatory. The difference between
|
||||
"process" and "export" methods is that export methods will be invoke after
|
||||
"process" and "export" methods is that export methods will be invoked after
|
||||
process methods for all output processors have been generated. Process methods
|
||||
may generated additional artifacts (metrics, files, etc), while export methods
|
||||
should not -- the should only handle existing results (upload them to a
|
||||
@ -927,8 +929,7 @@ database, archive on a filer, etc).
|
||||
|
||||
The output object passed to job methods is an instance of
|
||||
:class:`wa.framework.output.JobOutput`, the output object passed to run methods
|
||||
is an instance of :class:`wa.RunOutput <wa.framework.output.RunOutput>`. Please
|
||||
refer to their API documentation for details.
|
||||
is an instance of :class:`wa.RunOutput <wa.framework.output.RunOutput>`.
|
||||
|
||||
|
||||
Adding a Resource Getter
|
||||
@ -987,7 +988,7 @@ should typically be registered as ``preferred``.
|
||||
|
||||
You don't have to stick to standard priority levels (though you should, unless
|
||||
there is a good reason). Any integer is a valid priority. The standard priorities
|
||||
range from 0 to 50 in increments of 10.
|
||||
range from 0 to 40 in increments of 10.
|
||||
|
||||
Example
|
||||
^^^^^^^
|
||||
@ -1036,9 +1037,9 @@ The currently available platforms are:
|
||||
|
||||
The currently available targets from devlib are:
|
||||
:linux: A device running a Linux based OS.
|
||||
:android: A device running the Android OS.
|
||||
:android: A device running Android OS.
|
||||
:local: Used to run locally on a linux based host.
|
||||
:chromeos: A device running chromeos, supporting an android container if available.
|
||||
:chromeos: A device running ChromeOS, supporting an android container if available.
|
||||
|
||||
For an example of adding you own customized version of an existing devlib target,
|
||||
please see the how to section :ref:`Adding a Custom Target <adding-custom-target-example>`.
|
||||
@ -1113,7 +1114,7 @@ management tools, e.g. ::
|
||||
sudo pip install my_wa_exts-0.0.1.tar.gz
|
||||
|
||||
As part of the installation process, the setup.py in the package, will write the
|
||||
package's name into ``~/.workoad_automoation/packages``. This will tell WA that
|
||||
package's name into ``~/.workoad_automation/packages``. This will tell WA that
|
||||
the package contains plugin and it will load them next time it runs.
|
||||
|
||||
.. note:: There are no uninstall hooks in ``setuputils``, so if you ever
|
||||
|
||||
@ -37,8 +37,8 @@ install the application onto the device or source the apk and place into
|
||||
**Q:** I am trying to set a valid runtime parameters however I still receive the error ``"Unknown runtime parameter"``
|
||||
-------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
**A:** PLease ensure you have the corresponding module loaded on the device.
|
||||
Please see :ref:`Runtime Parameters <runtime-parameters>` for the list of
|
||||
**A:** Please ensure you have the corresponding module loaded on the device.
|
||||
See :ref:`Runtime Parameters <runtime-parameters>` for the list of
|
||||
runtime parameters and their containing modules, and the appropriate section in
|
||||
:ref:`setting up a device <setting-up-a-device>` for ensuring it is installed.
|
||||
|
||||
|
||||
@ -11,7 +11,7 @@ then the ``initialize`` method should be decorated with the ``@once``
|
||||
method which is decorated instead. Please note if doing this then any installed
|
||||
paths should be added as class attributes rather than instance variables. As a
|
||||
general rule if binaries are installed as part of ``initialize`` then they
|
||||
should be installed in the complementary ``finalize`` method.
|
||||
should be uninstalled in the complementary ``finalize`` method.
|
||||
|
||||
Part of an example workload demonstrating this is shown below:
|
||||
|
||||
@ -41,11 +41,11 @@ Part of an example workload demonstrating this is shown below:
|
||||
Adding a Workload Examples
|
||||
==========================
|
||||
|
||||
The easiest way to create a new workload is to use the create command. ``wa
|
||||
create workload <args>``. This will use predefined templates to create a
|
||||
workload based on the options that are supplied to be used as a starting point
|
||||
for the workload. For more information on using the create workload command see
|
||||
``wa create workload -h``
|
||||
The easiest way to create a new workload is to use the
|
||||
:ref:`create <create-command>` command. ``wa create workload <args>``. This
|
||||
will use predefined templates to create a workload based on the options that are
|
||||
supplied to be used as a starting point for the workload. For more information
|
||||
on using the create workload command see ``wa create workload -h``
|
||||
|
||||
The first thing to decide is the type of workload you want to create depending
|
||||
on the OS you will be using and the aim of the workload. The are currently 6
|
||||
@ -55,7 +55,7 @@ Once you have decided what type of workload you wish to choose this can be
|
||||
specified with ``-k <workload_kind>`` followed by the workload name. This
|
||||
will automatically generate a workload in the your ``WA_CONFIG_DIR/plugins``. If
|
||||
you wish to specify a custom location this can be provided with ``-p
|
||||
<directory>``
|
||||
<path>``
|
||||
|
||||
Adding a Basic Workload Example
|
||||
--------------------------------
|
||||
@ -68,10 +68,9 @@ This will generate a very basic workload with dummy methods for the workload
|
||||
interface and it is left to the developer to add any required functionality to
|
||||
the workload.
|
||||
|
||||
This example shows a simple workload that times how long it takes to compress a
|
||||
file of a particular size on the device, not all the methods are required to be
|
||||
implements however as many as possible have been used to demonstrate their
|
||||
purpose.
|
||||
Not all the methods are required to be implemented, this example shows how a
|
||||
subset might be used to implement a simple workload that times how long it takes
|
||||
to compress a file of a particular size on the device.
|
||||
|
||||
|
||||
.. note:: This is intended as an example of how to implement the Workload
|
||||
@ -92,7 +91,6 @@ purpose.
|
||||
|
||||
This workload was created for illustration purposes only. It should not be
|
||||
used to collect actual measurements.
|
||||
|
||||
'''
|
||||
|
||||
parameters = [
|
||||
@ -137,9 +135,9 @@ purpose.
|
||||
This method is used to extract any results from the target for
|
||||
example we want to pull the file containing the timing information
|
||||
that we will use to generate metrics for our workload and then we
|
||||
add this file as a artifact as a 'raw' file which once WA has
|
||||
finished processing it will allow it to decide whether to keep the
|
||||
file or not.
|
||||
add this file as an artifact with a 'raw' kind, which means once WA
|
||||
has finished processing it will allow it to decide whether to keep
|
||||
the file or not.
|
||||
"""
|
||||
super(ZipTestWorkload, self).extract_results(context)
|
||||
# Pull the results file to the host
|
||||
@ -149,9 +147,9 @@ purpose.
|
||||
|
||||
def update_output(self, context):
|
||||
"""
|
||||
In this method we can do any generation of metric that we wish to
|
||||
In this method we can do any generation of metrics that we wish to
|
||||
for our workload. In this case we are going to simply convert the
|
||||
times reported to seconds and add them as 'metrics' to WA which can
|
||||
times reported into seconds and add them as 'metrics' to WA which can
|
||||
then be displayed to the user along with any others in a format
|
||||
dependant on which output processors they have enabled for the run.
|
||||
"""
|
||||
@ -343,7 +341,8 @@ command::
|
||||
|
||||
This will generate a revent based workload you will end up with a very similar
|
||||
python file as to the one outlined in generating a :ref:`UiAutomator based
|
||||
workload <apkuiautomator-example>` except without the java automation files.
|
||||
workload <apkuiautomator-example>` however without the accompanying java
|
||||
automation files.
|
||||
|
||||
The main difference between the two is that this workload will subclass
|
||||
``ApkReventWorkload`` instead of ``ApkUiautomatorWorkload`` as shown below.
|
||||
@ -367,7 +366,7 @@ The main difference between the two is that this workload will subclass
|
||||
Adding an Instrument Example
|
||||
=============================
|
||||
This is an example of how we would create a instrument which will trace device
|
||||
errors. For more detailed information please see
|
||||
errors using a custom "trace" binary file. For more detailed information please see
|
||||
:ref:`here <instrument-reference>`. The first thing to do is to subclass
|
||||
:class:`Instrument`, overwrite the variable name with what we want our instrument
|
||||
to be called and locate our binary for our instrument.
|
||||
@ -388,7 +387,7 @@ We then declare and implement the required methods as detailed
|
||||
:ref:`here <instrument-api>`. For the ``initialize`` method, we want to install
|
||||
the executable file to the target so we can use the target's ``install``
|
||||
method which will try to copy the file to a location on the device that
|
||||
supports execution, will change the file mode appropriately and return the
|
||||
supports execution, change the file mode appropriately and return the
|
||||
file path on the target. ::
|
||||
|
||||
def initialize(self, context):
|
||||
@ -398,7 +397,7 @@ Then we implemented the start method, which will simply run the file to start
|
||||
tracing. Supposing that the call to this binary requires some overhead to begin
|
||||
collecting errors we might want to decorate the method with the ``@slow``
|
||||
decorator to try and reduce the impact on other running instruments. For more
|
||||
information on prioritization please see :ref:`here <prioritization>`::
|
||||
information on prioritization please see :ref:`here <prioritization>`. ::
|
||||
|
||||
@slow
|
||||
def start(self, context):
|
||||
@ -406,7 +405,7 @@ information on prioritization please see :ref:`here <prioritization>`::
|
||||
|
||||
Lastly, we need to stop tracing once the workload stops and this happens in the
|
||||
stop method, assuming stopping the collection also require some overhead we have
|
||||
again decorated the method::
|
||||
again decorated the method. ::
|
||||
|
||||
@slow
|
||||
def stop(self, context):
|
||||
@ -425,9 +424,9 @@ example for trace data we will want to pull it to the device and add it as a
|
||||
|
||||
Once we have retrieved the data we can now do any further processing and add any
|
||||
relevant :ref:`Metrics <metrics>` to the :ref:`context <context>`. For this we
|
||||
will use the the ``add_metric`` method and to add the instruments results to the
|
||||
final result for that workload. The method can be passed 4 params, which are the
|
||||
metric `key`, `value`, `unit` and `lower_is_better`. ::
|
||||
will use the the ``add_metric`` method to add the results to the final output
|
||||
for that workload. The method can be passed 4 params, which are the metric
|
||||
`key`, `value`, `unit` and `lower_is_better`. ::
|
||||
|
||||
def update_output(self, context):
|
||||
# parse the file if needs to be parsed, or add result directly to
|
||||
@ -442,7 +441,7 @@ instruments and the code to clear these file goes in teardown method. ::
|
||||
def teardown(self, context):
|
||||
self.target.remove(os.path.join(self.target.working_directory, 'trace.txt'))
|
||||
|
||||
At the very end of the run we would want to uninstall the binary we deployed earlier ::
|
||||
At the very end of the run we would want to uninstall the binary we deployed earlier. ::
|
||||
|
||||
def finalize(self, context):
|
||||
self.target.uninstall(self.binary_name)
|
||||
@ -499,7 +498,7 @@ Next we need to implement any relevant methods, (please see
|
||||
available methods). In this case we only want to implement the
|
||||
``export_run_output`` method as we are not generating any new artifacts and
|
||||
we only care about the overall output rather than the individual job
|
||||
outputs. And the implementation is very simple, it just loops through all
|
||||
outputs. The implementation is very simple, it just loops through all
|
||||
the available metrics for all the available jobs and adds them to a list
|
||||
which is written to file and then added as an :ref:`artifact <artifact>` to
|
||||
the :ref:`context <context>`.
|
||||
@ -514,7 +513,7 @@ the :ref:`context <context>`.
|
||||
class Table(OutputProcessor):
|
||||
|
||||
name = 'table'
|
||||
description = 'Generates a text file containing a column-aligned table with run results.'
|
||||
description = 'Generates a text file containing a column-aligned table of run results.'
|
||||
|
||||
def export_run_output(self, output, target_info):
|
||||
rows = []
|
||||
@ -527,7 +526,7 @@ the :ref:`context <context>`.
|
||||
outfile = output.get_path('table.txt')
|
||||
with open(outfile, 'w') as wfh:
|
||||
write_table(rows, wfh)
|
||||
output.add_artifact('results_table', 'table.txt, 'export')
|
||||
output.add_artifact('results_table', 'table.txt', 'export')
|
||||
|
||||
|
||||
.. _adding-custom-target-example:
|
||||
|
||||
@ -347,8 +347,8 @@ Classifiers
|
||||
------------
|
||||
|
||||
Classifiers can be used in 2 distinct ways, the first use is being supplied in
|
||||
an agenda as a set of key-value pairs which can be used to help identify sub-
|
||||
tests of a run, for example if you have multiple sections in your agenda running
|
||||
an agenda as a set of key-value pairs which can be used to help identify sub-tests
|
||||
of a run, for example if you have multiple sections in your agenda running
|
||||
your workloads at different frequencies you might want to set a classifier
|
||||
specifying which frequencies are being used. These can then be utilized later,
|
||||
for example with the ``csv`` :ref:`output processor <output-processors>` with
|
||||
@ -384,9 +384,9 @@ An example agenda is shown here:
|
||||
The other way that they can used is by being automatically added by some
|
||||
workloads to identify their results metrics and artifacts. For example some
|
||||
workloads perform multiple tests with the same execution run and therefore will
|
||||
use metrics to differentiate between them, For example the ``recentfling``
|
||||
workload will use classifiers to distinguish between which loop a particular
|
||||
result is for or whether it is an average across all loops ran.
|
||||
use metrics to differentiate between them, e.g. the ``recentfling`` workload
|
||||
will use classifiers to distinguish between which loop a particular result is
|
||||
for or whether it is an average across all loops ran.
|
||||
|
||||
The output from the agenda above will produce a csv file similar to what is
|
||||
shown below. Some columns have been omitted for clarity however as can been seen
|
||||
@ -500,7 +500,7 @@ turn override global settings.
|
||||
|
||||
|
||||
|
||||
Augmentationts
|
||||
Augmentations
|
||||
--------------
|
||||
|
||||
Augmentations are plugins that augment the execution of workload jobs with
|
||||
@ -509,7 +509,7 @@ metrics and/or artifacts, such as traces or logs. There are two types of
|
||||
augmentations:
|
||||
|
||||
Instruments
|
||||
These "instrument" a WA run in order to change it's behavior (e.g.
|
||||
These "instrument" a WA run in order to change it's behaviour (e.g.
|
||||
introducing delays between successive job executions), or collect
|
||||
additional measurements (e.g. energy usage). Some instruments may depend
|
||||
on particular features being enabled on the target (e.g. cpufreq), or
|
||||
@ -677,9 +677,11 @@ Each workload will be run in two configurations: once, to collect energy
|
||||
measurements, and once to collect thermal data and kernel trace. Trace can give
|
||||
insight into why a workload is using more or less energy than expected, but it
|
||||
can be relatively intrusive and might impact absolute energy and performance
|
||||
metrics, which is why it is collected separately. classifiers_ are used to
|
||||
metrics, which is why it is collected separately. Classifiers_ are used to
|
||||
separate metrics from the two configurations in the results.
|
||||
|
||||
.. _other-agenda-configuration:
|
||||
|
||||
Other Configuration
|
||||
-------------------
|
||||
|
||||
|
||||
@ -20,12 +20,11 @@ Android
|
||||
General Device Setup
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can specify the device interface by setting ``device`` setting in
|
||||
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by
|
||||
running ``wa list targets`` command. If you don't see your specific platform
|
||||
listed (which is likely unless you're using one of the Arm-supplied platforms), then
|
||||
you should use ``generic_android`` interface (this is set in the config by
|
||||
default).
|
||||
You can specify the device interface by setting ``device`` setting in a
|
||||
``config`` file or section. Available interfaces can be viewed by running ``wa
|
||||
list targets`` command. If you don't see your specific platform listed (which is
|
||||
likely unless you're using one of the Arm-supplied platforms), then you should
|
||||
use ``generic_android`` interface (this is what is used by the default config).
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -38,7 +37,8 @@ common parameters you might want to change are outlined below.
|
||||
.. confval:: device
|
||||
|
||||
If you have multiple Android devices connected to the host machine, you will
|
||||
need to set this to indicate to WA which device you want it to use.
|
||||
need to set this to indicate to WA which device you want it to use. The will
|
||||
be the adb name the is displayed when running ``adb devices``
|
||||
|
||||
.. confval:: working_directory
|
||||
|
||||
@ -81,7 +81,7 @@ A typical ``device_config`` inside ``config.yaml`` may look something like
|
||||
|
||||
device_config:
|
||||
device: 0123456789ABCDEF
|
||||
# ...
|
||||
# ...
|
||||
|
||||
|
||||
or a more specific config could be be
|
||||
@ -172,12 +172,11 @@ Linux
|
||||
General Device Setup
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can specify the device interface by setting ``device`` setting in
|
||||
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by
|
||||
running ``wa list targets`` command. If you don't see your specific platform
|
||||
listed (which is likely unless you're using one of the Arm-supplied platforms), then
|
||||
you should use ``generic_linux`` interface (this is set in the config by
|
||||
default).
|
||||
You can specify the device interface by setting ``device`` setting in a
|
||||
``config`` file or section. Available interfaces can be viewed by running
|
||||
``wa list targets`` command. If you don't see your specific platform listed
|
||||
(which is likely unless you're using one of the Arm-supplied platforms), then
|
||||
you should use ``generic_linux`` interface.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -239,25 +238,24 @@ Chrome OS
|
||||
General Device Setup
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can specify the device interface by setting ``device`` setting in
|
||||
``~/.workload_automation/config.yaml``. Available interfaces can be viewed by
|
||||
You can specify the device interface by setting ``device`` setting in a
|
||||
``config`` file or section. Available interfaces can be viewed by
|
||||
running ``wa list targets`` command. If you don't see your specific platform
|
||||
listed (which is likely unless you're using one of the Arm-supplied platforms), then
|
||||
you should use ``generic_chromeos`` interface (this is set in the config by
|
||||
default).
|
||||
you should use ``generic_chromeos`` interface.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
device: generic_chromeos
|
||||
|
||||
The device interface may be configured through ``device_config`` setting, who's
|
||||
value is a ``dict`` mapping setting names to their values. The chrome os target
|
||||
value is a ``dict`` mapping setting names to their values. The ChromeOS target
|
||||
is essentially the same as a linux device and requires a similar setup, however
|
||||
it also optionally supports connecting to an android container running on the
|
||||
device which will be automatically deted if present. If the device supports
|
||||
device which will be automatically detected if present. If the device supports
|
||||
android applications then the android configuration is also supported. In order
|
||||
to support this then WA will open 2 connections to the device, one via SSH to
|
||||
the main ChomeOS OS and another via ADB to the android container where a limited
|
||||
to support this WA will open 2 connections to the device, one via SSH to
|
||||
the main OS and another via ADB to the android container where a limited
|
||||
subset of functionality can be performed.
|
||||
|
||||
In order to distinguish between the two connections some of the android specific
|
||||
|
||||
@ -94,7 +94,7 @@ command with an example output shown below::
|
||||
INFO Recording(s) are available at: '$WA_USER_DIRECTORY/dependencies/angrybirds_rio/revent_files'
|
||||
|
||||
Once you have made your desired recordings, you can either manually playback
|
||||
individual recordings using the :ref:`reply <replay-command>` command or, with
|
||||
individual recordings using the :ref:`replay <replay-command>` command or, with
|
||||
the recordings in the appropriate dependencies location, simply run the workload
|
||||
using the :ref:`run <run-command>` command and then all the available recordings will be
|
||||
played back automatically.
|
||||
|
||||
@ -227,7 +227,7 @@ which contains addional information about how to build and to use the file.
|
||||
Some WA plugins have additional dependencies that need to be
|
||||
satisfied before they can be used. Not all of these can be provided with WA and
|
||||
so will need to be supplied by the user. They should be placed into
|
||||
``~/.workload_uatomation/dependencies/<extension name>`` so that WA can find
|
||||
``~/.workload_automation/dependencies/<extension name>`` so that WA can find
|
||||
them (you may need to create the directory if it doesn't already exist). You
|
||||
only need to provide the dependencies for workloads you want to use.
|
||||
|
||||
@ -237,16 +237,16 @@ APK Files
|
||||
---------
|
||||
|
||||
APKs are application packages used by Android. These are necessary to install on
|
||||
a device when running an :ref:`ApkWorkload <apk-workload>` or derivative. PLease
|
||||
see the workload description using the :ref:`show command <show-command>` to see
|
||||
a device when running an :ref:`ApkWorkload <apk-workload>` or derivative. Please
|
||||
see the workload description using the :ref:`show <show-command>` command to see
|
||||
which version of the apk the UI automation has been tested with and place the
|
||||
apk in the corresponding Automation may also work with other versions
|
||||
(especially if it's only a minor or revision difference -- major version
|
||||
differences are more likely to contain incompatible UI changes) but this has not
|
||||
been tested. As a general rule we do not guarantee support for the latest
|
||||
version of an app and they are updated on as needed basis. We do however attempt
|
||||
to support backwards compatibility with previous major releases however beyond
|
||||
this support will likely be dropped.
|
||||
apk in the corresponding workloads dependency folder. Automation may also work
|
||||
with other versions (especially if it's only a minor or revision difference --
|
||||
major version differences are more likely to contain incompatible UI changes)
|
||||
but this has not been tested. As a general rule we do not guarantee support for
|
||||
the latest version of an app and they are updated on an as needed basis. We do
|
||||
however attempt to support backwards compatibility with previous major releases
|
||||
however beyond this support will likely be dropped.
|
||||
|
||||
|
||||
Gaming Workloads
|
||||
|
||||
@ -15,12 +15,12 @@ Configuration
|
||||
|
||||
Default configuration file change
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Instead of the standard ``config.py`` file located at ``$WA_USER_HOME/config.py`
|
||||
WA not use a ``confg.yaml`` files which is written in the YAML format instead of
|
||||
python. Additionally upon first invocation WA3 will automatically try and detect
|
||||
whether a WA2 config file is present and convert it to use the new WA3 format.
|
||||
During this process any know parameter name changes should be detected and
|
||||
updated accordingly.
|
||||
Instead of the standard ``config.py`` file located at
|
||||
``$WA_USER_HOME/config.py`` WA now uses a ``confg.yaml`` file (at the same
|
||||
location) which is written in the YAML format instead of python. Additionally
|
||||
upon first invocation WA3 will automatically try and detect whether a WA2 config
|
||||
file is present and convert it to use the new WA3 format. During this process
|
||||
any known parameter name changes should be detected and updated accordingly.
|
||||
|
||||
Plugin Changes
|
||||
^^^^^^^^^^^^^^^
|
||||
@ -78,7 +78,7 @@ the new parameter names should be preferred:
|
||||
|
||||
- The workload parameter :confval:`check_apk` has been renamed to
|
||||
:confval:`prefer_host_package` to be more explicit in it's functionality to indicated
|
||||
whether a package on the target for the host should have priority when
|
||||
whether a package on the target or the host should have priority when
|
||||
searching for a suitable package.
|
||||
|
||||
Individual workload parameters have been attempted to be standardized for the
|
||||
@ -110,7 +110,7 @@ moved into a ``__failed`` subdirectory to allow for quicker analysis.
|
||||
Output API
|
||||
~~~~~~~~~~
|
||||
There is now an Output API which can be used to more easily post process the
|
||||
output from a workload. For more information please see the
|
||||
output from a run. For more information please see the
|
||||
:ref:`Output API <output-api>` documentation.
|
||||
|
||||
|
||||
@ -155,15 +155,15 @@ Workloads
|
||||
|
||||
Python Workload Structure
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
The ``update_results`` method has been split out into 2 stages. There is now
|
||||
``extract_results`` and ``update_output`` which should be used for extracting
|
||||
any results from the target back to the host system and to update the output
|
||||
with any metrics or artefacts for the specific workload iteration respectively.
|
||||
- The ``update_results`` method has been split out into 2 stages. There is now
|
||||
``extract_results`` and ``update_output`` which should be used for extracting
|
||||
any results from the target back to the host system and to update the output
|
||||
with any metrics or artefacts for the specific workload iteration respectively.
|
||||
|
||||
WA now features :ref:`decorators <decorators>` which can be used to allow for more efficient
|
||||
binary deployment and that they are only installed to the device once per run. For
|
||||
more information of implementing this please see
|
||||
:ref:`deploying executables to a target <deploying-executables>`.
|
||||
- WA now features :ref:`decorators <decorators>` which can be used to allow for more efficient
|
||||
binary deployment and that they are only installed to the device once per run. For
|
||||
more information of implementing this please see
|
||||
:ref:`deploying executables to a target <deploying-executables>`.
|
||||
|
||||
|
||||
APK Functionality
|
||||
@ -176,27 +176,25 @@ available as the apk attribute of the workload. This means that for example
|
||||
UiAutomator Java Structure
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Instead of a single ``runUiAutomation`` method to perform all of the UiAutomation,
|
||||
the structure has been refactored into 4 methods that can optionally be overridden.
|
||||
the structure has been refactored into 5 methods that can optionally be overridden.
|
||||
The available methods are ``initialize``, ``setup``, ``runWorkload``, ``extactResults``
|
||||
and ``teardown`` to better mimic the different stages in the python workload.
|
||||
|
||||
|
||||
``initialize`` should have the ``@Before`` tag attached to the method which will cause it
|
||||
to be ran before each of the stages of the workload. This method should be used to retrieve
|
||||
and set any relevant parameters required during the workload.
|
||||
- ``initialize`` should be used to retrieve
|
||||
and set any relevant parameters required during the workload.
|
||||
- ``setup`` should be used to perform any setup required for the workload, for
|
||||
example dismissing popups or configuring and required settings.
|
||||
- ``runWorkload`` should be used to perform the actual measurable work of the workload.
|
||||
- ``extractResults`` should be used to extract any relevant results from the
|
||||
target after the workload has been completed.
|
||||
- ``teardown`` should be used to perform any final clean up of the workload on the target.
|
||||
|
||||
The remaining method all have the ``@Test`` tag attached to the method to indicate that this
|
||||
is a test stage that should be called at the appropriate time.
|
||||
|
||||
``setup`` should be used to perform any setup required for the workload, for
|
||||
example dismissing popups or configuring and required settings.
|
||||
|
||||
``runWorkload`` should be used to perform the actual measurable work of the workload.
|
||||
|
||||
``extractResults`` should be used to extract any relevant results from the
|
||||
target after the workload has been completed.
|
||||
|
||||
``teardown`` should be used to perform any final clean up of the workload on the target.
|
||||
.. note:: The ``initialize`` method should have the ``@Before`` tag attached
|
||||
to the method which will cause it to be ran before each of the stages of
|
||||
the workload. The remaining method should all have the ``@Test`` tag
|
||||
attached to the method to indicate that this is a test stage that should be
|
||||
called at the appropriate time.
|
||||
|
||||
GUI Functionality
|
||||
^^^^^^^^^^^^^^^^^
|
||||
@ -208,28 +206,26 @@ you wish to pass parameters to a UiAuotmator workload you will now need to use
|
||||
|
||||
Attributes
|
||||
^^^^^^^^^^
|
||||
The ``device`` attribute of the workload is now a devlib ``target``. Some of the
|
||||
command names remain the same, however there will be differences. The API can be
|
||||
found here: http://devlib.readthedocs.io/en/latest/target.html however some of
|
||||
the more common changes can be found below:
|
||||
- The old ``package`` attribute has been replaced by ``package_names`` which
|
||||
expects a list of strings which allows for multiple package names to be
|
||||
specified if required. It is also no longer required to explicitly state the
|
||||
launch-able activity, this will be automatically discovered from the apk so this
|
||||
workload attribute can be removed.
|
||||
|
||||
- The ``device`` attribute of the workload is now a devlib ``target``. Some of the
|
||||
command names remain the same, however there will be differences. The API can be
|
||||
found at http://devlib.readthedocs.io/en/latest/target.html however some of
|
||||
the more common changes can be found below:
|
||||
|
||||
|
||||
+----------------------------------------------+---------------------------------+
|
||||
| Original Method | New Method |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.pull_file(file)`` | ``self.target.pull(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.push_file(file)`` | ``self.target.push(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.install_executable(file)`` | ``self.target.install(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.execute(cmd, background=True)`` | ``self.target.background(cmd)``|
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|
||||
|
||||
The old ``package`` attribute has been replaced by ``package_names`` which
|
||||
expects a list of strings which allows for multiple package names to be
|
||||
specified if required. It is also no longer required to explicitly state the
|
||||
launch-able activity, this will be automatically discovered from the apk so this
|
||||
workload attribute can be removed.
|
||||
|
||||
+----------------------------------------------+---------------------------------+
|
||||
| Original Method | New Method |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.pull_file(file)`` | ``self.target.pull(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.push_file(file)`` | ``self.target.push(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.install_executable(file)`` | ``self.target.install(file)`` |
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|``self.device.execute(cmd, background=True)`` | ``self.target.background(cmd)``|
|
||||
+----------------------------------------------+---------------------------------+
|
||||
|
||||
@ -67,7 +67,7 @@ List Command
|
||||
============
|
||||
|
||||
In order to get started with using WA we first we need to find
|
||||
out what is available to use. In order to do this we can use the "list"
|
||||
out what is available to use. In order to do this we can use the :ref:`list <list-command>`
|
||||
command followed by the type of plugin that you wish to see.
|
||||
|
||||
For example to see what workloads are available along with a short description
|
||||
@ -97,7 +97,9 @@ Which will give an output in the format of:
|
||||
|
||||
The same syntax can be used to display ``commands``,
|
||||
``energy_instrument_backends``, ``instruments``, ``output_processors``,
|
||||
``resource_getters`` and ``targets``. Alternatively please see the
|
||||
``resource_getters``, ``targets``. Once you have found the plugin you are
|
||||
looking for you can use the :ref:`show <show-command>` command to display more
|
||||
detailed information. Alternatively please see the
|
||||
:ref:`Plugin Reference <plugin-reference>` for an online version.
|
||||
|
||||
Show Command
|
||||
@ -141,13 +143,14 @@ configuration file which will in turn overwrite the default configuration file.
|
||||
Android
|
||||
-------
|
||||
|
||||
By default, the device is set to 'generic_android'. WA is configured to work
|
||||
with a generic Android device through ``adb``. If you only have one device listed
|
||||
when you execute ``adb devices``, and your device has a standard Android
|
||||
configuration, then no extra configuration is required.
|
||||
By default, the device WA will use is set to 'generic_android'. WA is configured
|
||||
to work with a generic Android device through ``adb``. If you only have one
|
||||
device listed when you execute ``adb devices``, and your device has a standard
|
||||
Android configuration, then no extra configuration is required.
|
||||
|
||||
However, if your device is connected via network, you will have to manually execute
|
||||
``adb connect <device ip>`` so that it appears in the device listing.
|
||||
However, if your device is connected via network, you will have to manually
|
||||
execute ``adb connect <device ip>`` (or specify this in your
|
||||
:ref:`agenda <agenda>`) so that it appears in the device listing.
|
||||
|
||||
If you have multiple devices connected, you will need to tell WA which one you
|
||||
want it to use. You can do that by setting ``device`` in the device_config section.
|
||||
|
||||
@ -12,5 +12,14 @@ User Reference
|
||||
---------------------------------------------------------------
|
||||
|
||||
.. include:: user_reference/configuration.rst
|
||||
|
||||
-------------------
|
||||
|
||||
.. include:: user_reference/invocation.rst
|
||||
|
||||
-------------------
|
||||
|
||||
.. include:: user_reference/output_directory.rst
|
||||
|
||||
|
||||
|
||||
|
||||
@ -16,16 +16,18 @@ Individual sub-commands are discussed in detail below.
|
||||
Run
|
||||
---
|
||||
|
||||
The most common sub-command you will use is ``run``. This will run specified
|
||||
workload(s) and process resulting output. This takes a single mandatory
|
||||
argument that specifies what you want WA to run. This could be either a
|
||||
workload name, or a path to an "agenda" file that allows to specify multiple
|
||||
workloads as well as a lot additional configuration (see :ref:`agenda`
|
||||
section for details). Executing ::
|
||||
The most common sub-command you will use is ``run``. This will run the specified
|
||||
workload(s) and process its resulting output. This takes a single mandatory
|
||||
argument which specifies what you want WA to run. This could be either a workload
|
||||
name, or a path to an agenda" file that allows to specify multiple workloads as
|
||||
well as a lot additional configuration (see :ref:`agenda` section for details).
|
||||
Executing ::
|
||||
|
||||
wa run -h
|
||||
|
||||
Will display help for this subcommand that will look something like this::
|
||||
Will display help for this subcommand that will look something like this:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
usage: wa run [-h] [-c CONFIG] [-v] [--version] [-d DIR] [-f] [-i ID]
|
||||
[--disable INSTRUMENT]
|
||||
@ -130,7 +132,7 @@ will produce something like: ::
|
||||
from the website:
|
||||
|
||||
The whole idea behind this application is to use the same Pi calculation
|
||||
algorithm on every Android Device and check how fast that proccess is.
|
||||
algorithm on every Android Device and check how fast that process is.
|
||||
Better calculation times, conclude to faster Android devices. This way you
|
||||
can also check how lightweight your custom made Android build is. Or not.
|
||||
|
||||
@ -185,9 +187,9 @@ will produce something like: ::
|
||||
|
||||
markers_enabled : boolean
|
||||
If set to ``True``, workloads will insert markers into logs
|
||||
at various points during execution. These markes may be used
|
||||
at various points during execution. These markers may be used
|
||||
by other plugins or post-processing scripts to provide
|
||||
measurments or statistics for specific parts of the workload
|
||||
measurements or statistics for specific parts of the workload
|
||||
execution.
|
||||
|
||||
.. note:: You can also use this command to view global settings by using ``wa show settings``
|
||||
@ -202,9 +204,8 @@ This aids in the creation of new WA-related objects for example agendas and work
|
||||
For more detailed information on creating workloads please see the
|
||||
:ref:`adding a workload <adding-a-workload>` section for more details.
|
||||
|
||||
agendas:
|
||||
As an example to create an agenda that will run the dhrystone and memcpy workloads
|
||||
that will use the status and hwmon augumentations, run each test 3 times and save
|
||||
that will use the status and hwmon augmentations, run each test 3 times and save
|
||||
into the file ``my_agenda.yaml`` the following command can be used::
|
||||
|
||||
wa create agenda dhrystone memcpy status hwmon -i 3 -o my_agenda.yaml
|
||||
@ -271,7 +272,7 @@ using the following command::
|
||||
Record
|
||||
------
|
||||
|
||||
This command simiplifies the process of recording revent files. It will
|
||||
This command simplifies the process of recording revent files. It will
|
||||
automatically deploy revent and has options to automatically open apps and
|
||||
record specified stages of a workload. Revent allows you to record raw inputs
|
||||
such as screen swipes or button presses. This can be useful for recording inputs
|
||||
@ -285,14 +286,14 @@ devices it is obtained from ``/proc/device-tree/model``. - suffix is used by WA
|
||||
to determine which part of the app execution the recording is for, currently
|
||||
these are either ``setup``, ``run``, ``extract_results`` or ``teardown``. All
|
||||
stages except ``run`` are optional for playback and to specify which stages
|
||||
shoule be recorded the ``-s``, ``-r``, ``-e`` or ``-t`` arguments respectively,
|
||||
should be recorded the ``-s``, ``-r``, ``-e`` or ``-t`` arguments respectively,
|
||||
or optionally ``-a`` to indicate all stages should be recorded.
|
||||
|
||||
|
||||
The full set of options for this command are::
|
||||
|
||||
usage: wa record [-h] [-c CONFIG] [-v] [--version] [-d DEVICE] [-o FILE] [-s]
|
||||
[-e] [-t] [-a] [-C] [-p PACKAGE | -w WORKLOAD]
|
||||
[-r] [-e] [-t] [-a] [-C] [-p PACKAGE | -w WORKLOAD]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
@ -307,8 +308,8 @@ The full set of options for this command are::
|
||||
-o FILE, --output FILE
|
||||
Specify the output file
|
||||
-s, --setup Record a recording for setup stage
|
||||
-e, --extract_results
|
||||
Record a recording for extract_results stage
|
||||
-r, --run Record a recording for run stage
|
||||
-e, --extract_results Record a recording for extract_results stage
|
||||
-t, --teardown Record a recording for teardown stage
|
||||
-a, --all Record recordings for available stages
|
||||
-C, --clear Clear app cache before launching it
|
||||
@ -324,8 +325,9 @@ For more information please see :ref:`Revent Recording <revent-recording>`.
|
||||
Replay
|
||||
------
|
||||
|
||||
Along side ``record`` wa also has a command to playback a single recorded revent file.
|
||||
It behaves similar to the ``record`` command taking a subset of the same options allowing you to automatically launch a package on the device ::
|
||||
Alongside ``record`` wa also has a command to playback a single recorded revent
|
||||
file. It behaves similar to the ``record`` command taking a subset of the same
|
||||
options allowing you to automatically launch a package on the device ::
|
||||
|
||||
usage: wa replay [-h] [-c CONFIG] [-v] [--debug] [--version] [-p PACKAGE] [-C]
|
||||
revent
|
||||
|
||||
@ -14,22 +14,27 @@ Example
|
||||
^^^^^^^
|
||||
Say we want to perform an experiment on an Android big.LITTLE devices to compare
|
||||
the power consumption between the big and LITTLE clusters running the dhrystone
|
||||
workload. Assuming we have additional instrumentation active for this device
|
||||
that we can measure the power the device is consuming, to reduce external
|
||||
factors we want to ensure that the device has its screen turned off and that it
|
||||
is in airplane mode turned on for both tests. We will then run 2 :ref:`sections
|
||||
<sections>` will each enable a single cluster on the device, set the cores to their
|
||||
maximum frequency and disable all available idle states.
|
||||
and benchmarkpi workloads. Assuming we have additional instrumentation active
|
||||
for this device that can measure the power the device is consuming, to reduce
|
||||
external factors we want to ensure that the device is in airplane mode turned on
|
||||
for all our tests and the screen is off only for our dhrystone run. We will then
|
||||
run 2 :ref:`sections <sections>` will each enable a single cluster on the
|
||||
device, set the cores to their maximum frequency and disable all available idle
|
||||
states.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
config:
|
||||
runtime_parameters:
|
||||
airplane_mode: true
|
||||
#..
|
||||
workloads:
|
||||
- name: dhrystone
|
||||
iterations: 1
|
||||
runtime_parameters:
|
||||
airplane_mode: true
|
||||
screen_on: false
|
||||
- name: benchmarkpi
|
||||
iterations: 1
|
||||
sections:
|
||||
- id: LITTLES
|
||||
runtime_parameters:
|
||||
@ -167,8 +172,9 @@ CPUIdle
|
||||
^^^^^^^
|
||||
|
||||
:idle_states: A ``string`` or list of strings which can be used to specify what
|
||||
idles states should be enabled for all cores if there are common frequencies
|
||||
available. 'all' and 'none' are also valid entries as a shorthand
|
||||
idles states should be enabled for all cores if there are common
|
||||
idle states available. 'all' and 'none' are also valid entries as a
|
||||
shorthand
|
||||
|
||||
:<core_name>_idle_states: A ``string`` or list of strings which can be used to
|
||||
specify what idles states should be enabled for cores of a particular type
|
||||
@ -200,7 +206,7 @@ Android Specific Runtime Parameters
|
||||
entries are ``NATURAL``, ``LEFT``, ``INVERTED``, ``RIGHT``.
|
||||
|
||||
:screen_on: A ``boolean`` to specify whether the devices screen should be
|
||||
turned on. Defaults to ``true``.
|
||||
turned on. Defaults to ``True``.
|
||||
|
||||
.. _setting-sysfiles:
|
||||
|
||||
@ -209,18 +215,19 @@ Setting Sysfiles
|
||||
In order to perform additional configuration of a target the ``sysfile_values``
|
||||
runtime parameter can be used. The value for this parameter is a mapping (an
|
||||
associative array, in YAML) of file paths onto values that should be written
|
||||
into those files. sysfile_values is the only runtime parameter that is available
|
||||
for any (Linux) device. Other runtime parameters will depend on the specifics of
|
||||
the device used (e.g. its CPU cores configuration) as detailed above.
|
||||
into those files. ``sysfile_values`` is the only runtime parameter that is
|
||||
available for any (Linux) device. Other runtime parameters will depend on the
|
||||
specifics of the device used (e.g. its CPU cores configuration) as detailed
|
||||
above.
|
||||
|
||||
.. note:: By default WA will attempt to verify that the sysfile value was
|
||||
.. note:: By default WA will attempt to verify that the any sysfile values were
|
||||
written correctly by reading the node back and comparing the two values. If
|
||||
you do not wish this check to happen, for example the node you are writing to
|
||||
is write only, you can append an ``!`` to the file path to disable this
|
||||
verification.
|
||||
|
||||
For example the following configuration could be used to enable and verify that cpu0
|
||||
is online, however will not attempt to check that it's governor have been set to
|
||||
is online, however will not attempt to check that its governor have been set to
|
||||
userspace::
|
||||
|
||||
- name: dhrystone
|
||||
|
||||
@ -652,7 +652,7 @@ class RunConfiguration(Configuration):
|
||||
The maximum number of times failed jobs will be retried before
|
||||
giving up. If not set.
|
||||
|
||||
.. note:: this number does not include the original attempt
|
||||
.. note:: This number does not include the original attempt
|
||||
''',
|
||||
),
|
||||
ConfigurationPoint(
|
||||
|
||||
@ -37,8 +37,8 @@ class Geekbench(ApkUiautoWorkload):
|
||||
Geekbench scores are calibrated against a baseline score of 1,000 (which is
|
||||
the score of a single-processor Power Mac G5 @ 1.6GHz). Higher scores are
|
||||
better, with double the score indicating double the performance.
|
||||
The benchmarks fall into one of four categories:
|
||||
|
||||
The benchmarks fall into one of four categories:
|
||||
- integer performance.
|
||||
- floating point performance.
|
||||
- memory performance.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user