sascha/workload-automation
1
0
Fork 0
mirror of https://github.com/ARM-software/workload-automation.git synced 2025-04-20 09:40:50 +01:00
Code Issues Releases Wiki Activity

doc/dev_reference: Update to use correct API formatting

Browse Source 
Update the interfaces for Workloads OutputProcessors and Instruments to be
formatted as an API correctly.
This commit is contained in:
Marc Bonnici 2018-06-21 17:51:01 +01:00 committed by setrofim
parent 9d160b1381
commit 750e206b85
1 changed files with 160 additions and 107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

267
doc/source/developer_information/developer_reference/writing_plugins.rst
View File

@ -747,33 +747,58 @@ Workload Interface
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
The workload interface should be implemented as follows: The workload interface should be implemented as follows:
:name: This identifies the workload (e.g. it is used to specify the workload .. class:: <workload_type>(TargetedPlugin)
in the :ref:`agenda <agenda>`).
:init_resources: This method may be optionally overridden to implement dynamic .. attribute:: name
resource discovery for the workload. This method executes
early on, before the device has been initialized, so it This identifies the workload (e.g. it is used to specify the
should only be used to initialize resources that do not workload in the :ref:`agenda <agenda>`).
depend on the device to resolve. This method is executed
once per run for each workload instance. .. method:: init_resources(context)
:validate: This method can be used to validate any assumptions your workload
makes about the environment (e.g. that required files are This method may be optionally overridden to implement dynamic
present, environment variables are set, etc) and should raise a resource discovery for the workload. This method executes
:class:`wa.WorkloadError <wa.framework.exception.WorkloadError>` early on, before the device has been initialized, so it
if that is not the case. The base class implementation only makes should only be used to initialize resources that do not
sure sure that the name attribute has been set. depend on the device to resolve. This method is executed
:initialize: This method is decorated with the ``@once_per_instance`` decorator, once per run for each workload instance.
(for more information please see `Execution Decorators`_)
therefore it will be executed exactly once per run (no matter .. method:: validate(
how many instances of the workload there are). It will run This method can be used to validate any assumptions your workload
after the device has been initialized, so it may be used to makes about the environment (e.g. that required files are
perform device-dependent initialization that does not need to present, environment variables are set, etc) and should raise a
be repeated on each iteration (e.g. as installing executables :class:`wa.WorkloadError <wa.framework.exception.WorkloadError>`
required by the workload on the device). if that is not the case. The base class implementation only makes
:setup: Everything that needs to be in place for workload execution should sure sure that the name attribute has been set.
.. method:: initialize(context)
This method is decorated with the ``@once_per_instance`` decorator,
(for more information please see `Execution Decorators`_)
therefore it will be executed exactly once per run (no matter
how many instances of the workload there are). It will run
after the device has been initialized, so it may be used to
perform device-dependent initialization that does not need to
be repeated on each iteration (e.g. as installing executables
required by the workload on the device).
.. method:: setup(context)
Everything that needs to be in place for workload execution should
be done in this method. This includes copying files to the device, be done in this method. This includes copying files to the device,
starting up an application, configuring communications channels, starting up an application, configuring communications channels,
etc. etc.
:run: This method should perform the actual task that is being measured.
.. method:: setup_rerun(context)
Everything that needs to be in place for workload execution should
be done in this method. This includes copying files to the device,
starting up an application, configuring communications channels,
etc.
.. method:: run(context)
This method should perform the actual task that is being measured.
When this method exits, the task is assumed to be complete. When this method exits, the task is assumed to be complete.
.. note:: Instruments are kicked off just before calling this .. note:: Instruments are kicked off just before calling this
@ -784,17 +809,29 @@ The workload interface should be implemented as follows:
installing or starting applications, processing results, or installing or starting applications, processing results, or
copying files to/from the device should be done elsewhere if copying files to/from the device should be done elsewhere if
possible. possible.
:extract_results: This method gets invoked after the task execution has
finished and should be used to extract metrics from the target. .. method:: extract_results(context)
:update_output: This method should be used to update the output within the
specified execution context with the metrics and artifacts This method gets invoked after the task execution has finished and
from this workload iteration. should be used to extract metrics from the target.
:teardown: This could be used to perform any cleanup you may wish to do,
e.g. Uninstalling applications, deleting file on the device, etc. .. method:: update_output(context)
:finalize: This is the complement to ``initialize``. This will be executed
exactly once at the end of the run. This should be used to This method should be used to update the output within the specified
perform any final clean up (e.g. uninstalling binaries installed execution context with the metrics and artifacts from this
in the ``initialize``). workload iteration.
.. method:: teardown(context)
This could be used to perform any cleanup you may wish to do, e.g.
Uninstalling applications, deleting file on the device, etc.
.. method:: finalize(context)
This is the complement to ``initialize``. This will be executed
exactly once at the end of the run. This should be used to
perform any final clean up (e.g. uninstalling binaries installed
in the ``initialize``).
Workload methods (except for ``validate``) take a single argument that is a Workload methods (except for ``validate``) take a single argument that is a
:class:`wa.framework.execution.ExecutionContext` instance. This object keeps :class:`wa.framework.execution.ExecutionContext` instance. This object keeps
@ -864,69 +901,62 @@ similar to the steps to add new workload and an example can be found in the
.. _instrument-api: .. _instrument-api:
The full interface of WA instruments is shown below:: To implement your own the relevant methods of the interface shown below should be implemented:
class Instrument(Plugin): .. class:: Instrument(TargetedInstrument)
name = None .. attribute:: name
description = None
parameters = [ The name of the instrument, this must be unique to WA.
]
def initialize(self, context): .. attribute:: description
"""
This method will only be called once during the workload run
therefore operations that only need to be performed initially should
be performed her for example pushing the files to the target device,
installing them.
"""
pass
def setup(self, context): A description of what the instrument can be used for.
"""
This method is invoked after the workload is setup. All the
necessary setup should go inside this method. Setup, includes
operations like clearing logs, additional configuration etc.
"""
pass
def start(self, context): .. attribute:: parameters
"""
It is invoked just before the workload start execution. Here is
where instrument measures start being registered/taken.
"""
pass
def stop(self, context): A list of additional :class:`Parameters` the instrument can take.
"""
It is invoked just after the workload execution stops. The measures
should stop being taken/registered.
"""
pass
def update_output(self, context): .. method:: initialize(context):
"""
It is invoked after the workload updated its result.
update_result is where the taken measures are added to the result so it
can be processed by Workload Automation.
"""
pass
def teardown(self, context): This method will only be called once during the workload run
""" therefore operations that only need to be performed initially should
It is invoked after the workload is teared down. It is a good place be performed here for example pushing the files to the target device,
to clean any logs generated by the instrument. installing them.
"""
pass .. method:: setup(context):
This method is invoked after the workload is setup. All the
necessary setup should go inside this method. Setup, includes
operations like clearing logs, additional configuration etc.
.. method:: start(context):
It is invoked just before the workload start execution. Here is
where instrument measures start being registered/taken.
.. method:: stop(context):
It is invoked just after the workload execution stops. The measures
should stop being taken/registered.
.. method:: update_output(context):
It is invoked after the workload updated its result.
update_result is where the taken measures are added to the result so it
can be processed by Workload Automation.
.. method:: teardown(context):
It is invoked after the workload is torn down. It is a good place
to clean any logs generated by the instrument.
.. method:: finalize(context):
This method is the complement to the initialize method and will also
only be called once so should be used to deleting/uninstalling files
pushed to the device.
def finalize(self, context):
"""
This method is the complement to the initialize method and will also
only be called once so should be used to deleting/uninstalling files
pushed to the device.
"""
pass
This is similar to a ``Workload``, except all methods are optional. In addition to This is similar to a ``Workload``, except all methods are optional. In addition to
the workload-like methods, instruments can define a number of other methods that the workload-like methods, instruments can define a number of other methods that
@ -1029,34 +1059,57 @@ results in a few common formats (such as csv or JSON).
You can add your own output processors by creating a Python file in You can add your own output processors by creating a Python file in
``~/.workload_automation/plugins`` with a class that derives from ``~/.workload_automation/plugins`` with a class that derives from
:class:`wa.OutputProcessor <wa.framework.processor.OutputProcessor>`, which has :class:`wa.OutputProcessor <wa.framework.processor.OutputProcessor>`, and should
the following interface:: implement the relevant methods from the following interface:
class OutputProcessor(Plugin): .. class:: OutputProcessor(Plugin):
name = None .. attribute:: name
description = None
parameters = [ The name of the output processor, this must be unique to WA.
]
def initialize(self): .. attribute:: description
pass
def process_job_output(self, output, target_info, run_ouput): A description of what the output processor can be used for.
pass
def export_job_output(self, output, target_info, run_ouput): .. attribute:: parameters
pass
def process_run_output(self, output, target_info): A list of additional :class:`Parameters` the output processor can take.
pass
def export_run_output(self, output, target_info): .. method:: initialize():
pass
def finalize(self): This method will only be called once during the workload run
pass therefore operations that only need to be performed initially should
be performed here.
.. method:: process_job_output(output, target_info, run_ouput):
This method should be used to perform the processing of the
output from an individual job output. This is where any
additional artifacts should be generated if applicable.
.. method:: export_job_output(output, target_info, run_ouput):
This method should be used to perform the exportation of the
existing data collected/generated for an individual job. E.g.
uploading them to a database etc.
.. method:: process_run_output(output, target_info):
This method should be used to perform the processing of the
output from the run as a whole. This is where any
additional artifacts should be generated if applicable.
.. method:: export_run_output(output, target_info):
This method should be used to perform the exportation of the
existing data collected/generated for the run as a whole. E.g.
uploading them to a database etc.
.. method:: finalize():
This method is the complement to the initialize method and will also
only be called once.
The method names should be fairly self-explanatory. The difference between The method names should be fairly self-explanatory. The difference between