mirror of
https://github.com/ARM-software/workload-automation.git
synced 2025-02-22 04:49:00 +00:00
doc: Reorganise Developer Guide/Reference
This commit is contained in:
Marc Bonnici
setrofim
parent
e66ae050a9
commit
116c260bae
@ -49,7 +49,7 @@ help:
|
|||||||
clean:
|
clean:
|
||||||
rm -rf $(BUILDDIR)/*
|
rm -rf $(BUILDDIR)/*
|
||||||
rm -rf source/plugins/*
|
rm -rf source/plugins/*
|
||||||
rm -rf source/developer_reference/instrument_method_map.rst
|
rm -rf source/developer_guide/instrument_method_map.rst
|
||||||
rm -rf source/run_config/*
|
rm -rf source/run_config/*
|
||||||
|
|
||||||
coverage:
|
coverage:
|
||||||
|
|||||||
@ -10,7 +10,7 @@ Workload
|
|||||||
The base :class:`Workload` interface is as follows, and is the base class for
|
The base :class:`Workload` interface is as follows, and is the base class for
|
||||||
all :ref:`workload types <workload-types>`. For more information about to
|
all :ref:`workload types <workload-types>`. For more information about to
|
||||||
implement your own workload please see the
|
implement your own workload please see the
|
||||||
:ref:`Developer How Tos <adding-a-workload>`.
|
:ref:`Developer How Tos <adding-a-workload-example>`.
|
||||||
|
|
||||||
All instances of a workload will have the following attributes:
|
All instances of a workload will have the following attributes:
|
||||||
|
|
||||||
|
|||||||
@ -314,7 +314,7 @@ def setup(app):
|
|||||||
generate_target_documentation('plugins')
|
generate_target_documentation('plugins')
|
||||||
generate_run_config_documentation('run_config')
|
generate_run_config_documentation('run_config')
|
||||||
generate_meta_config_documentation('run_config')
|
generate_meta_config_documentation('run_config')
|
||||||
generate_instrument_method_map(os.path.join('developer_information', 'developer_reference',
|
generate_instrument_method_map(os.path.join('developer_information', 'developer_guide',
|
||||||
'instrument_method_map.rst'))
|
'instrument_method_map.rst'))
|
||||||
app.add_object_type('confval', 'confval',
|
app.add_object_type('confval', 'confval',
|
||||||
objname='configuration value',
|
objname='configuration value',
|
||||||
|
|||||||
@ -34,129 +34,6 @@ This sub-section covers things common to implementing plugins of all types. It
|
|||||||
is recommended you familiarize yourself with the information here before
|
is recommended you familiarize yourself with the information here before
|
||||||
proceeding onto guidance for specific plugin types.
|
proceeding onto guidance for specific plugin types.
|
||||||
|
|
||||||
.. _context:
|
|
||||||
|
|
||||||
The Context
|
|
||||||
^^^^^^^^^^^
|
|
||||||
|
|
||||||
The majority of methods in plugins accept a context argument. This is an
|
|
||||||
instance of :class:`wa.framework.execution.ExecutionContext`. It contains
|
|
||||||
information about the current state of execution of WA and keeps track of things
|
|
||||||
like which workload is currently running.
|
|
||||||
|
|
||||||
Notable methods of the context are:
|
|
||||||
|
|
||||||
:context.get_resource(resource, strict=True):
|
|
||||||
This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
|
|
||||||
|
|
||||||
:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
|
|
||||||
Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
|
|
||||||
output directory for WA and associate them with a description and/or
|
|
||||||
:ref:`classifier <classifiers>`.
|
|
||||||
|
|
||||||
:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
|
|
||||||
This method should be used to add :ref:`metrics <metrics>` that have been
|
|
||||||
generated from a workload, this will allow WA to process the results
|
|
||||||
accordingly depending on which output processors are enabled.
|
|
||||||
|
|
||||||
Notable attributes of the context are:
|
|
||||||
|
|
||||||
:context.workload:
|
|
||||||
:class:`wa.framework.workload` object that is currently being executed.
|
|
||||||
|
|
||||||
:context.tm:
|
|
||||||
This is the target manager that can be used to access various information
|
|
||||||
about the target including initialization parameters.
|
|
||||||
|
|
||||||
:context.current_job:
|
|
||||||
This is an instance of :class:`wa.framework.job.Job` and contains all
|
|
||||||
the information relevant to the workload job currently being executed.
|
|
||||||
|
|
||||||
:context.current_job.spec:
|
|
||||||
The current workload specification being executed. This is an
|
|
||||||
instance of :class:`wa.framework.configuration.core.JobSpec`
|
|
||||||
and defines the workload and the parameters under which it is
|
|
||||||
being executed.
|
|
||||||
|
|
||||||
:context.current_job.current_iteration:
|
|
||||||
The current iteration of the spec that is being executed. Note that this
|
|
||||||
is the iteration for that spec, i.e. the number of times that spec has
|
|
||||||
been run, *not* the total number of all iterations have been executed so
|
|
||||||
far.
|
|
||||||
|
|
||||||
:context.job_output:
|
|
||||||
This is the output object for the current iteration which
|
|
||||||
is an instance of :class:`wa.framework.output.JobOutput`. It contains
|
|
||||||
the status of the iteration as well as the metrics and artifacts
|
|
||||||
generated by the workload.
|
|
||||||
|
|
||||||
|
|
||||||
In addition to these, context also defines a few useful paths (see below).
|
|
||||||
|
|
||||||
|
|
||||||
Paths
|
|
||||||
^^^^^
|
|
||||||
|
|
||||||
You should avoid using hard-coded absolute paths in your plugins whenever
|
|
||||||
possible, as they make your code too dependent on a particular environment and
|
|
||||||
may mean having to make adjustments when moving to new (host and/or device)
|
|
||||||
platforms. To help avoid hard-coded absolute paths, WA defines a number of
|
|
||||||
standard locations. You should strive to define your paths relative
|
|
||||||
to one of these.
|
|
||||||
|
|
||||||
On the host
|
|
||||||
~~~~~~~~~~~
|
|
||||||
|
|
||||||
Host paths are available through the context object, which is passed to most
|
|
||||||
plugin methods.
|
|
||||||
|
|
||||||
context.run_output_directory
|
|
||||||
This is the top-level output directory for all WA results (by default,
|
|
||||||
this will be "wa_output" in the directory in which WA was invoked.
|
|
||||||
|
|
||||||
context.output_directory
|
|
||||||
This is the output directory for the current iteration. This will an
|
|
||||||
iteration-specific subdirectory under the main results location. If
|
|
||||||
there is no current iteration (e.g. when processing overall run results)
|
|
||||||
this will point to the same location as ``root_output_directory``.
|
|
||||||
|
|
||||||
|
|
||||||
Additionally, the global ``wa.settings`` object exposes on other location:
|
|
||||||
|
|
||||||
settings.dependency_directory
|
|
||||||
this is the root directory for all plugin dependencies (e.g. media
|
|
||||||
files, assets etc) that are not included within the plugin itself.
|
|
||||||
|
|
||||||
As per Python best practice, it is recommended that methods and values in
|
|
||||||
``os.path`` standard library module are used for host path manipulation.
|
|
||||||
|
|
||||||
On the target
|
|
||||||
~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
Workloads and instruments have a ``target`` attribute, which is an interface to
|
|
||||||
the target used by WA. It defines the following location:
|
|
||||||
|
|
||||||
target.working_directory
|
|
||||||
This is the directory for all WA-related files on the target. All files
|
|
||||||
deployed to the target should be pushed to somewhere under this location
|
|
||||||
(the only exception being executables installed with ``target.install``
|
|
||||||
method).
|
|
||||||
|
|
||||||
Since there could be a mismatch between path notation used by the host and the
|
|
||||||
target, the ``os.path`` modules should *not* be used for on-target path
|
|
||||||
manipulation. Instead target has an equipment module exposed through
|
|
||||||
``target.path`` attribute. This has all the same attributes and behaves the
|
|
||||||
same way as ``os.path``, but is guaranteed to produce valid paths for the target,
|
|
||||||
irrespective of the host's path notation. For example:
|
|
||||||
|
|
||||||
.. code:: python
|
|
||||||
|
|
||||||
result_file = self.target.path.join(self.target.working_directory, "result.txt")
|
|
||||||
self.command = "{} -a -b -c {}".format(target_binary, result_file)
|
|
||||||
|
|
||||||
.. note:: Output processors, unlike workloads and instruments, do not have their
|
|
||||||
own target attribute as they are designed to be able to be ran offline.
|
|
||||||
|
|
||||||
.. _resource-resolution:
|
.. _resource-resolution:
|
||||||
|
|
||||||
Dynamic Resource Resolution
|
Dynamic Resource Resolution
|
||||||
@ -271,166 +148,6 @@ additional assets should have their on target paths added to the workload's
|
|||||||
``deployed_assests`` attribute or the corresponding ``remove_assets`` method
|
``deployed_assests`` attribute or the corresponding ``remove_assets`` method
|
||||||
should also be implemented.
|
should also be implemented.
|
||||||
|
|
||||||
.. _plugin-parmeters:
|
|
||||||
|
|
||||||
Parameters
|
|
||||||
----------
|
|
||||||
|
|
||||||
All plugins can be parametrized. Parameters are specified using
|
|
||||||
``parameters`` class attribute. This should be a list of
|
|
||||||
:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
|
|
||||||
specified on parameter creation:
|
|
||||||
|
|
||||||
:name:
|
|
||||||
This is the only mandatory argument. The name will be used to create a
|
|
||||||
corresponding attribute in the plugin instance, so it must be a valid
|
|
||||||
Python identifier.
|
|
||||||
|
|
||||||
:kind:
|
|
||||||
This is the type of the value of the parameter. This must be an
|
|
||||||
callable. Normally this should be a standard Python type, e.g. ``int``
|
|
||||||
or ``float``, or one the types defined in :mod:`wa.utils.types`.
|
|
||||||
If not explicitly specified, this will default to ``str``.
|
|
||||||
|
|
||||||
.. note:: Irrespective of the ``kind`` specified, ``None`` is always a
|
|
||||||
valid value for a parameter. If you don't want to allow
|
|
||||||
``None``, then set ``mandatory`` (see below) to ``True``.
|
|
||||||
|
|
||||||
:allowed_values:
|
|
||||||
A list of the only allowed values for this parameter.
|
|
||||||
|
|
||||||
.. note:: For composite types, such as ``list_of_strings`` or
|
|
||||||
``list_of_ints`` in :mod:`wa.utils.types`, each element of
|
|
||||||
the value will be checked against ``allowed_values`` rather
|
|
||||||
than the composite value itself.
|
|
||||||
|
|
||||||
:default:
|
|
||||||
The default value to be used for this parameter if one has not been
|
|
||||||
specified by the user. Defaults to ``None``.
|
|
||||||
|
|
||||||
:mandatory:
|
|
||||||
A ``bool`` indicating whether this parameter is mandatory. Setting this
|
|
||||||
to ``True`` will make ``None`` an illegal value for the parameter.
|
|
||||||
Defaults to ``False``.
|
|
||||||
|
|
||||||
.. note:: Specifying a ``default`` will mean that this parameter will,
|
|
||||||
effectively, be ignored (unless the user sets the param to ``None``).
|
|
||||||
|
|
||||||
.. note:: Mandatory parameters are *bad*. If at all possible, you should
|
|
||||||
strive to provide a sensible ``default`` or to make do without
|
|
||||||
the parameter. Only when the param is absolutely necessary,
|
|
||||||
and there really is no sensible default that could be given
|
|
||||||
(e.g. something like login credentials), should you consider
|
|
||||||
making it mandatory.
|
|
||||||
|
|
||||||
:constraint:
|
|
||||||
This is an additional constraint to be enforced on the parameter beyond
|
|
||||||
its type or fixed allowed values set. This should be a predicate (a function
|
|
||||||
that takes a single argument -- the user-supplied value -- and returns
|
|
||||||
a ``bool`` indicating whether the constraint has been satisfied).
|
|
||||||
|
|
||||||
:override:
|
|
||||||
A parameter name must be unique not only within an plugin but also
|
|
||||||
with that plugin's class hierarchy. If you try to declare a parameter
|
|
||||||
with the same name as already exists, you will get an error. If you do
|
|
||||||
want to override a parameter from further up in the inheritance
|
|
||||||
hierarchy, you can indicate that by setting ``override`` attribute to
|
|
||||||
``True``.
|
|
||||||
|
|
||||||
When overriding, you do not need to specify every other attribute of the
|
|
||||||
parameter, just the ones you what to override. Values for the rest will
|
|
||||||
be taken from the parameter in the base class.
|
|
||||||
|
|
||||||
|
|
||||||
Validation and cross-parameter constraints
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
A plugin will get validated at some point after construction. When exactly
|
|
||||||
this occurs depends on the plugin type, but it *will* be validated before it
|
|
||||||
is used.
|
|
||||||
|
|
||||||
You can implement ``validate`` method in your plugin (that takes no arguments
|
|
||||||
beyond the ``self``) to perform any additional *internal* validation in your
|
|
||||||
plugin. By "internal", I mean that you cannot make assumptions about the
|
|
||||||
surrounding environment (e.g. that the device has been initialized).
|
|
||||||
|
|
||||||
The contract for ``validate`` method is that it should raise an exception
|
|
||||||
(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
|
|
||||||
further on this page) if some validation condition has not, and cannot, been met.
|
|
||||||
If the method returns without raising an exception, then the plugin is in a
|
|
||||||
valid internal state.
|
|
||||||
|
|
||||||
Note that ``validate`` can be used not only to verify, but also to impose a
|
|
||||||
valid internal state. In particular, this where cross-parameter constraints can
|
|
||||||
be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
|
|
||||||
another parameter, there is no way to express that declaratively when specifying
|
|
||||||
the parameters. In that case the dependent attribute should be left unspecified
|
|
||||||
on creation and should instead be set inside ``validate``.
|
|
||||||
|
|
||||||
Logging
|
|
||||||
-------
|
|
||||||
|
|
||||||
Every plugin class has it's own logger that you can access through
|
|
||||||
``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
|
|
||||||
log everything it is doing, so you shouldn't need to add much additional logging
|
|
||||||
for device actions. However you might what to log additional information, e.g.
|
|
||||||
what settings your plugin is using, what it is doing on the host, etc.
|
|
||||||
(Operations on the host will not normally be logged, so your plugin should
|
|
||||||
definitely log what it is doing on the host). One situation in particular where
|
|
||||||
you should add logging is before doing something that might take a significant
|
|
||||||
amount of time, such as downloading a file.
|
|
||||||
|
|
||||||
|
|
||||||
Documenting
|
|
||||||
-----------
|
|
||||||
|
|
||||||
All plugins and their parameter should be documented. For plugins
|
|
||||||
themselves, this is done through ``description`` class attribute. The convention
|
|
||||||
for an plugin description is that the first paragraph should be a short
|
|
||||||
summary description of what the plugin does and why one would want to use it
|
|
||||||
(among other things, this will get extracted and used by ``wa list`` command).
|
|
||||||
Subsequent paragraphs (separated by blank lines) can then provide a more
|
|
||||||
detailed description, including any limitations and setup instructions.
|
|
||||||
|
|
||||||
For parameters, the description is passed as an argument on creation. Please
|
|
||||||
note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
|
|
||||||
parameter, they do not need to be explicitly mentioned in the description (wa
|
|
||||||
documentation utilities will automatically pull those). If the ``default`` is set
|
|
||||||
in ``validate`` or additional cross-parameter constraints exist, this *should*
|
|
||||||
be documented in the parameter description.
|
|
||||||
|
|
||||||
Both plugins and their parameters should be documented using reStructureText
|
|
||||||
markup (standard markup for Python documentation). See:
|
|
||||||
|
|
||||||
http://docutils.sourceforge.net/rst.html
|
|
||||||
|
|
||||||
Aside from that, it is up to you how you document your plugin. You should try
|
|
||||||
to provide enough information so that someone unfamiliar with your plugin is
|
|
||||||
able to use it, e.g. you should document all settings and parameters your
|
|
||||||
plugin expects (including what the valid values are).
|
|
||||||
|
|
||||||
|
|
||||||
Error Notification
|
|
||||||
------------------
|
|
||||||
|
|
||||||
When you detect an error condition, you should raise an appropriate exception to
|
|
||||||
notify the user. The exception would typically be :class:`ConfigError` or
|
|
||||||
(depending the type of the plugin)
|
|
||||||
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
|
|
||||||
All these errors are defined in :mod:`wa.framework.exception` module.
|
|
||||||
|
|
||||||
A :class:`ConfigError` should be raised where there is a problem in configuration
|
|
||||||
specified by the user (either through the agenda or config files). These errors
|
|
||||||
are meant to be resolvable by simple adjustments to the configuration (and the
|
|
||||||
error message should suggest what adjustments need to be made. For all other
|
|
||||||
errors, such as missing dependencies, mis-configured environment, problems
|
|
||||||
performing operations, etc., the plugin type-specific exceptions should be
|
|
||||||
used.
|
|
||||||
|
|
||||||
If the plugin itself is capable of recovering from the error and carrying
|
|
||||||
on, it may make more sense to log an ERROR or WARNING level message using the
|
|
||||||
plugin's logger and to continue operation.
|
|
||||||
|
|
||||||
.. _instrument-reference:
|
.. _instrument-reference:
|
||||||
|
|
||||||
Adding an Instrument
|
Adding an Instrument
|
||||||
@ -595,7 +312,7 @@ Below is a simple instrument that measures the execution time of a workload::
|
|||||||
context.add_metric('execution_time', execution_time, 'seconds')
|
context.add_metric('execution_time', execution_time, 'seconds')
|
||||||
|
|
||||||
|
|
||||||
.. include:: developer_information/developer_reference/instrument_method_map.rst
|
.. include:: developer_information/developer_guide/instrument_method_map.rst
|
||||||
|
|
||||||
.. _adding-an-output-processor:
|
.. _adding-an-output-processor:
|
||||||
|
|
||||||
|
|||||||
@ -32,6 +32,289 @@ Plugin Basics
|
|||||||
|
|
||||||
This section contains reference information common to plugins of all types.
|
This section contains reference information common to plugins of all types.
|
||||||
|
|
||||||
|
.. _context:
|
||||||
|
|
||||||
|
The Context
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
The majority of methods in plugins accept a context argument. This is an
|
||||||
|
instance of :class:`wa.framework.execution.ExecutionContext`. It contains
|
||||||
|
information about the current state of execution of WA and keeps track of things
|
||||||
|
like which workload is currently running.
|
||||||
|
|
||||||
|
Notable methods of the context are:
|
||||||
|
|
||||||
|
:context.get_resource(resource, strict=True):
|
||||||
|
This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
|
||||||
|
|
||||||
|
:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
|
||||||
|
Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
|
||||||
|
output directory for WA and associate them with a description and/or
|
||||||
|
:ref:`classifier <classifiers>`.
|
||||||
|
|
||||||
|
:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
|
||||||
|
This method should be used to add :ref:`metrics <metrics>` that have been
|
||||||
|
generated from a workload, this will allow WA to process the results
|
||||||
|
accordingly depending on which output processors are enabled.
|
||||||
|
|
||||||
|
Notable attributes of the context are:
|
||||||
|
|
||||||
|
:context.workload:
|
||||||
|
:class:`wa.framework.workload` object that is currently being executed.
|
||||||
|
|
||||||
|
:context.tm:
|
||||||
|
This is the target manager that can be used to access various information
|
||||||
|
about the target including initialization parameters.
|
||||||
|
|
||||||
|
:context.current_job:
|
||||||
|
This is an instance of :class:`wa.framework.job.Job` and contains all
|
||||||
|
the information relevant to the workload job currently being executed.
|
||||||
|
|
||||||
|
:context.current_job.spec:
|
||||||
|
The current workload specification being executed. This is an
|
||||||
|
instance of :class:`wa.framework.configuration.core.JobSpec`
|
||||||
|
and defines the workload and the parameters under which it is
|
||||||
|
being executed.
|
||||||
|
|
||||||
|
:context.current_job.current_iteration:
|
||||||
|
The current iteration of the spec that is being executed. Note that this
|
||||||
|
is the iteration for that spec, i.e. the number of times that spec has
|
||||||
|
been run, *not* the total number of all iterations have been executed so
|
||||||
|
far.
|
||||||
|
|
||||||
|
:context.job_output:
|
||||||
|
This is the output object for the current iteration which
|
||||||
|
is an instance of :class:`wa.framework.output.JobOutput`. It contains
|
||||||
|
the status of the iteration as well as the metrics and artifacts
|
||||||
|
generated by the workload.
|
||||||
|
|
||||||
|
|
||||||
|
In addition to these, context also defines a few useful paths (see below).
|
||||||
|
|
||||||
|
|
||||||
|
Paths
|
||||||
|
^^^^^
|
||||||
|
|
||||||
|
You should avoid using hard-coded absolute paths in your plugins whenever
|
||||||
|
possible, as they make your code too dependent on a particular environment and
|
||||||
|
may mean having to make adjustments when moving to new (host and/or device)
|
||||||
|
platforms. To help avoid hard-coded absolute paths, WA defines a number of
|
||||||
|
standard locations. You should strive to define your paths relative
|
||||||
|
to one of these.
|
||||||
|
|
||||||
|
On the host
|
||||||
|
~~~~~~~~~~~
|
||||||
|
|
||||||
|
Host paths are available through the context object, which is passed to most
|
||||||
|
plugin methods.
|
||||||
|
|
||||||
|
context.run_output_directory
|
||||||
|
This is the top-level output directory for all WA results (by default,
|
||||||
|
this will be "wa_output" in the directory in which WA was invoked.
|
||||||
|
|
||||||
|
context.output_directory
|
||||||
|
This is the output directory for the current iteration. This will an
|
||||||
|
iteration-specific subdirectory under the main results location. If
|
||||||
|
there is no current iteration (e.g. when processing overall run results)
|
||||||
|
this will point to the same location as ``root_output_directory``.
|
||||||
|
|
||||||
|
|
||||||
|
Additionally, the global ``wa.settings`` object exposes on other location:
|
||||||
|
|
||||||
|
settings.dependency_directory
|
||||||
|
this is the root directory for all plugin dependencies (e.g. media
|
||||||
|
files, assets etc) that are not included within the plugin itself.
|
||||||
|
|
||||||
|
As per Python best practice, it is recommended that methods and values in
|
||||||
|
``os.path`` standard library module are used for host path manipulation.
|
||||||
|
|
||||||
|
On the target
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Workloads and instruments have a ``target`` attribute, which is an interface to
|
||||||
|
the target used by WA. It defines the following location:
|
||||||
|
|
||||||
|
target.working_directory
|
||||||
|
This is the directory for all WA-related files on the target. All files
|
||||||
|
deployed to the target should be pushed to somewhere under this location
|
||||||
|
(the only exception being executables installed with ``target.install``
|
||||||
|
method).
|
||||||
|
|
||||||
|
Since there could be a mismatch between path notation used by the host and the
|
||||||
|
target, the ``os.path`` modules should *not* be used for on-target path
|
||||||
|
manipulation. Instead target has an equipment module exposed through
|
||||||
|
``target.path`` attribute. This has all the same attributes and behaves the
|
||||||
|
same way as ``os.path``, but is guaranteed to produce valid paths for the target,
|
||||||
|
irrespective of the host's path notation. For example:
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
result_file = self.target.path.join(self.target.working_directory, "result.txt")
|
||||||
|
self.command = "{} -a -b -c {}".format(target_binary, result_file)
|
||||||
|
|
||||||
|
.. note:: Output processors, unlike workloads and instruments, do not have their
|
||||||
|
own target attribute as they are designed to be able to be ran offline.
|
||||||
|
|
||||||
|
.. _plugin-parmeters:
|
||||||
|
|
||||||
|
Parameters
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
All plugins can be parametrized. Parameters are specified using
|
||||||
|
``parameters`` class attribute. This should be a list of
|
||||||
|
:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
|
||||||
|
specified on parameter creation:
|
||||||
|
|
||||||
|
:name:
|
||||||
|
This is the only mandatory argument. The name will be used to create a
|
||||||
|
corresponding attribute in the plugin instance, so it must be a valid
|
||||||
|
Python identifier.
|
||||||
|
|
||||||
|
:kind:
|
||||||
|
This is the type of the value of the parameter. This must be an
|
||||||
|
callable. Normally this should be a standard Python type, e.g. ``int``
|
||||||
|
or ``float``, or one the types defined in :mod:`wa.utils.types`.
|
||||||
|
If not explicitly specified, this will default to ``str``.
|
||||||
|
|
||||||
|
.. note:: Irrespective of the ``kind`` specified, ``None`` is always a
|
||||||
|
valid value for a parameter. If you don't want to allow
|
||||||
|
``None``, then set ``mandatory`` (see below) to ``True``.
|
||||||
|
|
||||||
|
:allowed_values:
|
||||||
|
A list of the only allowed values for this parameter.
|
||||||
|
|
||||||
|
.. note:: For composite types, such as ``list_of_strings`` or
|
||||||
|
``list_of_ints`` in :mod:`wa.utils.types`, each element of
|
||||||
|
the value will be checked against ``allowed_values`` rather
|
||||||
|
than the composite value itself.
|
||||||
|
|
||||||
|
:default:
|
||||||
|
The default value to be used for this parameter if one has not been
|
||||||
|
specified by the user. Defaults to ``None``.
|
||||||
|
|
||||||
|
:mandatory:
|
||||||
|
A ``bool`` indicating whether this parameter is mandatory. Setting this
|
||||||
|
to ``True`` will make ``None`` an illegal value for the parameter.
|
||||||
|
Defaults to ``False``.
|
||||||
|
|
||||||
|
.. note:: Specifying a ``default`` will mean that this parameter will,
|
||||||
|
effectively, be ignored (unless the user sets the param to ``None``).
|
||||||
|
|
||||||
|
.. note:: Mandatory parameters are *bad*. If at all possible, you should
|
||||||
|
strive to provide a sensible ``default`` or to make do without
|
||||||
|
the parameter. Only when the param is absolutely necessary,
|
||||||
|
and there really is no sensible default that could be given
|
||||||
|
(e.g. something like login credentials), should you consider
|
||||||
|
making it mandatory.
|
||||||
|
|
||||||
|
:constraint:
|
||||||
|
This is an additional constraint to be enforced on the parameter beyond
|
||||||
|
its type or fixed allowed values set. This should be a predicate (a function
|
||||||
|
that takes a single argument -- the user-supplied value -- and returns
|
||||||
|
a ``bool`` indicating whether the constraint has been satisfied).
|
||||||
|
|
||||||
|
:override:
|
||||||
|
A parameter name must be unique not only within an plugin but also
|
||||||
|
with that plugin's class hierarchy. If you try to declare a parameter
|
||||||
|
with the same name as already exists, you will get an error. If you do
|
||||||
|
want to override a parameter from further up in the inheritance
|
||||||
|
hierarchy, you can indicate that by setting ``override`` attribute to
|
||||||
|
``True``.
|
||||||
|
|
||||||
|
When overriding, you do not need to specify every other attribute of the
|
||||||
|
parameter, just the ones you what to override. Values for the rest will
|
||||||
|
be taken from the parameter in the base class.
|
||||||
|
|
||||||
|
|
||||||
|
Validation and cross-parameter constraints
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
A plugin will get validated at some point after construction. When exactly
|
||||||
|
this occurs depends on the plugin type, but it *will* be validated before it
|
||||||
|
is used.
|
||||||
|
|
||||||
|
You can implement ``validate`` method in your plugin (that takes no arguments
|
||||||
|
beyond the ``self``) to perform any additional *internal* validation in your
|
||||||
|
plugin. By "internal", I mean that you cannot make assumptions about the
|
||||||
|
surrounding environment (e.g. that the device has been initialized).
|
||||||
|
|
||||||
|
The contract for ``validate`` method is that it should raise an exception
|
||||||
|
(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
|
||||||
|
further on this page) if some validation condition has not, and cannot, been met.
|
||||||
|
If the method returns without raising an exception, then the plugin is in a
|
||||||
|
valid internal state.
|
||||||
|
|
||||||
|
Note that ``validate`` can be used not only to verify, but also to impose a
|
||||||
|
valid internal state. In particular, this where cross-parameter constraints can
|
||||||
|
be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
|
||||||
|
another parameter, there is no way to express that declaratively when specifying
|
||||||
|
the parameters. In that case the dependent attribute should be left unspecified
|
||||||
|
on creation and should instead be set inside ``validate``.
|
||||||
|
|
||||||
|
Logging
|
||||||
|
^^^^^^^
|
||||||
|
|
||||||
|
Every plugin class has it's own logger that you can access through
|
||||||
|
``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
|
||||||
|
log everything it is doing, so you shouldn't need to add much additional logging
|
||||||
|
for device actions. However you might what to log additional information, e.g.
|
||||||
|
what settings your plugin is using, what it is doing on the host, etc.
|
||||||
|
(Operations on the host will not normally be logged, so your plugin should
|
||||||
|
definitely log what it is doing on the host). One situation in particular where
|
||||||
|
you should add logging is before doing something that might take a significant
|
||||||
|
amount of time, such as downloading a file.
|
||||||
|
|
||||||
|
|
||||||
|
Documenting
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
All plugins and their parameter should be documented. For plugins
|
||||||
|
themselves, this is done through ``description`` class attribute. The convention
|
||||||
|
for an plugin description is that the first paragraph should be a short
|
||||||
|
summary description of what the plugin does and why one would want to use it
|
||||||
|
(among other things, this will get extracted and used by ``wa list`` command).
|
||||||
|
Subsequent paragraphs (separated by blank lines) can then provide a more
|
||||||
|
detailed description, including any limitations and setup instructions.
|
||||||
|
|
||||||
|
For parameters, the description is passed as an argument on creation. Please
|
||||||
|
note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
|
||||||
|
parameter, they do not need to be explicitly mentioned in the description (wa
|
||||||
|
documentation utilities will automatically pull those). If the ``default`` is set
|
||||||
|
in ``validate`` or additional cross-parameter constraints exist, this *should*
|
||||||
|
be documented in the parameter description.
|
||||||
|
|
||||||
|
Both plugins and their parameters should be documented using reStructureText
|
||||||
|
markup (standard markup for Python documentation). See:
|
||||||
|
|
||||||
|
http://docutils.sourceforge.net/rst.html
|
||||||
|
|
||||||
|
Aside from that, it is up to you how you document your plugin. You should try
|
||||||
|
to provide enough information so that someone unfamiliar with your plugin is
|
||||||
|
able to use it, e.g. you should document all settings and parameters your
|
||||||
|
plugin expects (including what the valid values are).
|
||||||
|
|
||||||
|
|
||||||
|
Error Notification
|
||||||
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
When you detect an error condition, you should raise an appropriate exception to
|
||||||
|
notify the user. The exception would typically be :class:`ConfigError` or
|
||||||
|
(depending the type of the plugin)
|
||||||
|
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
|
||||||
|
All these errors are defined in :mod:`wa.framework.exception` module.
|
||||||
|
|
||||||
|
A :class:`ConfigError` should be raised where there is a problem in configuration
|
||||||
|
specified by the user (either through the agenda or config files). These errors
|
||||||
|
are meant to be resolvable by simple adjustments to the configuration (and the
|
||||||
|
error message should suggest what adjustments need to be made. For all other
|
||||||
|
errors, such as missing dependencies, mis-configured environment, problems
|
||||||
|
performing operations, etc., the plugin type-specific exceptions should be
|
||||||
|
used.
|
||||||
|
|
||||||
|
If the plugin itself is capable of recovering from the error and carrying
|
||||||
|
on, it may make more sense to log an ERROR or WARNING level message using the
|
||||||
|
plugin's logger and to continue operation.
|
||||||
|
|
||||||
.. _metrics:
|
.. _metrics:
|
||||||
|
|
||||||
Metrics
|
Metrics
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user