mirror of
https://github.com/ARM-software/workload-automation.git
synced 2025-02-20 20:09:11 +00:00
doc: Reorganise Developer Guide/Reference
This commit is contained in:
Marc Bonnici
setrofim
parent
e66ae050a9
commit
116c260bae
@ -49,7 +49,7 @@ help:
|
||||
clean:
|
||||
rm -rf $(BUILDDIR)/*
|
||||
rm -rf source/plugins/*
|
||||
rm -rf source/developer_reference/instrument_method_map.rst
|
||||
rm -rf source/developer_guide/instrument_method_map.rst
|
||||
rm -rf source/run_config/*
|
||||
|
||||
coverage:
|
||||
|
||||
@ -10,7 +10,7 @@ Workload
|
||||
The base :class:`Workload` interface is as follows, and is the base class for
|
||||
all :ref:`workload types <workload-types>`. For more information about to
|
||||
implement your own workload please see the
|
||||
:ref:`Developer How Tos <adding-a-workload>`.
|
||||
:ref:`Developer How Tos <adding-a-workload-example>`.
|
||||
|
||||
All instances of a workload will have the following attributes:
|
||||
|
||||
|
||||
@ -314,7 +314,7 @@ def setup(app):
|
||||
generate_target_documentation('plugins')
|
||||
generate_run_config_documentation('run_config')
|
||||
generate_meta_config_documentation('run_config')
|
||||
generate_instrument_method_map(os.path.join('developer_information', 'developer_reference',
|
||||
generate_instrument_method_map(os.path.join('developer_information', 'developer_guide',
|
||||
'instrument_method_map.rst'))
|
||||
app.add_object_type('confval', 'confval',
|
||||
objname='configuration value',
|
||||
|
||||
@ -34,129 +34,6 @@ This sub-section covers things common to implementing plugins of all types. It
|
||||
is recommended you familiarize yourself with the information here before
|
||||
proceeding onto guidance for specific plugin types.
|
||||
|
||||
.. _context:
|
||||
|
||||
The Context
|
||||
^^^^^^^^^^^
|
||||
|
||||
The majority of methods in plugins accept a context argument. This is an
|
||||
instance of :class:`wa.framework.execution.ExecutionContext`. It contains
|
||||
information about the current state of execution of WA and keeps track of things
|
||||
like which workload is currently running.
|
||||
|
||||
Notable methods of the context are:
|
||||
|
||||
:context.get_resource(resource, strict=True):
|
||||
This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
|
||||
|
||||
:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
|
||||
Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
|
||||
output directory for WA and associate them with a description and/or
|
||||
:ref:`classifier <classifiers>`.
|
||||
|
||||
:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
|
||||
This method should be used to add :ref:`metrics <metrics>` that have been
|
||||
generated from a workload, this will allow WA to process the results
|
||||
accordingly depending on which output processors are enabled.
|
||||
|
||||
Notable attributes of the context are:
|
||||
|
||||
:context.workload:
|
||||
:class:`wa.framework.workload` object that is currently being executed.
|
||||
|
||||
:context.tm:
|
||||
This is the target manager that can be used to access various information
|
||||
about the target including initialization parameters.
|
||||
|
||||
:context.current_job:
|
||||
This is an instance of :class:`wa.framework.job.Job` and contains all
|
||||
the information relevant to the workload job currently being executed.
|
||||
|
||||
:context.current_job.spec:
|
||||
The current workload specification being executed. This is an
|
||||
instance of :class:`wa.framework.configuration.core.JobSpec`
|
||||
and defines the workload and the parameters under which it is
|
||||
being executed.
|
||||
|
||||
:context.current_job.current_iteration:
|
||||
The current iteration of the spec that is being executed. Note that this
|
||||
is the iteration for that spec, i.e. the number of times that spec has
|
||||
been run, *not* the total number of all iterations have been executed so
|
||||
far.
|
||||
|
||||
:context.job_output:
|
||||
This is the output object for the current iteration which
|
||||
is an instance of :class:`wa.framework.output.JobOutput`. It contains
|
||||
the status of the iteration as well as the metrics and artifacts
|
||||
generated by the workload.
|
||||
|
||||
|
||||
In addition to these, context also defines a few useful paths (see below).
|
||||
|
||||
|
||||
Paths
|
||||
^^^^^
|
||||
|
||||
You should avoid using hard-coded absolute paths in your plugins whenever
|
||||
possible, as they make your code too dependent on a particular environment and
|
||||
may mean having to make adjustments when moving to new (host and/or device)
|
||||
platforms. To help avoid hard-coded absolute paths, WA defines a number of
|
||||
standard locations. You should strive to define your paths relative
|
||||
to one of these.
|
||||
|
||||
On the host
|
||||
~~~~~~~~~~~
|
||||
|
||||
Host paths are available through the context object, which is passed to most
|
||||
plugin methods.
|
||||
|
||||
context.run_output_directory
|
||||
This is the top-level output directory for all WA results (by default,
|
||||
this will be "wa_output" in the directory in which WA was invoked.
|
||||
|
||||
context.output_directory
|
||||
This is the output directory for the current iteration. This will an
|
||||
iteration-specific subdirectory under the main results location. If
|
||||
there is no current iteration (e.g. when processing overall run results)
|
||||
this will point to the same location as ``root_output_directory``.
|
||||
|
||||
|
||||
Additionally, the global ``wa.settings`` object exposes on other location:
|
||||
|
||||
settings.dependency_directory
|
||||
this is the root directory for all plugin dependencies (e.g. media
|
||||
files, assets etc) that are not included within the plugin itself.
|
||||
|
||||
As per Python best practice, it is recommended that methods and values in
|
||||
``os.path`` standard library module are used for host path manipulation.
|
||||
|
||||
On the target
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Workloads and instruments have a ``target`` attribute, which is an interface to
|
||||
the target used by WA. It defines the following location:
|
||||
|
||||
target.working_directory
|
||||
This is the directory for all WA-related files on the target. All files
|
||||
deployed to the target should be pushed to somewhere under this location
|
||||
(the only exception being executables installed with ``target.install``
|
||||
method).
|
||||
|
||||
Since there could be a mismatch between path notation used by the host and the
|
||||
target, the ``os.path`` modules should *not* be used for on-target path
|
||||
manipulation. Instead target has an equipment module exposed through
|
||||
``target.path`` attribute. This has all the same attributes and behaves the
|
||||
same way as ``os.path``, but is guaranteed to produce valid paths for the target,
|
||||
irrespective of the host's path notation. For example:
|
||||
|
||||
.. code:: python
|
||||
|
||||
result_file = self.target.path.join(self.target.working_directory, "result.txt")
|
||||
self.command = "{} -a -b -c {}".format(target_binary, result_file)
|
||||
|
||||
.. note:: Output processors, unlike workloads and instruments, do not have their
|
||||
own target attribute as they are designed to be able to be ran offline.
|
||||
|
||||
.. _resource-resolution:
|
||||
|
||||
Dynamic Resource Resolution
|
||||
@ -271,166 +148,6 @@ additional assets should have their on target paths added to the workload's
|
||||
``deployed_assests`` attribute or the corresponding ``remove_assets`` method
|
||||
should also be implemented.
|
||||
|
||||
.. _plugin-parmeters:
|
||||
|
||||
Parameters
|
||||
----------
|
||||
|
||||
All plugins can be parametrized. Parameters are specified using
|
||||
``parameters`` class attribute. This should be a list of
|
||||
:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
|
||||
specified on parameter creation:
|
||||
|
||||
:name:
|
||||
This is the only mandatory argument. The name will be used to create a
|
||||
corresponding attribute in the plugin instance, so it must be a valid
|
||||
Python identifier.
|
||||
|
||||
:kind:
|
||||
This is the type of the value of the parameter. This must be an
|
||||
callable. Normally this should be a standard Python type, e.g. ``int``
|
||||
or ``float``, or one the types defined in :mod:`wa.utils.types`.
|
||||
If not explicitly specified, this will default to ``str``.
|
||||
|
||||
.. note:: Irrespective of the ``kind`` specified, ``None`` is always a
|
||||
valid value for a parameter. If you don't want to allow
|
||||
``None``, then set ``mandatory`` (see below) to ``True``.
|
||||
|
||||
:allowed_values:
|
||||
A list of the only allowed values for this parameter.
|
||||
|
||||
.. note:: For composite types, such as ``list_of_strings`` or
|
||||
``list_of_ints`` in :mod:`wa.utils.types`, each element of
|
||||
the value will be checked against ``allowed_values`` rather
|
||||
than the composite value itself.
|
||||
|
||||
:default:
|
||||
The default value to be used for this parameter if one has not been
|
||||
specified by the user. Defaults to ``None``.
|
||||
|
||||
:mandatory:
|
||||
A ``bool`` indicating whether this parameter is mandatory. Setting this
|
||||
to ``True`` will make ``None`` an illegal value for the parameter.
|
||||
Defaults to ``False``.
|
||||
|
||||
.. note:: Specifying a ``default`` will mean that this parameter will,
|
||||
effectively, be ignored (unless the user sets the param to ``None``).
|
||||
|
||||
.. note:: Mandatory parameters are *bad*. If at all possible, you should
|
||||
strive to provide a sensible ``default`` or to make do without
|
||||
the parameter. Only when the param is absolutely necessary,
|
||||
and there really is no sensible default that could be given
|
||||
(e.g. something like login credentials), should you consider
|
||||
making it mandatory.
|
||||
|
||||
:constraint:
|
||||
This is an additional constraint to be enforced on the parameter beyond
|
||||
its type or fixed allowed values set. This should be a predicate (a function
|
||||
that takes a single argument -- the user-supplied value -- and returns
|
||||
a ``bool`` indicating whether the constraint has been satisfied).
|
||||
|
||||
:override:
|
||||
A parameter name must be unique not only within an plugin but also
|
||||
with that plugin's class hierarchy. If you try to declare a parameter
|
||||
with the same name as already exists, you will get an error. If you do
|
||||
want to override a parameter from further up in the inheritance
|
||||
hierarchy, you can indicate that by setting ``override`` attribute to
|
||||
``True``.
|
||||
|
||||
When overriding, you do not need to specify every other attribute of the
|
||||
parameter, just the ones you what to override. Values for the rest will
|
||||
be taken from the parameter in the base class.
|
||||
|
||||
|
||||
Validation and cross-parameter constraints
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
A plugin will get validated at some point after construction. When exactly
|
||||
this occurs depends on the plugin type, but it *will* be validated before it
|
||||
is used.
|
||||
|
||||
You can implement ``validate`` method in your plugin (that takes no arguments
|
||||
beyond the ``self``) to perform any additional *internal* validation in your
|
||||
plugin. By "internal", I mean that you cannot make assumptions about the
|
||||
surrounding environment (e.g. that the device has been initialized).
|
||||
|
||||
The contract for ``validate`` method is that it should raise an exception
|
||||
(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
|
||||
further on this page) if some validation condition has not, and cannot, been met.
|
||||
If the method returns without raising an exception, then the plugin is in a
|
||||
valid internal state.
|
||||
|
||||
Note that ``validate`` can be used not only to verify, but also to impose a
|
||||
valid internal state. In particular, this where cross-parameter constraints can
|
||||
be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
|
||||
another parameter, there is no way to express that declaratively when specifying
|
||||
the parameters. In that case the dependent attribute should be left unspecified
|
||||
on creation and should instead be set inside ``validate``.
|
||||
|
||||
Logging
|
||||
-------
|
||||
|
||||
Every plugin class has it's own logger that you can access through
|
||||
``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
|
||||
log everything it is doing, so you shouldn't need to add much additional logging
|
||||
for device actions. However you might what to log additional information, e.g.
|
||||
what settings your plugin is using, what it is doing on the host, etc.
|
||||
(Operations on the host will not normally be logged, so your plugin should
|
||||
definitely log what it is doing on the host). One situation in particular where
|
||||
you should add logging is before doing something that might take a significant
|
||||
amount of time, such as downloading a file.
|
||||
|
||||
|
||||
Documenting
|
||||
-----------
|
||||
|
||||
All plugins and their parameter should be documented. For plugins
|
||||
themselves, this is done through ``description`` class attribute. The convention
|
||||
for an plugin description is that the first paragraph should be a short
|
||||
summary description of what the plugin does and why one would want to use it
|
||||
(among other things, this will get extracted and used by ``wa list`` command).
|
||||
Subsequent paragraphs (separated by blank lines) can then provide a more
|
||||
detailed description, including any limitations and setup instructions.
|
||||
|
||||
For parameters, the description is passed as an argument on creation. Please
|
||||
note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
|
||||
parameter, they do not need to be explicitly mentioned in the description (wa
|
||||
documentation utilities will automatically pull those). If the ``default`` is set
|
||||
in ``validate`` or additional cross-parameter constraints exist, this *should*
|
||||
be documented in the parameter description.
|
||||
|
||||
Both plugins and their parameters should be documented using reStructureText
|
||||
markup (standard markup for Python documentation). See:
|
||||
|
||||
http://docutils.sourceforge.net/rst.html
|
||||
|
||||
Aside from that, it is up to you how you document your plugin. You should try
|
||||
to provide enough information so that someone unfamiliar with your plugin is
|
||||
able to use it, e.g. you should document all settings and parameters your
|
||||
plugin expects (including what the valid values are).
|
||||
|
||||
|
||||
Error Notification
|
||||
------------------
|
||||
|
||||
When you detect an error condition, you should raise an appropriate exception to
|
||||
notify the user. The exception would typically be :class:`ConfigError` or
|
||||
(depending the type of the plugin)
|
||||
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
|
||||
All these errors are defined in :mod:`wa.framework.exception` module.
|
||||
|
||||
A :class:`ConfigError` should be raised where there is a problem in configuration
|
||||
specified by the user (either through the agenda or config files). These errors
|
||||
are meant to be resolvable by simple adjustments to the configuration (and the
|
||||
error message should suggest what adjustments need to be made. For all other
|
||||
errors, such as missing dependencies, mis-configured environment, problems
|
||||
performing operations, etc., the plugin type-specific exceptions should be
|
||||
used.
|
||||
|
||||
If the plugin itself is capable of recovering from the error and carrying
|
||||
on, it may make more sense to log an ERROR or WARNING level message using the
|
||||
plugin's logger and to continue operation.
|
||||
|
||||
.. _instrument-reference:
|
||||
|
||||
Adding an Instrument
|
||||
@ -595,7 +312,7 @@ Below is a simple instrument that measures the execution time of a workload::
|
||||
context.add_metric('execution_time', execution_time, 'seconds')
|
||||
|
||||
|
||||
.. include:: developer_information/developer_reference/instrument_method_map.rst
|
||||
.. include:: developer_information/developer_guide/instrument_method_map.rst
|
||||
|
||||
.. _adding-an-output-processor:
|
||||
|
||||
|
||||
@ -32,6 +32,289 @@ Plugin Basics
|
||||
|
||||
This section contains reference information common to plugins of all types.
|
||||
|
||||
.. _context:
|
||||
|
||||
The Context
|
||||
^^^^^^^^^^^
|
||||
|
||||
The majority of methods in plugins accept a context argument. This is an
|
||||
instance of :class:`wa.framework.execution.ExecutionContext`. It contains
|
||||
information about the current state of execution of WA and keeps track of things
|
||||
like which workload is currently running.
|
||||
|
||||
Notable methods of the context are:
|
||||
|
||||
:context.get_resource(resource, strict=True):
|
||||
This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
|
||||
|
||||
:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
|
||||
Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
|
||||
output directory for WA and associate them with a description and/or
|
||||
:ref:`classifier <classifiers>`.
|
||||
|
||||
:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
|
||||
This method should be used to add :ref:`metrics <metrics>` that have been
|
||||
generated from a workload, this will allow WA to process the results
|
||||
accordingly depending on which output processors are enabled.
|
||||
|
||||
Notable attributes of the context are:
|
||||
|
||||
:context.workload:
|
||||
:class:`wa.framework.workload` object that is currently being executed.
|
||||
|
||||
:context.tm:
|
||||
This is the target manager that can be used to access various information
|
||||
about the target including initialization parameters.
|
||||
|
||||
:context.current_job:
|
||||
This is an instance of :class:`wa.framework.job.Job` and contains all
|
||||
the information relevant to the workload job currently being executed.
|
||||
|
||||
:context.current_job.spec:
|
||||
The current workload specification being executed. This is an
|
||||
instance of :class:`wa.framework.configuration.core.JobSpec`
|
||||
and defines the workload and the parameters under which it is
|
||||
being executed.
|
||||
|
||||
:context.current_job.current_iteration:
|
||||
The current iteration of the spec that is being executed. Note that this
|
||||
is the iteration for that spec, i.e. the number of times that spec has
|
||||
been run, *not* the total number of all iterations have been executed so
|
||||
far.
|
||||
|
||||
:context.job_output:
|
||||
This is the output object for the current iteration which
|
||||
is an instance of :class:`wa.framework.output.JobOutput`. It contains
|
||||
the status of the iteration as well as the metrics and artifacts
|
||||
generated by the workload.
|
||||
|
||||
|
||||
In addition to these, context also defines a few useful paths (see below).
|
||||
|
||||
|
||||
Paths
|
||||
^^^^^
|
||||
|
||||
You should avoid using hard-coded absolute paths in your plugins whenever
|
||||
possible, as they make your code too dependent on a particular environment and
|
||||
may mean having to make adjustments when moving to new (host and/or device)
|
||||
platforms. To help avoid hard-coded absolute paths, WA defines a number of
|
||||
standard locations. You should strive to define your paths relative
|
||||
to one of these.
|
||||
|
||||
On the host
|
||||
~~~~~~~~~~~
|
||||
|
||||
Host paths are available through the context object, which is passed to most
|
||||
plugin methods.
|
||||
|
||||
context.run_output_directory
|
||||
This is the top-level output directory for all WA results (by default,
|
||||
this will be "wa_output" in the directory in which WA was invoked.
|
||||
|
||||
context.output_directory
|
||||
This is the output directory for the current iteration. This will an
|
||||
iteration-specific subdirectory under the main results location. If
|
||||
there is no current iteration (e.g. when processing overall run results)
|
||||
this will point to the same location as ``root_output_directory``.
|
||||
|
||||
|
||||
Additionally, the global ``wa.settings`` object exposes on other location:
|
||||
|
||||
settings.dependency_directory
|
||||
this is the root directory for all plugin dependencies (e.g. media
|
||||
files, assets etc) that are not included within the plugin itself.
|
||||
|
||||
As per Python best practice, it is recommended that methods and values in
|
||||
``os.path`` standard library module are used for host path manipulation.
|
||||
|
||||
On the target
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Workloads and instruments have a ``target`` attribute, which is an interface to
|
||||
the target used by WA. It defines the following location:
|
||||
|
||||
target.working_directory
|
||||
This is the directory for all WA-related files on the target. All files
|
||||
deployed to the target should be pushed to somewhere under this location
|
||||
(the only exception being executables installed with ``target.install``
|
||||
method).
|
||||
|
||||
Since there could be a mismatch between path notation used by the host and the
|
||||
target, the ``os.path`` modules should *not* be used for on-target path
|
||||
manipulation. Instead target has an equipment module exposed through
|
||||
``target.path`` attribute. This has all the same attributes and behaves the
|
||||
same way as ``os.path``, but is guaranteed to produce valid paths for the target,
|
||||
irrespective of the host's path notation. For example:
|
||||
|
||||
.. code:: python
|
||||
|
||||
result_file = self.target.path.join(self.target.working_directory, "result.txt")
|
||||
self.command = "{} -a -b -c {}".format(target_binary, result_file)
|
||||
|
||||
.. note:: Output processors, unlike workloads and instruments, do not have their
|
||||
own target attribute as they are designed to be able to be ran offline.
|
||||
|
||||
.. _plugin-parmeters:
|
||||
|
||||
Parameters
|
||||
^^^^^^^^^^^
|
||||
|
||||
All plugins can be parametrized. Parameters are specified using
|
||||
``parameters`` class attribute. This should be a list of
|
||||
:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
|
||||
specified on parameter creation:
|
||||
|
||||
:name:
|
||||
This is the only mandatory argument. The name will be used to create a
|
||||
corresponding attribute in the plugin instance, so it must be a valid
|
||||
Python identifier.
|
||||
|
||||
:kind:
|
||||
This is the type of the value of the parameter. This must be an
|
||||
callable. Normally this should be a standard Python type, e.g. ``int``
|
||||
or ``float``, or one the types defined in :mod:`wa.utils.types`.
|
||||
If not explicitly specified, this will default to ``str``.
|
||||
|
||||
.. note:: Irrespective of the ``kind`` specified, ``None`` is always a
|
||||
valid value for a parameter. If you don't want to allow
|
||||
``None``, then set ``mandatory`` (see below) to ``True``.
|
||||
|
||||
:allowed_values:
|
||||
A list of the only allowed values for this parameter.
|
||||
|
||||
.. note:: For composite types, such as ``list_of_strings`` or
|
||||
``list_of_ints`` in :mod:`wa.utils.types`, each element of
|
||||
the value will be checked against ``allowed_values`` rather
|
||||
than the composite value itself.
|
||||
|
||||
:default:
|
||||
The default value to be used for this parameter if one has not been
|
||||
specified by the user. Defaults to ``None``.
|
||||
|
||||
:mandatory:
|
||||
A ``bool`` indicating whether this parameter is mandatory. Setting this
|
||||
to ``True`` will make ``None`` an illegal value for the parameter.
|
||||
Defaults to ``False``.
|
||||
|
||||
.. note:: Specifying a ``default`` will mean that this parameter will,
|
||||
effectively, be ignored (unless the user sets the param to ``None``).
|
||||
|
||||
.. note:: Mandatory parameters are *bad*. If at all possible, you should
|
||||
strive to provide a sensible ``default`` or to make do without
|
||||
the parameter. Only when the param is absolutely necessary,
|
||||
and there really is no sensible default that could be given
|
||||
(e.g. something like login credentials), should you consider
|
||||
making it mandatory.
|
||||
|
||||
:constraint:
|
||||
This is an additional constraint to be enforced on the parameter beyond
|
||||
its type or fixed allowed values set. This should be a predicate (a function
|
||||
that takes a single argument -- the user-supplied value -- and returns
|
||||
a ``bool`` indicating whether the constraint has been satisfied).
|
||||
|
||||
:override:
|
||||
A parameter name must be unique not only within an plugin but also
|
||||
with that plugin's class hierarchy. If you try to declare a parameter
|
||||
with the same name as already exists, you will get an error. If you do
|
||||
want to override a parameter from further up in the inheritance
|
||||
hierarchy, you can indicate that by setting ``override`` attribute to
|
||||
``True``.
|
||||
|
||||
When overriding, you do not need to specify every other attribute of the
|
||||
parameter, just the ones you what to override. Values for the rest will
|
||||
be taken from the parameter in the base class.
|
||||
|
||||
|
||||
Validation and cross-parameter constraints
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
A plugin will get validated at some point after construction. When exactly
|
||||
this occurs depends on the plugin type, but it *will* be validated before it
|
||||
is used.
|
||||
|
||||
You can implement ``validate`` method in your plugin (that takes no arguments
|
||||
beyond the ``self``) to perform any additional *internal* validation in your
|
||||
plugin. By "internal", I mean that you cannot make assumptions about the
|
||||
surrounding environment (e.g. that the device has been initialized).
|
||||
|
||||
The contract for ``validate`` method is that it should raise an exception
|
||||
(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
|
||||
further on this page) if some validation condition has not, and cannot, been met.
|
||||
If the method returns without raising an exception, then the plugin is in a
|
||||
valid internal state.
|
||||
|
||||
Note that ``validate`` can be used not only to verify, but also to impose a
|
||||
valid internal state. In particular, this where cross-parameter constraints can
|
||||
be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
|
||||
another parameter, there is no way to express that declaratively when specifying
|
||||
the parameters. In that case the dependent attribute should be left unspecified
|
||||
on creation and should instead be set inside ``validate``.
|
||||
|
||||
Logging
|
||||
^^^^^^^
|
||||
|
||||
Every plugin class has it's own logger that you can access through
|
||||
``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
|
||||
log everything it is doing, so you shouldn't need to add much additional logging
|
||||
for device actions. However you might what to log additional information, e.g.
|
||||
what settings your plugin is using, what it is doing on the host, etc.
|
||||
(Operations on the host will not normally be logged, so your plugin should
|
||||
definitely log what it is doing on the host). One situation in particular where
|
||||
you should add logging is before doing something that might take a significant
|
||||
amount of time, such as downloading a file.
|
||||
|
||||
|
||||
Documenting
|
||||
^^^^^^^^^^^
|
||||
|
||||
All plugins and their parameter should be documented. For plugins
|
||||
themselves, this is done through ``description`` class attribute. The convention
|
||||
for an plugin description is that the first paragraph should be a short
|
||||
summary description of what the plugin does and why one would want to use it
|
||||
(among other things, this will get extracted and used by ``wa list`` command).
|
||||
Subsequent paragraphs (separated by blank lines) can then provide a more
|
||||
detailed description, including any limitations and setup instructions.
|
||||
|
||||
For parameters, the description is passed as an argument on creation. Please
|
||||
note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
|
||||
parameter, they do not need to be explicitly mentioned in the description (wa
|
||||
documentation utilities will automatically pull those). If the ``default`` is set
|
||||
in ``validate`` or additional cross-parameter constraints exist, this *should*
|
||||
be documented in the parameter description.
|
||||
|
||||
Both plugins and their parameters should be documented using reStructureText
|
||||
markup (standard markup for Python documentation). See:
|
||||
|
||||
http://docutils.sourceforge.net/rst.html
|
||||
|
||||
Aside from that, it is up to you how you document your plugin. You should try
|
||||
to provide enough information so that someone unfamiliar with your plugin is
|
||||
able to use it, e.g. you should document all settings and parameters your
|
||||
plugin expects (including what the valid values are).
|
||||
|
||||
|
||||
Error Notification
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
When you detect an error condition, you should raise an appropriate exception to
|
||||
notify the user. The exception would typically be :class:`ConfigError` or
|
||||
(depending the type of the plugin)
|
||||
:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
|
||||
All these errors are defined in :mod:`wa.framework.exception` module.
|
||||
|
||||
A :class:`ConfigError` should be raised where there is a problem in configuration
|
||||
specified by the user (either through the agenda or config files). These errors
|
||||
are meant to be resolvable by simple adjustments to the configuration (and the
|
||||
error message should suggest what adjustments need to be made. For all other
|
||||
errors, such as missing dependencies, mis-configured environment, problems
|
||||
performing operations, etc., the plugin type-specific exceptions should be
|
||||
used.
|
||||
|
||||
If the plugin itself is capable of recovering from the error and carrying
|
||||
on, it may make more sense to log an ERROR or WARNING level message using the
|
||||
plugin's logger and to continue operation.
|
||||
|
||||
.. _metrics:
|
||||
|
||||
Metrics
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user