When an agenda contains multiple sections, the same workload entry might
be used in construction of multiple job specs. Job spec construction may
mangle the workload entry. To prevent this from impacting other
jobs, use a deep copy of the workload entry when constructing a job
spec.
PluginCache.get_plugin_config assumes that no more configuration is to
be processed, and therefore config is final. As such, it is validating
that mandatory parameters are set. This assumption is invalid for
workload_parameters, however, as those need to be resolved on per-spec
basis, and cannot be globally cached.
This commit adds a prameter for get_plugin_config that indicates whether
or not it should consider the config to be final.
- Re-order Status entries so that higher severity entries have higher
enum values.
- Add set_status() to Job that ensures that a status is only set if it
is of higher severity (e.g. a Job that has been marked as PARTIAL by
an instrument will not be overwritten as OK by the runner).
- Retry no generates a new job, rather than re-enqueuing the existing
object; this ensures that the output status is tracked properly.
- Adjust ManagedCallback to set set job status to FAILED if it sees a
WorkloadError, and to PARTIAL other wise. The idea being that
instruments raise WorkloadError if they have a reason to believe
workload did not execute properly and indicated failure even if the
workload itself has failed to detect it (e.g. FPS instrument detecting
crashed content, where the workload might lack any feedback regarding
the crash). Other errors would indicate an issue with the instrument
itself, and so the job is marked as PARTIAL, as there is no reason to
suspect that the workload is at fault and the other results generated
for this execution may be valid.
- What used to be enum.values is now enum.levels.
- Add enum.names and enum.values that are lists of enum's levels' names
and values respectively.
- Add a check on creation to make sure that provided level names do not
conflict with the atomatically created members.
Previously a config point would try to validate the name attribute of the object
it was meant to be setting, rather than what it was meant to be setting it to.
- Implemented result processor infrastructured
- Corrected some status tracking issues (differed between states
and output).
- Added "csv" and "status" result processors (these will be the default
enabled).
Changing the way target descriptions work from a static mapping to
something that is dynamically generated and is extensible via plugins.
Also moving core target implementation stuff under "framework".
Changing the way target descriptions work from a static mapping to
something that is dynamically generated and is extensible via plugins.
Also moving core target implementation stuff under "framework".
