diff --git a/doc/source/developer_information/developer_reference/execution_model.rst b/doc/source/developer_information/developer_reference/execution_model.rst index 9dfa450f..944bd5ae 100644 --- a/doc/source/developer_information/developer_reference/execution_model.rst +++ b/doc/source/developer_information/developer_reference/execution_model.rst @@ -90,7 +90,7 @@ to describe every single thing that happens. communication between the core framework and plugins. After this any required instruments and output processors are initialized and installed. #. Finally, the Executor instantiates a :class:`wa.framework.execution.Runner`, - initializes its job queue with workload specs from the RunConfiguraiton, and + initializes its job queue with workload specs from the RunConfiguration, and kicks it off. #. The Runner performs the run time configuration of the device and goes through the workload specs (in the order defined by ``execution_order`` diff --git a/doc/source/developer_information/developer_reference/plugins.rst b/doc/source/developer_information/developer_reference/plugins.rst index 8f48f2f6..bfb3e0ab 100644 --- a/doc/source/developer_information/developer_reference/plugins.rst +++ b/doc/source/developer_information/developer_reference/plugins.rst @@ -37,6 +37,9 @@ This section contains reference information common to plugins of all types. The Context ~~~~~~~~~~~ +.. note:: For clarification on the meaning of "workload specification" ("spec"), "job" + and "workload" and the distiction between them, please see the :ref:`glossary `. + The majority of methods in plugins accept a context argument. This is an instance of :class:`wa.framework.execution.ExecutionContext`. It contains information about the current state of execution of WA and keeps track of things @@ -86,7 +89,7 @@ Notable attributes of the context are: This is the output object for the current iteration which is an instance of :class:`wa.framework.output.JobOutput`. It contains the status of the iteration as well as the metrics and artifacts - generated by the workload. + generated by the job. In addition to these, context also defines a few useful paths (see below). @@ -153,7 +156,7 @@ irrespective of the host's path notation. For example: self.command = "{} -a -b -c {}".format(target_binary, result_file) .. note:: Output processors, unlike workloads and instruments, do not have their - own target attribute as they are designed to be able to be ran offline. + own target attribute as they are designed to be able to be run offline. .. _plugin-parmeters: diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index 710edd46..b8593d20 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -1,3 +1,5 @@ +.. _glossary: + Glossary ======== @@ -73,9 +75,9 @@ Glossary Job An single execution of a workload. A job is defined by an associated - :term:`spec`. However, multiple jobs can share the same spec; e.g. Even - if you only have 1 workload to run but wanted 5 iterations then 5 - individual jobs will be generated to be ran. + :term:`spec`. However, multiple jobs can share the same spec; + E.g. Even if you only have 1 workload to run but wanted 5 iterations + then 5 individual jobs will be generated to be run. Metric A single numeric measurement or score collected during job execution. @@ -93,11 +95,26 @@ Glossary :term:`job`\ s, and results in a single output directory structure containing job results and metadata. + Section + A set of configurations for how jobs should be run. The + settings in them take less precedence than workload-specific settings. For + every section, all jobs will be run again, with the changes + specified in the section's agenda entry. Sections + are useful for several runs in which global settings change. + Spec A specification of a workload. For example you can have a single workload specification that is then executed multiple times if you desire multiple iterations but the configuration for the workload will - remain the same. + remain the same. In WA2 the term "iteration" used to refer to the same + underlying idea as spec now does. It should be noted however, that this + is no longer the case and an iteration is merely a configuration point + in WA3. Spec is to blueprint as job is to product. WA Workload Automation. The full name of this framework. + + Workload + A workload is the lowest level specification for tasks that need to be run + on a target. A workload can have multiple iterations, and be run additional + multiples of times dependent on the number of sections. diff --git a/doc/source/user_information/user_reference/agenda.rst b/doc/source/user_information/user_reference/agenda.rst index 6ad49aa8..83d94a49 100644 --- a/doc/source/user_information/user_reference/agenda.rst +++ b/doc/source/user_information/user_reference/agenda.rst @@ -59,7 +59,7 @@ An example agenda can be seen here: num_big_cores: 4 num_little_cores: 0 - workloads: # List which workloads should be ran + workloads: # List which workloads should be run - name: benchmarkpi augmentations: - ~trace-cmd # Disable the trace-cmd instrument for this workload @@ -160,15 +160,15 @@ Plugins workloads ^^^^^^^^^ -Here you can specify a list of workloads to be ran. If you wish to run a +Here you can specify a list of workloads to be run. If you wish to run a workload with all default values then you can specify the workload name directly as an entry, otherwise a dict mapping should be provided. Any settings provided here will be the most specific and therefore override any other more generalised configuration for that particular workload spec. The valid entries are as follows: -:workload_name: **(Mandatory)** The name of the workload to be ran -:iterations: Specify how many iterations the workload should be ran +:workload_name: **(Mandatory)** The name of the workload to be run +:iterations: Specify how many iterations the workload should be run :label: Similar to IDs but do not have the uniqueness restriction. If specified, labels will be used by some output processors instead of (or in addition to) the workload name. For example, the csv output processor will put diff --git a/doc/source/user_information/user_reference/configuration.rst b/doc/source/user_information/user_reference/configuration.rst index 525d8ffb..88260e3a 100644 --- a/doc/source/user_information/user_reference/configuration.rst +++ b/doc/source/user_information/user_reference/configuration.rst @@ -21,7 +21,9 @@ if it does not already exist. This file must always exist and will always be loaded. You can add to or override the contents of that file on invocation of Workload Automation by specifying an additional configuration file using ``--config`` option. Variables with specific names will be picked up by the -framework and used to modify the behaviour of Workload automation. +framework and used to modify the behaviour of Workload automation e.g. +the ``iterations`` variable might be specified to tell WA how many times to run +each workload. --------------------- @@ -97,3 +99,9 @@ it is not possible to know the end users intention and WA will error. This functionality allows for defaults for plugins, targets etc. to be configured at a global level and then seamless overridden without the need to remove the high level configuration. + +Dependent on specificity, configuration parameters from different sources will +have different inherent priorities. Within an agenda, the configuration in +"workload" entries wil be more specific than "sections" entries, which in turn +are more specific than parameters in the "config" entry. + diff --git a/doc/source/user_information/user_reference/output_directory.rst b/doc/source/user_information/user_reference/output_directory.rst index e68b94a1..50497a78 100644 --- a/doc/source/user_information/user_reference/output_directory.rst +++ b/doc/source/user_information/user_reference/output_directory.rst @@ -43,6 +43,11 @@ iterations each of ``dhrystone`` and ``memcpy`` workloads with no augmentations enabled, and with the first attempt at the first iteration of dhrystone having failed. +You may notice that a number of directories named ``wk*-x-x`` were generated in the +output directory structure. Each of these directories represents a +:term:`job`. The name of the output directory is as stated :ref:`here `. + + Output Directory Entries ------------------------ @@ -81,6 +86,7 @@ __failed failed and were re-run. This directory contains output directories for the failed attempts. +.. _job_execution_subd: job execution output subdirectory Each subdirectory will be named ``__``, and will, at minimum, contain a ``result.json`` (see above).