Docs: Run Output - Clarification and Additions

Clarification of the meanings of and the distinction between the
output-related terms iteration, job, workload and section for those
unfamiliar
- Add entries to the glossary and references to the glossary
- Added section on Configuration parameter hierarchy
- Added section on the number and names of directories generated for
jobs in the run output folder

doc/source/developer_information/developer_reference/execution_model.rst

@@ -90,7 +90,7 @@ to describe every single thing that happens.
    communication between the core framework and plugins. After this any required
    instruments and output processors are initialized and installed.
 #. Finally, the Executor instantiates a :class:`wa.framework.execution.Runner`,
-   initializes its job queue with workload specs from the RunConfiguraiton, and
+   initializes its job queue with workload specs from the RunConfiguration, and
    kicks it off.
 #. The Runner performs the run time configuration of the device and goes
    through the workload specs (in the order defined by ``execution_order``

doc/source/developer_information/developer_reference/plugins.rst

@@ -37,6 +37,9 @@ This section contains reference information common to plugins of all types.
 The Context
 ~~~~~~~~~~~
 
+.. note:: For clarification on the meaning of "workload specification" ("spec"), "job"
+  and "workload" and the distiction between them, please see the :ref:`glossary <glossary>`.
+
 The majority of methods in plugins accept a context argument. This is an
 instance of :class:`wa.framework.execution.ExecutionContext`. It contains
 information about the current state of execution of WA and keeps track of things
@@ -86,7 +89,7 @@ Notable attributes of the context are:
         This is the output object for the current iteration which
         is an instance of :class:`wa.framework.output.JobOutput`. It contains
         the status of the iteration as well as the metrics and artifacts
-        generated by the workload.
+        generated by the job.
 
 
 In addition to these, context also defines a few useful paths (see below).
@@ -153,7 +156,7 @@ irrespective of the host's path notation. For example:
     self.command = "{} -a -b -c {}".format(target_binary, result_file)
 
 .. note:: Output processors, unlike workloads and instruments, do not have their
-          own target attribute as they are designed to be able to be ran offline.
+          own target attribute as they are designed to be able to be run offline.
 
 .. _plugin-parmeters:
 

doc/source/glossary.rst

@@ -1,3 +1,5 @@
+.. _glossary:
+
 Glossary
 ========
 
@@ -73,9 +75,9 @@ Glossary
 
     Job
         An single execution of a workload. A job is defined by an associated
-        :term:`spec`. However, multiple jobs can share the same spec; e.g. Even
-        if you only have 1 workload to run but wanted 5 iterations then 5
-        individual jobs will be generated to be ran.
+        :term:`spec`. However, multiple jobs can share the same spec;
+        E.g. Even if you only have 1 workload to run but wanted 5 iterations
+        then 5 individual jobs will be generated to be run.
 
     Metric
         A single numeric measurement or score collected during job execution.
@@ -93,11 +95,26 @@ Glossary
         :term:`job`\ s, and results in a single output directory structure
         containing job results and metadata.
 
+    Section
+        A set of configurations for how jobs should be run. The
+        settings in them take less precedence than workload-specific settings. For
+        every section, all jobs will be run again, with the changes
+        specified in the section's agenda entry. Sections
+        are useful for several runs in which global settings change.
+
     Spec
         A specification of a workload. For example you can have a single
         workload specification that is then executed multiple times if you
         desire multiple iterations but the configuration for the workload will
-        remain the same.
+        remain the same. In WA2 the term "iteration" used to refer to the same
+        underlying idea as spec now does. It should be noted however, that this
+        is no longer the case and an iteration is merely a configuration point
+        in WA3. Spec is to blueprint as job is to product.
 
     WA
         Workload Automation. The full name of this framework.
+
+    Workload
+        A workload is the lowest level specification for tasks that need to be run
+        on a target. A workload can have multiple iterations, and be run additional
+        multiples of times dependent on the number of sections.

doc/source/user_information/user_reference/agenda.rst

@@ -59,7 +59,7 @@ An example agenda can be seen here:
                 num_big_cores: 4
                 num_little_cores: 0
 
-    workloads:                  # List which workloads should be ran
+    workloads:                  # List which workloads should be run
     -   name: benchmarkpi
         augmentations:
             - ~trace-cmd        # Disable the trace-cmd instrument for this workload
@@ -160,15 +160,15 @@ Plugins
 workloads
 ^^^^^^^^^
 
-Here you can specify a list of workloads to be ran. If you wish to run a
+Here you can specify a list of workloads to be run. If you wish to run a
 workload with all default values then you can specify the workload name directly
 as an entry, otherwise a dict mapping should be provided. Any settings provided
 here will be the most specific and therefore override any other more generalised
 configuration for that particular workload spec. The valid entries are as
 follows:
 
-:workload_name: **(Mandatory)** The name of the workload to be ran
-:iterations: Specify how many iterations the workload should be ran
+:workload_name: **(Mandatory)** The name of the workload to be run
+:iterations: Specify how many iterations the workload should be run
 :label: Similar to IDs but do not have the uniqueness restriction.
     If specified, labels will be used by some output processors instead of (or in
     addition to) the workload name. For example, the csv output processor will put

doc/source/user_information/user_reference/configuration.rst

@@ -21,7 +21,9 @@ if it does not already exist. This file must always exist and will always be
 loaded. You can add to or override the contents of that file on invocation of
 Workload Automation by specifying an additional configuration file using
 ``--config`` option. Variables with specific names  will be picked up by the
-framework and used to modify the behaviour of Workload automation.
+framework and used to modify the behaviour of Workload automation e.g.
+the ``iterations`` variable might be specified to tell WA how many times to run
+each workload.
 
 ---------------------
 
@@ -97,3 +99,9 @@ it is not possible to know the end users intention and WA will error.
 This functionality allows for defaults for plugins, targets etc. to be
 configured at a global level and then seamless overridden without the need to
 remove the high level configuration.
+
+Dependent on specificity, configuration parameters from different sources will
+have different inherent priorities. Within an agenda, the configuration in
+"workload" entries wil be more specific than "sections" entries, which in turn
+are more specific than parameters in the "config" entry.
+

doc/source/user_information/user_reference/output_directory.rst

@@ -43,6 +43,11 @@ iterations each of ``dhrystone`` and ``memcpy`` workloads with no augmentations
 enabled, and with the first attempt at the first iteration of dhrystone having
 failed.
 
+You may notice that a number of directories named ``wk*-x-x`` were generated in the
+output directory structure. Each of these directories represents a
+:term:`job`. The name of the output directory is as stated :ref:`here <job_execution_subd>`.
+
+
 Output Directory Entries
 ------------------------
 
@@ -81,6 +86,7 @@ __failed
         failed and were re-run. This directory contains output directories for
         the failed attempts.
 
+.. _job_execution_subd:
 job execution output subdirectory
         Each subdirectory will be named ``<job id>_<workload label>_<iteration
         number>``, and will, at minimum, contain a ``result.json`` (see above).