mirror of
https://github.com/ARM-software/workload-automation.git
synced 2024-10-06 10:51:13 +01:00
192bebbc23
- Move output processing API docs into developer reference. - Add a section on output directory structure to user reference. - Rewrite the output section of the user guide to provide a very brief overview of the output directory and to link to the two sections above.
133 lines
5.4 KiB
ReStructuredText
133 lines
5.4 KiB
ReStructuredText
.. _output_directory_structure:
|
|
|
|
Output Directory Structure
|
|
==========================
|
|
|
|
This is an overview of WA output directory structure.
|
|
|
|
.. note:: In addition to files and subdirectories described here,
|
|
other content may present in the output directory for
|
|
a run, depending on the enabled augmentations.
|
|
|
|
Overview
|
|
--------
|
|
|
|
The output directory will contain a subdirectory for every job iteration that
|
|
was run, as well as some additional entries. The following diagram illustrates
|
|
the typical structure of WA output directory::
|
|
|
|
wa_output/
|
|
├── __meta/
|
|
│ ├── config.json
|
|
│ ├── jobs.json
|
|
│ ├── raw_config
|
|
│ │ ├── cfg0-config.yaml
|
|
│ │ └── agenda.yaml
|
|
│ ├── run_info.json
|
|
│ └── target_info.json
|
|
├── __failed/
|
|
│ └── wk1-dhrystone-1-attempt1
|
|
├── wk1-dhrystone-1/
|
|
│ └── result.json
|
|
├── wk1-dhrystone-2/
|
|
│ └── result.json
|
|
├── wk2-memcpy-1/
|
|
│ └── result.json
|
|
├── wk2-memcpy-2/
|
|
│ └── result.json
|
|
├── result.json
|
|
└── run.log
|
|
|
|
This is the directory structure that would be generated after running two
|
|
iterations each of ``dhrystone`` and ``memcpy`` workloads with no augmentations
|
|
enabled, and with the first attempt at the first iteration of dhrystone having
|
|
failed.
|
|
|
|
Output Directory Entries
|
|
------------------------
|
|
|
|
result.json
|
|
Contains a JSON structure describing the result of the execution,
|
|
including collected metrics and artifacts. There will be one for each
|
|
job execution, and one for the overall run. The run ``result.json`` will
|
|
only contain metrics/artifacts for the run as a whole, and will not
|
|
contain results for individual jobs.
|
|
|
|
You typically would not access ``result.json`` files directly. Instead
|
|
you would either enable augmentations to format the results in easier to
|
|
manage form (such as CSV table), or use :ref:`output_processing_api` to
|
|
access the results from scripts.
|
|
|
|
|
|
run.log
|
|
This is a log of everything that happened during the run, including all
|
|
interactions with the target, and all the decisions made by the
|
|
framework. The output is equivalent to what you would see on the console
|
|
when running with ``--verbose`` option.
|
|
|
|
.. note:: WA source contains a syntax file for Vim that will color the
|
|
initial part of each log line, in a similar way to what you
|
|
see on the console. This may be useful for quickly spotting
|
|
error and warning messages when scrolling through the log.
|
|
|
|
https://github.com/ARM-software/workload-automation/blob/next/extras/walog.vim
|
|
|
|
__meta
|
|
This directory contains configuration and run metadata. See
|
|
:ref:`config_and_meta` below for details.
|
|
|
|
__failed
|
|
This directory will only be present if one or more job executions has
|
|
failed and were re-run. This directory contains output directories for
|
|
the failed attempts.
|
|
|
|
job execution output subdirectory
|
|
Each subdirectory will be named ``<job id>_<workload label>_<iteration
|
|
number>``, and will, at minimum, contain a ``result.json`` (see above).
|
|
Additionally, it may contain raw output from the workload, and any
|
|
additional artifacts (e.g. traces) generated by augmentations. Finally,
|
|
if workload execution has failed, WA may gather some additional logging
|
|
(such as the UI state at the time of failure) and place it here.
|
|
|
|
|
|
.. _config_and_meta:
|
|
|
|
Configuration and Metadata
|
|
--------------------------
|
|
|
|
As stated above, the ``__meta`` directory contains run configuration and
|
|
metadata. Typically, you would not access these files directly, but would use
|
|
the :ref:`output_processing_api` to query the metadata.
|
|
|
|
For more details about WA configuration see :ref:`configuration-specification`.
|
|
|
|
config.json
|
|
Contains the overall run configuration, such as target interface
|
|
configuration, and job execution order, and various "meta-configuration"
|
|
settings, such as default output path, verbosity level, and logging
|
|
formatting.
|
|
|
|
jobs.json
|
|
Final configuration for all jobs, including enabled augmentations,
|
|
workload and runtime parameters, etc.
|
|
|
|
raw_config
|
|
This directory contains copies of config file(s) and the agenda that
|
|
were parsed in order to generate configuration for this run. Each config
|
|
file is prefixed with ``cfg<N>-``, where ``<N>`` is the number
|
|
indicating the order (with respect to the other other config files) in
|
|
which it was parsed, e.g. ``cfg0-config.yaml`` is always a copy of
|
|
``$WA_USER_DIRECTORY/config.yaml``. The one file without a prefix is the
|
|
agenda.
|
|
|
|
run_info.json
|
|
Run metadata, e.g. duration, start/end timestamps and duration.
|
|
|
|
target_info.json
|
|
Extensive information about the target. This includes information about
|
|
the target's CPUS configuration, kernel and userspace versions, etc. The
|
|
exact content will vary depending on the target type (Android vs Linux)
|
|
and what could accessed on a particular device (e.g. if
|
|
``/proc/config.gz`` exists on the target, the kernel config will be
|
|
included).
|