workload-automation/doc/source/developer_reference/output_processing_api.rst

.. _output_processing_api:

Output Processing API
=====================

WA3 now has an output API that can be used to post process a runs
:ref:`Output Directory Structure <output_directory>` which can be performed by using WA's
:class:`RunOutput` object.

This object provides a simple way to access the various aspects of the run. A
few of the most useful attributes of the ``RunOutput`` object are as follows:

- :confval:`jobs` - A list of all the job outputs that were generated for the
  run. These can be used to used to access more detailed information about
  individual jobs for example their ``status``, any ``metrics`` or ``artifacts``
  that were generated etc.
- :confval:`run_config` - A :ref:`RunConfiguration <run-configuration>` object
  that can be used to access all the configuration of the run itself, for
  example the ``reboot_policy``, ``execution_order``, ``device_config`` etc.
- :confval:`target_info` - A :ref:`TargetInfo <target-info-api>` object which can be used to
  access various information about the target that was used during the run for
  example it's ``abi``, ``hostname``, ``os`` etc.
- :confval:`info` - A :ref:`RunInfo <run-info-api>` object that contains
  information about the run itself for example it's duration, name, uuid etc.
- :confval:`settings` - The :ref:`Meta Configuration <meta-configuration>` used
  during the run.
- :confval:`status` - The overall status of the run.
- :confval:`artifacts` - A list containing any run level artifacts (for example
  logs, output generated by output processors etc.).
- :confval:`metrics` - A list of :ref:`Metrics <metrics>` that were generated
  for the overall run.
- :confval:`classifiers` - A dict mapping of any run level
  :ref:`classifiers  <classifiers>`.
- :confval:`augmentations` - A list of the augmentations that were enabled
  during the run.
- :confval:`event_summary` - A condensed summary of any events that occurred
  during the run.
- :confval:`events` - A list of any events that occurred during the run like job
  failures etc.
- :confval:`basepath` - The name of the base WA output directory.
- :confval:`configfile` - The relative file path to the config file generated
  for the run.
- :confval:`failed_dir` - The relative file path to the directory containing any
  failed jobs during the run.
- :confval:`infofile` - The file location of the json serialized file of the
  :class:`RunInfo` object.
- :confval:`list_workloads()` - A method that generated a list of the workload
  names that were executed during the run.
- :confval:`logfile` - A relative file path to the run log.
- :confval:`metadir` - A relative file path to the metadata directory.
- :confval:`targetfile` - The file location of the json serialized file of the
  :class:`TargetInfo` object.


Output Processing API Example
------------------------------

To demonstrate how we can use the output API if we have an existing WA output
called ``wa_output`` in the current working directory we can initialize a
``RunOutput`` as follows:

.. code-block:: python

    In [1]: from wa import RunOutput
       ...:
       ...: output_directory = 'wa_output'
       ...: run_output = RunOutput(output_directory)



From here we can retrieve various information about the run. For example if we
want to see what the overall status of the run was, along with the runtime
parameters and the metrics recorded from the first job was we can do the following:

.. code-block:: python

    In [2]: run_output.status
    Out[2]: OK(7)

    # List all of the jobs for the run
    In [3]: run_output.jobs
    Out[3]:
    [<wa.framework.output.JobOutput at 0x7f70358a1f10>,
     <wa.framework.output.JobOutput at 0x7f70358a1150>,
     <wa.framework.output.JobOutput at 0x7f7035862810>,
     <wa.framework.output.JobOutput at 0x7f7035875090>]

    # Examine the first job that was ran
    In [4]: job_1 = run_output.jobs[0]

    In [5]: job_1.label
    Out[5]: u'dhrystone'

    # Print out all the runtime parameters and their values for this job
    In [6]: for k, v in job_1.spec.runtime_parameters.iteritems():
       ...:     print (k, v)
    (u'airplane_mode': False)
    (u'brightness': 100)
    (u'governor': 'userspace')
    (u'big_frequency': 1700000)
    (u'little_frequency': 1400000)

    # Print out all the metrics avalible for this job
    In [7]: job_1.metrics
    Out[7]:
    [<thread 0 score: 14423105 (+)>,
     <thread 0 DMIPS: 8209 (+)>,
     <thread 1 score: 14423105 (+)>,
     <thread 1 DMIPS: 8209 (+)>,
     <thread 2 score: 14423105 (+)>,
     <thread 2 DMIPS: 8209 (+)>,
     <thread 3 score: 18292638 (+)>,
     <thread 3 DMIPS: 10411 (+)>,
     <thread 4 score: 17045532 (+)>,
     <thread 4 DMIPS: 9701 (+)>,
     <thread 5 score: 14150917 (+)>,
     <thread 5 DMIPS: 8054 (+)>,
     <time: 0.184497 seconds (-)>,
     <total DMIPS: 52793 (+)>,
     <total score: 92758402 (+)>]



We can also retrieve information about the target that the run was performed on
for example:

.. code-block:: python

    # Print out the target's abi:
    In [9]: run_output.target_info.abi
    Out[9]: u'arm64'

    # The os the target was running
    In [9]: run_output.target_info.os
    Out[9]: u'android'

    # And other information about the os version
    In [10]: run_output.target_info.os_version
    Out[10]:
    OrderedDict([(u'all_codenames', u'REL'),
                 (u'incremental', u'3687331'),
                 (u'preview_sdk', u'0'),
                 (u'base_os', u''),
                 (u'release', u'7.1.1'),
                 (u'codename', u'REL'),
                 (u'security_patch', u'2017-03-05'),
                 (u'sdk', u'25')])