mirror of
https://github.com/ARM-software/workload-automation.git
synced 2025-01-18 12:06:08 +00:00
22750b15c7
With the perf instrument configured as:
perf:
perf_type: simpleperf
command: stat
optionstring: '-a --interval-only-values --csv'
WA fails to parse simpleperf's output:
INFO Extracting reports from target...
ERROR Error in instrument perf
ERROR File "/work/workload_automation/workload-automation/wa/framework/instrument.py", line 272, in __call__
ERROR self.callback(context)
ERROR File "/work/workload_automation/workload-automation/wa/instruments/perf.py", line 142, in update_output
ERROR self._process_simpleperf_output(context)
ERROR File "/work/workload_automation/workload-automation/wa/instruments/perf.py", line 155, in _process_simpleperf_output
ERROR self._process_simpleperf_stat_output(context)
ERROR File "/work/workload_automation/workload-automation/wa/instruments/perf.py", line 233, in _process_simpleperf_stat_output
ERROR self._process_simpleperf_stat_from_csv(stat_file, context, label)
ERROR File "/work/workload_automation/workload-automation/wa/instruments/perf.py", line 245, in _process_simpleperf_stat_from_csv
ERROR context.add_metric('{}_{}'.format(label, row[1]), row[0], 'count', classifiers=classifiers)
ERROR File "/work/workload_automation/workload-automation/wa/framework/execution.py", line 222, in add_metric
ERROR self.output.add_metric(name, value, units, lower_is_better, classifiers)
ERROR File "/work/workload_automation/workload-automation/wa/framework/output.py", line 142, in add_metric
ERROR self.result.add_metric(name, value, units, lower_is_better, classifiers)
ERROR File "/work/workload_automation/workload-automation/wa/framework/output.py", line 390, in add_metric
ERROR metric = Metric(name, value, units, lower_is_better, classifiers)
ERROR File "/work/workload_automation/workload-automation/wa/framework/output.py", line 653, in __init__
ERROR self.value = numeric(value)
ERROR File "/work/workload_automation/devlib/devlib/utils/types.py", line 88, in numeric
ERROR raise ValueError('Not numeric: {}'.format(value))
ERROR
ERROR ValueError(Not numeric: Performance counter statistics)
With the above options, the csv that simpleperf produces looks like
this:
Performance counter statistics,
123456789,raw-l1-dtlb,,(60%),
42424242,raw-l1-itlb,,(60%),
Total test time,1.001079,seconds,
Performance counter statistics,
123456789,raw-l1-dtlb,,(60%),
42424242,raw-l1-itlb,,(60%),
Total test time,2.001178,seconds,
Performance counter statistics,
[...]
That is, with "--interval-only-values", the "Performance counter
statistics," header is repeated every interval. WA current expects
it only in the first line. Modify the condition so that it is ignored
every time we find it in the file and not just the first time.
Workload Automation
+++++++++++++++++++
Workload Automation (WA) is a framework for executing workloads and collecting
measurements on Android and Linux devices. WA includes automation for nearly 40
workloads and supports some common instrumentation (ftrace, hwmon) along with a
number of output formats.
WA is designed primarily as a developer tool/framework to facilitate data driven
development by providing a method of collecting measurements from a device in a
repeatable way.
WA is highly extensible. Most of the concrete functionality is implemented via
plug-ins, and it is easy to write new plug-ins to support new device types,
workloads, instruments or output processing.
Requirements
============
- Python 3.5+
- Linux (should work on other Unixes, but untested)
- Latest Android SDK (ANDROID_HOME must be set) for Android devices, or
- SSH for Linux devices
Installation
============
To install::
git clone git@github.com:ARM-software/workload-automation.git workload-automation
sudo -H python setup [install|develop]
Note: A `requirements.txt` is included however this is designed to be used as a
reference for known working versions rather than as part of a standard
installation.
Please refer to the `installation section <http://workload-automation.readthedocs.io/en/latest/user_information.html#install>`_
in the documentation for more details.
Basic Usage
===========
Please see the `Quickstart <http://workload-automation.readthedocs.io/en/latest/user_information.html#user-guide>`_
section of the documentation.
Documentation
=============
You can view pre-built HTML documentation `here <http://workload-automation.readthedocs.io/en/latest/>`_.
Documentation in reStructuredText format may be found under ``doc/source``. To
compile it into cross-linked HTML, make sure you have `Sphinx
<http://sphinx-doc.org/install.html>`_ installed, and then ::
cd doc
make html
License
=======
Workload Automation is distributed under `Apache v2.0 License
<http://www.apache.org/licenses/LICENSE-2.0>`_. Workload automation includes
binaries distributed under different licenses (see LICENSE files in specific
directories).
Feedback, Contributions and Support
===================================
- Please use the GitHub Issue Tracker associated with this repository for
feedback.
- ARM licensees may contact ARM directly via their partner managers.
- We welcome code contributions via GitHub Pull requests. Please see
"Contributing Code" section of the documentation for details.