1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 02:41:11 +01:00
Commit Graph

84 Commits

Author SHA1 Message Date
Marc Bonnici
3fd40e5f19 utils/types: Fix correct arguments for join
Ensure that the join method for a ``str`` is passed a single list
instead of 2 arguments.
2018-07-04 16:33:08 +01:00
Marc Bonnici
cbb8fdab28 utils/diff: Fix missing import for logging 2018-07-04 16:33:08 +01:00
Marc Bonnici
f4838b4466 utils/android: Fix typo in variable assignment 2018-07-04 16:33:08 +01:00
Marc Bonnici
9d34fc0a4d utils/serializer: Fix exception handling in Python3
Allow for the fact that exceptions do not have a 'message' attribute in
Python3.
2018-07-04 11:54:17 +01:00
Sergei Trofimov
06d351f054 utils/misc: fix write_table column widths
Consider headers when calculating column widths. This will ensure proper
alignment in cases where the column header is wider than any of the
column values.
2018-06-28 10:32:36 +01:00
Sergei Trofimov
687d09cd53 utils/types: fix typos 2018-06-25 14:14:00 +01:00
Waleed El-Geresy
578d2d3a16 utils/serializer: Handle empty file extensions
The ValueError does not given enough information in the case of an empty file
extension. Add a special case to handle files with no extension and
prompt the user.
2018-06-18 20:46:56 +01:00
Sergei Trofimov
6ee40c2170 utils/types: implement __ne__ for level
This should have been handled by the @total_ordering decorator, but
isn't due to

        https://bugs.python.org/issue25732

(briefly, total_ordering is back-ported from Python 3, where the base
object provides the default implementation of __ne__ based on __eq__, so
total_ordering did not override it; this, however does not happen in
Python 2).

Also update unit tests to catch this edge case.
2018-06-14 15:04:56 +01:00
Sergei Trofimov
64f9cf79e4 utils/diff: fix diff_interrupt_files on Python 2
During Python 3 migration, the izip call inside diff_interrupt_files has
been replaced with a zip call (zip returning an iterator in Python 3).
Import zip from builtins to ensure that it also produces an iterator in
Python 2.

Also fix the associated unit test by importing the correct function.
2018-06-14 15:04:56 +01:00
Sergei Trofimov
c4838b4021 utils/serializer: add basestring to POD_TYPES
Was replaced with str during Python 3 porting.
2018-06-12 13:30:39 +01:00
Sergei Trofimov
657a10c09d utils/serializer: fix level deserialization
Fix a regression introduced with Python 3 port -- JSON deserializer
should check for basestring rather than str when deciding whether to try
to decode a custom type.
2018-06-12 13:27:19 +01:00
Sergei Trofimov
b3de85455a Add support for Python 3
Add support for running under Python 3, while maintaining compatibility
with Python 2.

See http://python-future.org/compatible_idioms.html for more details
behind these changes.
2018-06-07 14:48:40 +01:00
Sergei Trofimov
a865d8f30c utils/serializer: add cpu_mask to POD types 2018-05-25 15:27:14 +01:00
Sergei Trofimov
4cc5aec39a utils/serializer: make is_pod() recursive
If the object being passed into is_pod() is iterable (and not a
string), recursively check that its elements are also PODs.
2018-05-25 10:21:06 +01:00
Marc Bonnici
c338962c48 utils/doc: Add missing newline when generating aliases docs
Ensure when extracting aliases documentation it is followed by 2
newlines so that it formats correctly.
2018-05-24 16:57:47 +01:00
Marc Bonnici
6cb7031713 wa/misc: Fix format_duration() to handle missing time
If a run is interrupted this can cause the time to be `None` causing an
error, if this is the case then set time to 0 and therefore return `N/A`.
2018-05-24 11:49:10 +01:00
Marc Bonnici
614a590b35 utils/counters: Remove duplicated counter
The implementation of the `counter` was duplicated in both
`wa.utils.counters` and `wa.utils.types.counter` the latter of which is
currently unsed, therefore port missing functionality to the former and
remove.
2018-05-24 11:49:10 +01:00
Sergei Trofimov
2a2eafa282 utils/doc: include plugin aliases in the docs
If a plugin defines aliases, include the aliases and the parameter
values they correspond to inside the docs.
2018-05-23 14:51:54 +01:00
Marc Bonnici
3efff81a5c utils/exec_control: Fix issue with once_per_instance decorator
Previously the `once_per_instance` used the output of `__repr__` to
identify the class of where the decorated method was called from. For
classes that override this method to produce the same output for
different instances of the same class this caused different instances to
be mistakenly treated as one. Now use the hash of the containing class
instead of the string representation and update the tests to catch this
error.
2018-05-22 17:18:40 +01:00
Marc Bonnici
670fe6fb83 wa/types: Add cpu_mask type
Add a cpu_mask type to allow for users to use a consistent way of representing a
cpus mask with properties to provide conversions between the various required
forms. The mask can be specified directly as a mask, as a list of cpus indexes
or a sysfs-style string.
Also add support for the type to be serialized/deserialzed for the json
and yaml serializers.
2018-05-14 17:07:49 +01:00
Sergei Trofimov
a2b0705ff0 processors: add cpustates
Add cpustates output processor. This is more-or-less a port of the
cpustates processor from WA, however there are some differences:

- Idle states are now tracked individually per-CPU. This will fix
  processing traces from targets that have different number of idle
  states on different clusters.
- Simplify the parameter list for report_power_stats:
    - Replace paths to individual report files with a path to a single
      directory. A subdirectory will be created under it which will
      contain all the reports.
    - Replace the individual bits bits of information about CPUs (core
      names, idle states, etc) with a list of CpuInfo objects.
- Clean up and simplify the code a bit:
    - Make all reports mandatory -- the marginal cost of generating
      an additional report is minimal compared to tracking power states
      in the first place.
    - Standardize the interface for Reporters and Reports.
- Rename some of the reports to something a bit more meaningful.
- The stand-alone command line interface is not ported for now, as it
  is now possible to run this offline on existing results using
  "wa process".
2018-05-14 17:05:06 +01:00
Sergei Trofimov
e7e272cd03 utils/trace_cmd: add trace_has_marker()
Add a function to check whether a trace file contains start/stop markers
(actually only check for start marker -- stop marker is assumed).
2018-05-14 17:05:06 +01:00
Sergei Trofimov
795b3485ce utils/trace_cmd: move params to __init__()
Move the check_for_markers and events parameters from parser() to
__init__(). These parameters control the behavior of the parser, and do
not relate to a particular trace file, so it makes more sense to have
them there.
2018-05-14 17:05:06 +01:00
Sergei Trofimov
1ffbce68c4 fw/execution: handle error on run initialize
- Use indentcontext inside initialize_run to make sure log level
  is dedented on error.
- Ensure Executor postamble always runs, event if runner errors.
- Fix format_duration() to handle sub-second timedeltas.
2018-05-10 11:49:30 +01:00
sergei Trofimov
ecc68ee367 utils/log: add indentcontext
Add indentcontext context manager to save having to add try/finally
clauses around indenting code just to ensure dedenting.
2018-05-09 15:06:31 +01:00
sergei Trofimov
c1a0f842d0 utils/log: maintain indent for buffered records
Ensure buffered records are indented properly by saving the indent level
at the time the record was generated as part of the record, and
preferring that over the current indent level inside LineFormatter.
2018-05-09 15:06:31 +01:00
sergei Trofimov
1e750c659b utils/log: add early record buffering
Buffer early log records until a log file becomes available, flush
records to said file afterwards. This ensures that early log output,
which, among other things, contains plugin discovery logging, is not
lost. Buffer capacity is configurable via an environment variable.
2018-05-09 15:06:31 +01:00
sergei Trofimov
aacba47f9a utils/types: toggle_set: add "~~" semantics
Add support for "~~" special value that indicates that, when merging
into another set, all values from that set should in fact be dropped.
Apart from the unique merge semantics, "~~" just like any other "~"
value.
2018-05-08 10:50:48 +01:00
sergei Trofimov
9093477f1b utils/types: fix toggle_set merge methods
- The merge_with and merge_into implementation was reversed. Fix this, so that
  it is implemented correctly, i.e. that
  	a.merge_with(b) === merge(a, b)
	a.merge_into(b) === merge(b, a)
- Ensure that in case of merge_into(), a toggle_set is always returned.
2018-05-08 10:50:48 +01:00
Marc Bonnici
a0dfac78c3 utils/doc: Add function to generate a line break 2018-04-30 09:14:22 +01:00
Marc Bonnici
98bed3822a utils/docs: Update format of parameter type rst output 2018-04-24 09:59:57 +01:00
Marc Bonnici
3ddb1272aa utils/doc: Add support for dicts to format literal
Now supports cleaner outputing of python dicts
2018-04-24 09:59:57 +01:00
Sergei Trofimov
67ea7c8ee1 utils/types: toggle_set: TypeError on string
Strings are iterable, so can be used to instantiate sets (resulting in a
set of chars). This is never what we want for toggle_set's though, and
may result in difficult-to-interpret errors when parsing configuration,
so raise a TypeError if attempting to create a toggle_set with a string.
2018-03-22 12:28:48 +00:00
Marc Bonnici
2371364ce4 utils/doc: Adds support for showing aliases when formatting parameters
Now displays all available local and global aliases when generating the
rst for a parameter.
2018-03-21 13:27:31 +00:00
Marc Bonnici
f5ed4e317a utils/misc: Fix missing import 2018-03-20 08:24:58 +00:00
Sergei Trofimov
fda418093d fw/execution: change how error events get added
Rather than relying on a custom Logger with a context to add events when
they are logged, have the runner register hooks for corresponding
signals that do that.

The issue with the previous approach is that the event ended up
being added too late -- after the current job was already cleared, so
all events were added at run level.

With the new approach, the event will be added when it is first logged,
while there is still a current job being set. There will be no
duplication for Exceptions being re-raised and handled at different
levels because log_error() will ensure that each Exception is logged
only once.
2018-03-08 11:18:12 +00:00
Sergei Trofimov
04ab336afc fw: fix error logging
- Keep track of logged exceptions inside log_error itself.
- signal: log the exception, if there is one  in the finally clause of
  the  signal wrapper; this will ensure that the error will be logged
  closer to the command that originated.
- entrypoint: use log.log_error for top-level error logging, rather than
  the  entrypoint logger directly; this will ensure that errors are not
  repeated unnecessarily.
- Log CTRL-C message at zeroth indent level to make it easier to see in
  the  non-verbose output where it occurred.
2018-03-08 11:18:12 +00:00
Sergei Trofimov
e3b3799737 utils/misc: remove unused to_identifier
This has been moved to devlib, but, it appears, never removed from WA.
2018-02-16 13:56:03 +00:00
Marc Bonnici
d9a7e1c475 utils/types: Add add method to toggle_set
Allows adding of elements that will respect the state of an existing
element in the toggle set.
2018-02-01 14:26:08 +00:00
Marc Bonnici
e5ed64d8d5 trace_cmd: Rename trace-cmd to trace_cmd
Both the instrument and utility functions were called `trace-cmd.py`.
`"-"`'s are not valid python identifiers therefore when trying to import
the `trace-cmd` module this causes an error.
2018-01-25 16:28:51 +00:00
Marc Bonnici
f26d819aad utils/exec_control: Fix once decorator implementation
Ensures that the once decorator does not affect classes in a parallel
inheritance hierarchy.
2018-01-15 15:42:26 +00:00
Marc Bonnici
950427f478 utils/revent: Refactor ReventRecording to avoid duplicate target calls
Changed location of revent binary on target to be a class attribute so that
it is shared across all instances. Also changed discovery method to a
static method which can be memoized to prevent the target being queried
unnecessarily.
2018-01-08 10:36:02 +00:00
Marc Bonnici
9832db9fa5 utils/doc: Rename get_rst_from_extension for consistency 2017-12-14 09:50:26 +00:00
Marc Bonnici
adb14442c8 utils/doc: Change get_params_rst to be more flexible
Instead of taking an extension as a parameter and assuming there it has
an attribute 'parameters' instead take a list of parameters directly
which allows the method to be used with parameter lists with different
names.
2017-12-14 09:50:26 +00:00
Brendan Jackman
aa2abd16a3 Run autoflake to remove unused imports & variables 2017-12-07 15:37:49 +00:00
Marc Bonnici
bc5b43df30 Utils/Android: Fix missing space in check
Logcat entries are split on a ':' followed by a space so ensure this
is present in the line instead of just a ':'. Some devices have entires
without the trailing space causing an error.
2017-11-24 16:38:01 +00:00
Sergei Trofimov
39e63f1358 utils/android: add a parser for Logcat
Add a simple streaming parser for Logcat logs. At the moment, only the
recent logcat format is supported.
2017-11-24 14:30:09 +00:00
Sergei Trofimov
2ff1a87e53 utils/types: fix toggle_set docstring
The docstring claimed that the toggle_set was "A list that...".
2017-11-03 17:33:32 +00:00
Brendan Jackman
4b7d43de9e utils/log: Convert exception to string before passing to Logger
Now that we add an event when logging errors, we need to avoid passing an
exception object directly to the logger. Otherwise we end up adding the
unserializable Exception object to the events list, which causes an error when
trying to write the output.
2017-10-23 17:25:15 +01:00
Brendan Jackman
b11bd6cf12 utils/log: Whitespace 2017-10-23 17:25:02 +01:00