1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

115 Commits

Author SHA1 Message Date
Marc Bonnici
76e6f14212 utils/serializer: pylint fixes 2019-05-15 19:48:39 +01:00
Marc Bonnici
ce59318e66 utils/serializer: Fix using incorrect loader and imports
- Ensure that the new loader is used when opening files to ensure that our
custom constructors are used.
- Fix missing imports
2019-05-15 19:48:39 +01:00
Sergei Trofimov
5652057adb utils/serializer: fix support for YAML anchors.
Change the way maps get processed by YAML constructor to support YAML
features, such as anchors, while still maintaining dict ordering.
2019-05-15 09:59:14 +01:00
Sergei Trofimov
e9f5577237 utils/serializer: fix error reporting for YAML
When attempting to access the message of a exception check not only that
e.args is populated, but also that e.args[0] actually contains
something, before defaulting to str(e).
2019-05-15 09:57:52 +01:00
Marc Bonnici
769aae3047 utils/serializer: Explicitly state yaml loader
In newer versions of PyYAML we need to manually specify the `Loader` to
be used as per https://msg.pyyaml.org/load.
`FullLoader` is now the default loader which attempts to avoid arbitrary
code execution, however if we are running an older version where this is
not available default back to the original Loader.
2019-03-04 15:50:13 +00:00
Marc Bonnici
a1ba3c6f69 utils/misc: Update load structure to use WA's yaml wrapper 2019-03-04 15:50:13 +00:00
Sergei Trofimov
ed95755af5 fw/output: better classifiers format for metrics
Use a dict-like string representation for classifiers, rather than the
default OrderedDict one, which is a lot more verbose and difficult to
read.
2019-01-10 13:03:29 +00:00
Marc Bonnici
4f67cda89f utils/types: When creating an enum also try to deserialize from POD
Allows for recreating an Enum from a full string representation of the Enum
rather than just the name of the Enum.
2018-12-07 09:55:17 +00:00
Marc Bonnici
0426a966da utils/postgres: Relocate functions to retrieve schema information
Move the functions to retrieve schema information to general utilities to
be used in other classes.
2018-12-07 09:55:17 +00:00
Marc Bonnici
64f7c2431e utils/postgres: Rename postgres_covert to house more general methods
Rename the postgres_covert file to allow for place more general postgres
utility functions.
2018-12-07 09:55:17 +00:00
Marc Bonnici
86287831b3 utilts/serializer: Update exception method to support Python3 2018-12-07 09:55:17 +00:00
Marc Bonnici
88a4677434 utils/serializer: Fix attempting to deserialize a single value. 2018-12-07 08:46:12 +00:00
Marc Bonnici
4ff7e4aab0 utils/serializer: Add Podable Mix-in class
Add a new mix-in class for classes that are serialized to PODs, the aim
of this class is to provide a way to ensure that both the original data
version and the current serialization version are known. When attempting
to de-serialize a POD the serialization version will be compared to the
latest version in WA if not matching will call the appropriate method to
upgrade the pod to a known structure state populating any missing fields
with a sensible default or converting the existing data to the new
format.
2018-12-07 08:46:12 +00:00
Marc Bonnici
bb282eb19c wa: Remove reference to devlibs escaping methods
As part of https://github.com/ARM-software/devlib/pull/339 the escaping
method are being removed in favour of using `quote` from `pipes` so
also make reflecting changes here.
2018-11-21 15:07:25 +00:00
Marc Bonnici
f2f210c37f utils/postgres_convert: PEP8 fix
Remove unused local variable.
2018-10-24 10:34:44 +01:00
Marc Bonnici
375a36c155 utils/log: Ensure to convert all arguments to strings
Ensure that all arguments provided for an Exception are converted to
stings before attempting to join them for debug information.
2018-10-09 15:26:53 +01:00
Sergei Trofimov
98727bce30 utils/revent: recording parser fixes
- change magic string literal to a b'' string so that the comparison
  works in python 3
- expand timestamp tuples (struct.unpack always returns a tuple) before
  attempting to cast to float.
2018-10-08 17:46:35 +01:00
Waleed El-Geresy
6d654157b2 Add Postgres Output Processor
The Output processor which is used to upload the results found in the
wa_output folder to a Postgres database, whose schema is defined by the
WA Create Database command.
2018-09-12 10:13:34 +01:00
Sergei Trofimov
6d9ec3138c In lint we trust
Quieten pylint with regard to import order.
2018-07-18 11:20:48 +01:00
Sergei Trofimov
96dd100b70 utils/toggle_set: fix merge behavior
- Change how "source" and "dest" are handled inside merge() to be more
  sane and less confusing, ensuring that disabling toggles are merged
  correctly.
- Do not drop disabling values during merge, to ensure that merging
  is a transitive operation.
- Add unit tests for the above fixes.
2018-07-11 10:48:00 +01:00
Marc Bonnici
5714c8e6a1 wa: Additional pylint fixes 2018-07-10 12:56:51 +01:00
Marc Bonnici
791d9496a7 wa: Pylint Fixes for Travis
Pylint has trouble using imports from the distutils module in
virtualenvs so we need to explicitly ignore these imports.
2018-07-10 12:56:51 +01:00
Marc Bonnici
e8b0d42758 wa: PEP8 Fixes 2018-07-10 12:56:51 +01:00
Sergei Trofimov
c410d2e1a1 I lint, therefore I am
Implement fixes for the most recent pylint version.
2018-07-09 15:59:40 +01:00
Marc Bonnici
925bc8b719 wa: pep8 fixes 2018-07-06 14:39:41 +01:00
Marc Bonnici
185bff9029 wa/utils: Fix pylint errors 2018-07-06 14:39:41 +01:00
Marc Bonnici
1550e9a788 wa/utils: Remove unrequired imports 2018-07-06 14:39:41 +01:00
Marc Bonnici
4d1c03cc5c utils/types: Rename cls argument 2018-07-06 14:39:41 +01:00
Marc Bonnici
cbc7b2647f utils/types: Ignore import errors 2018-07-06 14:39:41 +01:00
Marc Bonnici
1a08b2a6c7 utils: pep8 fixes 2018-07-06 14:39:41 +01:00
Sergei Trofimov
8878cc20d4 Update copyright headers.
- Add copyright headers to files that did not already have them
- Update the year to the last year the file was modified in files with
  existing headers.
2018-07-04 16:33:31 +01:00
Marc Bonnici
3fd40e5f19 utils/types: Fix correct arguments for join
Ensure that the join method for a ``str`` is passed a single list
instead of 2 arguments.
2018-07-04 16:33:08 +01:00
Marc Bonnici
cbb8fdab28 utils/diff: Fix missing import for logging 2018-07-04 16:33:08 +01:00
Marc Bonnici
f4838b4466 utils/android: Fix typo in variable assignment 2018-07-04 16:33:08 +01:00
Marc Bonnici
9d34fc0a4d utils/serializer: Fix exception handling in Python3
Allow for the fact that exceptions do not have a 'message' attribute in
Python3.
2018-07-04 11:54:17 +01:00
Sergei Trofimov
06d351f054 utils/misc: fix write_table column widths
Consider headers when calculating column widths. This will ensure proper
alignment in cases where the column header is wider than any of the
column values.
2018-06-28 10:32:36 +01:00
Sergei Trofimov
687d09cd53 utils/types: fix typos 2018-06-25 14:14:00 +01:00
Waleed El-Geresy
578d2d3a16 utils/serializer: Handle empty file extensions
The ValueError does not given enough information in the case of an empty file
extension. Add a special case to handle files with no extension and
prompt the user.
2018-06-18 20:46:56 +01:00
Sergei Trofimov
6ee40c2170 utils/types: implement __ne__ for level
This should have been handled by the @total_ordering decorator, but
isn't due to

        https://bugs.python.org/issue25732

(briefly, total_ordering is back-ported from Python 3, where the base
object provides the default implementation of __ne__ based on __eq__, so
total_ordering did not override it; this, however does not happen in
Python 2).

Also update unit tests to catch this edge case.
2018-06-14 15:04:56 +01:00
Sergei Trofimov
64f9cf79e4 utils/diff: fix diff_interrupt_files on Python 2
During Python 3 migration, the izip call inside diff_interrupt_files has
been replaced with a zip call (zip returning an iterator in Python 3).
Import zip from builtins to ensure that it also produces an iterator in
Python 2.

Also fix the associated unit test by importing the correct function.
2018-06-14 15:04:56 +01:00
Sergei Trofimov
c4838b4021 utils/serializer: add basestring to POD_TYPES
Was replaced with str during Python 3 porting.
2018-06-12 13:30:39 +01:00
Sergei Trofimov
657a10c09d utils/serializer: fix level deserialization
Fix a regression introduced with Python 3 port -- JSON deserializer
should check for basestring rather than str when deciding whether to try
to decode a custom type.
2018-06-12 13:27:19 +01:00
Sergei Trofimov
b3de85455a Add support for Python 3
Add support for running under Python 3, while maintaining compatibility
with Python 2.

See http://python-future.org/compatible_idioms.html for more details
behind these changes.
2018-06-07 14:48:40 +01:00
Sergei Trofimov
a865d8f30c utils/serializer: add cpu_mask to POD types 2018-05-25 15:27:14 +01:00
Sergei Trofimov
4cc5aec39a utils/serializer: make is_pod() recursive
If the object being passed into is_pod() is iterable (and not a
string), recursively check that its elements are also PODs.
2018-05-25 10:21:06 +01:00
Marc Bonnici
c338962c48 utils/doc: Add missing newline when generating aliases docs
Ensure when extracting aliases documentation it is followed by 2
newlines so that it formats correctly.
2018-05-24 16:57:47 +01:00
Marc Bonnici
6cb7031713 wa/misc: Fix format_duration() to handle missing time
If a run is interrupted this can cause the time to be `None` causing an
error, if this is the case then set time to 0 and therefore return `N/A`.
2018-05-24 11:49:10 +01:00
Marc Bonnici
614a590b35 utils/counters: Remove duplicated counter
The implementation of the `counter` was duplicated in both
`wa.utils.counters` and `wa.utils.types.counter` the latter of which is
currently unsed, therefore port missing functionality to the former and
remove.
2018-05-24 11:49:10 +01:00
Sergei Trofimov
2a2eafa282 utils/doc: include plugin aliases in the docs
If a plugin defines aliases, include the aliases and the parameter
values they correspond to inside the docs.
2018-05-23 14:51:54 +01:00
Marc Bonnici
3efff81a5c utils/exec_control: Fix issue with once_per_instance decorator
Previously the `once_per_instance` used the output of `__repr__` to
identify the class of where the decorated method was called from. For
classes that override this method to produce the same output for
different instances of the same class this caused different instances to
be mistakenly treated as one. Now use the hash of the containing class
instead of the string representation and update the tests to catch this
error.
2018-05-22 17:18:40 +01:00