1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

504 Commits

Author SHA1 Message Date
Marc Bonnici
cc0cfaafe3 fw/workload: Add attribute to control if package data should be cleared.
Allow specifying that the package data should not be cleared
before starting the workload.
2018-11-15 07:54:43 +00:00
Sergei Trofimov
ce94638436 fw/target: record page size as part of TargetInfo
Record target.page_size_kb as part of target info.
2018-11-02 12:11:00 +00:00
Sergei Trofimov
d1fba957b3 fw/target: add versioning to TargetInfo
Add format_version class attribute to TargetInfo to track format
changes. This is checked when deserializing from POD to catch format
changes between cached and obtained TargetInfo's.
2018-11-02 12:11:00 +00:00
Marc Bonnici
7632ee8288 fw/uiauto: Add method to baseclass to dismiss android version popup
In Android Q a popup will be displayed warning if the application has
not been designed for the latest version of android. This has currently been
dealt with on a per workload basis however this is a common popup so
add a method to dismiss the popup if present to the base class.
2018-10-24 10:34:44 +01:00
Marc Bonnici
891ef60f4d configuration: Add support for section groups
Now allows for specifying a `group` value for each section which will
cross product the sections within that group with the sections in each
other group. Additionally classifiers will automatically be added to
each job spec with the relevant group information.
2018-09-24 10:17:26 +01:00
Marc Bonnici
57aa5ca588 fw/version: Add developement tag to version number 2018-09-21 15:06:30 +01:00
Marc Bonnici
a7d0b6fdbd target/descriptor: Do not convert the module list to strings
Change the type of the `modules` to `list` so that additional
configuration can be supplied to individual modules as a dict of values.
2018-09-21 15:06:30 +01:00
Marc Bonnici
7d01258bce fw/target/manager: Do no finalize target if not instantiated
In the case of an error occurring during target initialization do not
try and check for disconnecting upon finalizing.
2018-09-21 15:06:30 +01:00
Marc Bonnici
8cd79f2ac4 fw/instrument: Fix compatibility with Python2
The Python2 inspect module does not contain the `getfullargspec` method so call
the appropriate method depending on Python version.
2018-09-05 15:44:48 +01:00
Waleed El-Geresy
718f2c1c90 Expose context in OP initialize and finalize
Expose the context to the initialize and finalize functions for Output
Processors. This was found to be necessary for the upcoming PostgreSQL
Output Processor.
2018-09-05 14:40:42 +01:00
Marc Bonnici
6366a2c264 framework/version: Specify default encoding when parsing commit id 2018-08-22 14:41:12 +01:00
Marc Bonnici
d858435c3d utils/version: Fix check to only decode bytes
When using Python3 the returned value of the commit is a byte string and
therefore needs to be decoded.
2018-07-27 10:11:32 +01:00
Marc Bonnici
fc226fbb6e fw/execution: Ensure that identifiers are used when retrieving plugins.
Make sure that when retrieving plugin information from the plugin
cache the name is converted to an identifier first.
2018-07-24 11:34:19 +01:00
Sergei Trofimov
e4a856ad03 fw/config: preserve included config files
Save included config files, along with the explicitly-specified config
that included them, under run output's __meta/raw_config/.
2018-07-23 16:47:10 +01:00
Sergei Trofimov
7d833ec112 fw/config: add includes
Add the ability to include other YAML files inside agendas and config
files using "include#:" entries.
2018-07-23 16:47:10 +01:00
Marc Bonnici
b729f7c9e4 fw/parsers: Ensure plug-in names are converted to an identifier
Ensure that a plug-ins config entry is converted to an identifier before being
stored in the PluginCache so that the relevant configuration can be
retrieved appropriately. For example this allows for both 'trace-cmd' and
'trace_cmd' to be used as config entries to provide configuration for the
'trace-cmd' plugin.
2018-07-19 17:15:26 +01:00
Sergei Trofimov
e8f545861d fw: cache target info
Cache target info after pulling it from the device. Attempt to retrieve
from cache before querying target.
2018-07-13 15:53:01 +01:00
Sergei Trofimov
770d2b2f0e fw: add cache subdir under $WA_USER_DIRECTORY
Add a sub-directory for caching stuff.
2018-07-13 15:53:01 +01:00
Sergei Trofimov
dacb350992 fw/target: add system_id to TargetInfo
Add target's system_id to TargetInfo. This ID is intended to be unique
of the combination of hardware and software on the target.
2018-07-13 13:28:50 +01:00
Sergei Trofimov
ce93823967 fw/execution: write config after installing augs
Add Context.write_config() to write the combined config into run output
__meta. Use it after instruments and result processors get installed to
make sure their configuration gets serialized in the output.
2018-07-11 13:28:04 +01:00
Sergei Trofimov
7755363efd fw/config: add get_config() to ConfigManager
Add a method to allow obtaining the combined config after config
manager has been finalized.
2018-07-11 13:28:04 +01:00
Sergei Trofimov
bcea1bd0af fw/config: add resource getter to run config
Track resource getter configuration as part of the run config.
2018-07-11 13:28:04 +01:00
Sergei Trofimov
a062a39f78 fw/config: add installed aug configs to run config
Track configuration used for installed augmentations inside RunConfig.
2018-07-11 13:28:04 +01:00
Sergei Trofimov
b1a01f777f fw/execution: rename things for clarity
- Rename "instrument_name" to "instrument" inside do_execute(), as
  ConfigManger.get_instrument() returns a list of Instrument objects,
  not names.
- To avoid name clash, rename the imported instrument module to
  "instrumentation".
2018-07-11 13:28:04 +01:00
Marc Bonnici
e485b9ed39 utils/version: do not decode bytes
Check that the resulting output inside get_commit() is a str before
attempting to decode it when running on Python 3.
2018-07-11 10:39:38 +01:00
Marc Bonnici
5714c8e6a1 wa: Additional pylint fixes 2018-07-10 12:56:51 +01:00
Sergei Trofimov
fd1dd789bf fw/output: update internal state on write_config()
Update the internal _combined_config object with the one that
has been written to ensure that the serialized and run time states are
the same.
2018-07-09 16:00:07 +01:00
Sergei Trofimov
c410d2e1a1 I lint, therefore I am
Implement fixes for the most recent pylint version.
2018-07-09 15:59:40 +01:00
Marc Bonnici
925bc8b719 wa: pep8 fixes 2018-07-06 14:39:41 +01:00
Sergei Trofimov
03eafe6b33 framework: pep8 fixes
Fix issues reported by flake8.
2018-07-06 14:39:41 +01:00
Sergei Trofimov
f74b7ae78c framework: fix pylint issues
Fix/disable checks for issues reported by pylint under wa/framework.
2018-07-06 14:39:41 +01:00
Sergei Trofimov
8878cc20d4 Update copyright headers.
- Add copyright headers to files that did not already have them
- Update the year to the last year the file was modified in files with
  existing headers.
2018-07-04 16:33:31 +01:00
Marc Bonnici
5231bbcf71 fw/exception: Add 'message' property to SerializerSyntaxError
Allow for the fact that Exceptions do not have a message attribute in
Python3 so mimic the functionality.
2018-07-04 11:54:17 +01:00
Marc Bonnici
1648a58c11 framework/workload: Add check for platform
Remove unused ``platform`` attributes and add ``supported_platforms``
along with check so that workloads can fail early when attempting to run
on an incorrect platform.
2018-07-02 12:11:00 +01:00
Sergei Trofimov
e55f48089b fw/output: expose metadata
Expose result.metadata as a property of the output.
2018-06-29 17:25:29 +01:00
Marc Bonnici
98de37807d fw/version: Change format of version to conform with PEP440
When installing from source WA attempts to include the commit ID in the
version of the installed pacakge however this caused issues with package
managers like pip. PEP440 specifies that local identifiers must be in the
form `<public version identifier>[+<localversion label>]` so update the
version to conform.

https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
2018-06-27 11:41:24 +01:00
Marc Bonnici
22fcc42903 fw/core: Correct parameter description 2018-06-25 18:19:30 +01:00
Marc Bonnici
c255576fd7 fw/core: Allow for a user_directory to be specified using ~
Previously if a path for the `user_directory` was specified using `~`
this would not be expanded causing issues due to incorrect path joining.
2018-06-25 13:45:43 +01:00
Sergei Trofimov
3ecc46b662 fw/exec: ensure correct state on early abort
Ensure the serialized state gets written out in case of early abort
(before the run has been fully initialized and job execution has
started), and  that the appropriate final output is generated.

This requires that ExecutionContext is created much earlier (in order
for postamble to be possible), and that KeyboardInteruptException is
caught and handled properly by the Executor.
2018-06-22 18:04:16 +01:00
Sergei Trofimov
d0dd6c1e04 fw/exec: context: add write_job_specs
Add a method to encapsulate the writing of ConfigManager's job specs
into run_output.
2018-06-22 18:04:16 +01:00
Sergei Trofimov
861dff75d4 fw/exec: context: add write_output()
Add a convenience method for invoking write_info(), write_state(), and
write_result() of run_output. This will make it easier to call it from
multiple places.
2018-06-22 18:04:16 +01:00
Sergei Trofimov
1b498fa42d fw/exec: context: make target_info a property
TargetManger.get_target_info() gets invoked twice: first on context
creation, and then a second lime shortly afterwards to populate
RunOutput. Change target_info attribute inside the context to be a
property that redirects to run_output. This will allow the context to be
created earlier, before a target connection  is available.
2018-06-22 18:04:16 +01:00
Sergei Trofimov
8da911e9d5 fw/target: add initialize() to TargetManger
Add explicit initialization to TargetManger in order to separate the
creation of the TargetManger form connection to the target.
2018-06-22 18:04:16 +01:00
Sergei Trofimov
0f2bd28284 fw/output: record wa/devlib version in output
Add wa and devlib versions to RunOutput's metadata on creation. They
already get logged, but this will make them readily accessible to
scripts and output processors.
2018-06-20 10:04:18 +01:00
Sergei Trofimov
9f06950fb1 fw/target: account for platform_target_defaults
When iterating over PLATFORMS tuples, account for the
platform_target_defaults introduced by 7470fb5ee.

Also, correct the tuple description in the comments.
2018-06-19 11:23:12 +01:00
Joe Savage
6cc1ac147c framework/workload: Add rerun infrastructure 2018-06-18 10:51:32 +01:00
Sergei Trofimov
bcbde9a197 fw/entrypoint: log devlib version
Log devlib version alongside WA version.
2018-06-15 08:58:39 +01:00
Waleed El-Geresy
488bd57659 framework/version: Redirect git stderr output
Add an stderr argument to redirect errors ('fatal not a git repository'
is the most common) to PIPE so that errors do not output to STDOUT.
2018-06-14 15:07:18 +01:00
Waleed El-Geresy
71a32dd724 framework/command: Fix subparsers in Python3
Make subcommands required arguments in the ComplexCommand class, which
is used by the 'wa create' command.
2018-06-14 15:06:50 +01:00
Waleed El-Geresy
7470fb5ee9 Add target-params to Platform Tuple
Added support for specifying target parameters in the Platform tuple.
2018-06-14 12:06:33 +01:00
Sergei Trofimov
0ca29bb337 fw/entrypoint: make command mandatory for Python 3
Python 3 has changed the behavior of subparsers so that they are no
longer mandatory by default. As a result, executing just "wa" under
Python 3 results in a random error, rather than a help message.

Fix this by making the subparsers mandatory.
2018-06-13 11:08:18 +01:00
Sergei Trofimov
2cb1f47e11 fw/exec: pass context to run started/completed signals
To be consistent with the rest of exec signals.
2018-06-12 16:07:53 +01:00
Sergei Trofimov
ea97372ece fw/exec: add JOB_QUEUE_EXECUTION signals
Add signals just before and just after commencing the execution of the
job queue.
2018-06-12 16:07:53 +01:00
Sergei Trofimov
5f7fde206d fw/instrument: add hostside decorator
Add a decorator to indicate that a callback runs entirely host-side and
does not rely on a connection to the target. This  means it will be
invoked even if the target was detected to be unresponsive.
2018-06-12 16:07:53 +01:00
Sergei Trofimov
2f99137ad5 fw/target: fix signals reboot on unresponsive
- Add "hard" parameter to TargetManager's reboot().
- Pass in context to TargetManager's reboot() and
  verify_target_responsive() -- this is needed for signal dispatch.
- Call self.reboot() instead of self.target.reboot() when attempting a
  reset for unresponsive targets inside TargetManager, in order to
  ensure that appropriate signals are dispatched.
2018-06-12 16:07:53 +01:00
Sergei Trofimov
e816980471 fw/execution: add missing sender in signal
Add the missing sender for RUN_OUTPUT_PROCESSED signal wrap call.
2018-06-12 16:07:53 +01:00
Sascha Bischoff
b6531feb52 framework/instrument: Fix typo for on_job_failure 2018-06-11 15:59:53 +01:00
Marc Bonnici
1879dd62f1 fw/core: Fix description of 'label' CongfigPoint
Update reference of result processors to output processors.
2018-06-11 08:10:07 +01:00
Sergei Trofimov
b3de85455a Add support for Python 3
Add support for running under Python 3, while maintaining compatibility
with Python 2.

See http://python-future.org/compatible_idioms.html for more details
behind these changes.
2018-06-07 14:48:40 +01:00
Marc Bonnici
c3ddb31d4d fw/entrypoint: Fix help information for '--help' argument
In a previous commit we filtered parameters so that '-h' is never passed
to 'parse_known_commands' to ensure our own custom help message is displayed,
now also filter for '--help' for the same reason.
2018-06-07 11:04:29 +01:00
Marc Bonnici
6d9ae419c2 framework/signal: Remove JOB_FINALIZED signal
Remove the currenlty unused JOB_FINALIZED signal as we already have a
WORKLOAD_FINALIZED signal.
2018-06-06 08:38:54 +01:00
Marc Bonnici
baf570fc62 fw/signal: Rename and implement signal for RUN_OUTPUT_PROCESSED
Rename the OVERALL_RESULTS_PROCESSING signal to be called
RUN_OUTPUT_PROCESSED and provide implementation.
2018-06-06 08:38:54 +01:00
Marc Bonnici
b15b937863 fw/parsers: Ensure that a section is a valid dict
Ensure that all section entries are valid dicts otherwise raise an error
informing the user.
2018-06-05 17:54:34 +01:00
Marc Bonnici
2cd355195f fw/rt_config: Fix validation checks
Remove unnecessary validation checks and fix remaining so they don't fail
under certain combinations of parameters.
2018-06-01 15:33:10 +01:00
Marc Bonnici
9035c654bd fw/rt_config: Allow for auto determine 'max' and 'min' for other freqs
Allow users to specify 'max' and 'min' for the maximum and minimum
frequencies to be set for a cpu as well as the current frequency.
2018-06-01 15:33:10 +01:00
Marc Bonnici
e1518e8c5d fw/rt_config: Refactor and fix method calls to resolve 'max'/'min'
Previously incorrect method calls were used to try and convert 'max' and
'min' into their corresponding frequencies, now call the correct methods
and refactor the conversion into a separate method.
2018-06-01 15:33:10 +01:00
Sergei Trofimov
ee3c01418f fw/execution: fix get_resource with strict=False
If strict=False, resolver.get_resource will return None, rather
than raising NotFoundError. Do not attempt to record the md5 hash in
that case.
2018-06-01 14:01:57 +01:00
Marc Bonnici
b14c3d8868 framework/instrument: Fix lefover references to instrument proiorities
Update instrument proiorities to use signal proiorities instead.
2018-05-30 12:23:15 +01:00
Marc Bonnici
703e32e511 framework/signals: Remove unnecessary job signals
Remove overlap of signals between jobs and workload.
2018-05-30 12:23:15 +01:00
Sergei Trofimov
864ded60ae fw/instrument: use signal priorities
Use signal priorities to define priority decorators, rather than
defining own priority enum.
2018-05-30 12:02:28 +01:00
Sergei Trofimov
d79e8324f0 fw/signal: use enum to define priorities
Use an enum rather than a random class to define the priority levels
for the signals.
2018-05-30 12:02:28 +01:00
Sergei Trofimov
42bae98d50 fw/instrument: remove commented code
Remove the commented out mappings from WA2.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
673b61342d fw/instrument: add mappings for job/run signals
Add missing mappings for job and run signals.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
2cad2dbb10 fw/execution: implement abnormal job status signals
Implement signals for failed/aborted jobs.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
e90b12e984 fw/instrument: add job output mappings
Add method name mappings for JOB_OUTPUT_PROCESSED signals.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
b593b7ed59 fw/execution: wrap processors' process_job_output.
Modify execution so that output processors' process_job_output() (but
not export_job_output()) is wrapped by the JOB_OUTPUT_PROCESSED signals.

This makes sense conceptually, and is more useful in practice, as there
are already WORKLOAD_RESULT_EXTRACTION and WORKLOAD_OUTPUT_UPDATE
signals sent by the job, if it's desirable to run before output
processors, but after the job results have been collected.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
43d047eeb5 fw: implement reboot signals
- Wire up reboot signals via the TargetManager
- Add instrument mappings for the same.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
3a6fcd688e fw/signal: rename "boot" to "reboot"
Rename "boot" signals to "reboot", as it would more accurately reflect
how they are used.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
9f36d543e1 fw/instrument: Add log signal maps
Add method name mappings for ERROR_LOGGED and WARNING_LOGGED signals.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
573c6abcb5 fw/signal: add job signals
Add matched signals for before-after job execution. These are the
equivalent of "spec" signals in WA2.
2018-05-29 11:56:39 +01:00
Sergei Trofimov
2ff06af632 fw/execution: record resource hashes in metadata
As part of resolving a resource, record its MD5 hash in the output
metadata. In order to enable this, resource resolution is now done via
the context, rather than directly via the ResourceResolver (in order not
to introduce a context dependency in the Gui object, context now
emulates the resolver interface).
2018-05-25 10:21:06 +01:00
Sergei Trofimov
79141582e9 fw/workload: record app version in metadata
Add the package version for an ApkWorkload to the metadata for that job.
2018-05-25 10:21:06 +01:00
Sergei Trofimov
9257a787f9 fw/output: add metadata attribute to Result
Metadata is a key-value mapping for arbitrary data, similar to
classifiers. Unlike classifiers, metadata does not directly relate to
the results of the execution, but to the execution itself, and typically
would not be processed by Output Processors in the same way as
classifiers. Metadata can also be a lot more free-form in it's value
structure; while classifier values are simple scalars, metadata values
can be arbitrary POD structures.
2018-05-25 10:21:06 +01:00
Marc Bonnici
86ba51be72 framework/execution: Implement each_spec reboot policy
Implement the `each_spec` reboot policy so that the device will be
rebooted if the spec has changed.
2018-05-25 10:02:17 +01:00
Marc Bonnici
f13e3adf99 framework/execution: Fix previous/next job
The next and previous job attribute implementations of context were
swapped.
2018-05-25 10:02:17 +01:00
Marc Bonnici
d6e9e503fa framework/execution: Allow for retrying the intial reboot
If the device is to perform an initial reboot, allow up to `max_retries`
from the run configuration for the reboot to succeed and add logging to
inform the user.
2018-05-25 10:02:17 +01:00
Marc Bonnici
8654fcfc30 framework/execution: Treat reboot as part of job allowing retires
Treat the optional reboot of the device as part of the job itself so that
if the reboot fails it will be retried in accordance to the job
configuration.
2018-05-25 10:02:17 +01:00
Marc Bonnici
4a427f830b framework/execution: Move start/stop calls to the TargetManager to Runner
Instead of starting and stopping the target from inside the Context,
move the calls to the Runner. This allows for the any initilizaton
errors to be dealt with as part of the job and retried as
appropriate.
2018-05-25 10:02:17 +01:00
Marc Bonnici
c3e4bdc964 config/core: Re-add by_spec reboot policy
The `by_spec` reboot policy was removed in commit
0a549bd28e however this is still
desirable so re-add.
2018-05-25 10:02:17 +01:00
Sergei Trofimov
23946620d6 fw/job: add missing log indents
Add "with indentcontext():" to Job stages that were missing them.
2018-05-24 17:44:03 +01:00
Marc Bonnici
03e5c6c0af framework/pluginloader: Make sure to also clear aliases
When 'clearing' the pluginloader previously any aliases already
discovered were not removed, now ensure all discovered items are
removed.
2018-05-24 16:57:47 +01:00
Marc Bonnici
30f856abe0 framework/execution: Only add screenshots as an artifact if present
When attempting to take a screenshot from a device do not try to add it
as an artifact if screenshot failed and is not present.
2018-05-24 11:49:10 +01:00
Marc Bonnici
c3854e89c7 framework/host: Fix Typo 2018-05-24 11:49:10 +01:00
Marc Bonnici
e008134145 framework/host: Add missing defaults for agenda conversion
Ensure that defaults are correctly populated when converting a WA2 agenda
to the WA3 style.
2018-05-24 11:49:10 +01:00
Marc Bonnici
f49dfeac70 config/core: Remove JobGenerator unused sections attribute
Remove the used attribute `sections` from the JobGenerator
2018-05-24 11:49:10 +01:00
Marc Bonnici
a13e205280 fw/configuration: Allow for numerical ids to be used
Previously WA would fail if a purely numerical id was used, now
explicitly convert the id to a string before use.
2018-05-24 11:49:10 +01:00
Marc Bonnici
2abfe68b07 fw/parsers: Raise error if no workload is specified
Ensure that an error is raised if no workload name is specified in an
agenda.
2018-05-24 11:49:10 +01:00
Sergei Trofimov
063212c2a9 fw/config: resolve aliases inside plugin_cache
- expose pluginloader's resolve_alias in plugin_cache (it's supposed
  to be a transparent proxy for pluginloader).
- resolve aliases inside _set_plugin_defaults() to make sure the
  correct defaults are used when creating job specs.
2018-05-23 14:21:11 +01:00
Sergei Trofimov
775f3d55f2 fw/plugin: return a copy of alias params
When resolving aliases, return a copy of the alias' params to ensure
they don't get modified by the calling code.
2018-05-23 14:21:11 +01:00
Marc Bonnici
dc41af1f3f framework/tm: Ensure RebootPolicy is respected with unresponsive target
Expose the RebootPolicy as an attribute of the ExectionContext and ensure
that that the target is not automatically rebooted if the reboot
policy is set to "Never".
2018-05-23 10:14:34 +01:00