1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

306 Commits

Author SHA1 Message Date
Sergei Trofimov
42fb3eb148 framework/workload: make "no matching package" message more useful
Make the error message reported when no matching Android app package
could be found more useful by including the workload name, and, if
applicable, sought version.
2018-02-09 17:45:04 +00:00
Marc Bonnici
22a2ccf7d6 target/logcatpoller: Make capitalisation consistent 2018-02-07 10:01:12 +00:00
Marc Bonnici
728057dbba target/androidassistant: Fix Logcat poller creation
Threads cannot be restarted therfore we need to create a new polling
thread for each time the poller is started.
2018-02-07 10:01:12 +00:00
Marc Bonnici
07cd770288 framework/plugin_cache: Fix missing assistant parameters 2018-02-07 10:01:12 +00:00
Marc Bonnici
36309a4cf2 target/assistant: Fix logcat poller
Rename the `start` method to `run` as this is what is what is called
by the threading module's `start` method, otherwise this causes the
polling to be done in the main thread blocking execution.
2018-02-07 10:01:12 +00:00
Sergei Trofimov
c5df74eb56 framework/config: load plugins from extra packages
Add support for loading plugins from packages that register themselves
with WA by writing their names into $WA_USER_DIRECTORY/packages.

This is functionality that existed in WA2 and was omitted in WA3 until
now.
2018-02-06 17:30:49 +00:00
Sergei Trofimov
58c4ae609c framework/output_processor: fix get_output_processor()
Output processor names are converted to identifiers when they are
loaded. Ensure the same thing happens when looking up installed output
processors with get_output_processor().
2018-02-06 09:49:58 +00:00
Marc Bonnici
9a556721b6 Augmentations: Allow configuring of augmentations per workload
Add support for enabling and disabling of augmentations (instruments and
output processors) on a per workload basis.
2018-02-01 14:26:08 +00:00
Marc Bonnici
557d62ce86 framework/configuration: Fix disabling of augmentations
Previously disabling of augmentaions via using the `--disable`
flag or by using the "~" notation in agendas was ignored.
2018-02-01 14:26:08 +00:00
Sergei Trofimov
dae4d9a934 framework/execution: add context to job signals
Pass the execution context to job execution signal callbacks.
2018-01-25 10:50:02 +00:00
Sergei Trofimov
a2c113e0f9 framework/signal: log on connect/disconnect
Add debug level logging for connect() and disconnect() functions.
2018-01-25 10:50:02 +00:00
Sergei Trofimov
fc56de8842 framework/instruments: add ManagedCallback __repr__
Add a __repr__ for ManagedCallback callback to prove a useful
representation in logging.
2018-01-25 10:50:02 +00:00
Marc Bonnici
1101c358d0 framework/rt_config: Add support for chromeos for some AndroidRTConfig
Allows setting of select android runtime configuration for devices running
chromeos and which support android. Currently only 'brightness' is functioning
correctly therefore the other parameters are only enable for standard android devices.
2018-01-25 06:32:37 +00:00
Marc Bonnici
320da77ac0 framework/workload: Adds support for ChromeOs Targets
Adds a check if the target is running ChromeOs and if so only use the
Android Container portion for running thr workload.
2018-01-25 06:32:37 +00:00
Marc Bonnici
b4d6017dd4 framework/descriptor: Add ChromeOsTarget 2018-01-25 06:32:37 +00:00
Marc Bonnici
a7bf5c2df9 ChromeOsAssistant: Add assistant for ChromeOS 2018-01-25 06:32:37 +00:00
Marc Bonnici
776a4d850b framework/rt_config: Remove over cautious error checking.
Previously when validating cpufreq runtime parameters it would check too
early whether the cpu was online, not allowing for the fact that the cpu
might be being onlined before the parameter was committed to the device.
2018-01-18 16:16:45 +00:00
Marc Bonnici
3cf789e352 framework/rt_config: Fix not being able to turn off all cpus in cluster
Add missing equality sign in constraint for runtime parameters.
2018-01-18 16:16:45 +00:00
Marc Bonnici
60c16c802f framework/rt_config: Fix typo in hotplug big.LITTLE RT Params 2018-01-18 16:16:45 +00:00
Marc Bonnici
7100165376 configuration/core: Fix alias for 'results_processors'
Adds missing 's' for alias name.
2018-01-17 16:41:49 +00:00
Marc Bonnici
446a1cfbb0 wa: Rename Instrumentation to Instruments
To be maintain a consistent naming scheme, rename all instances of
`Instrumentation` to `Instruments`
2018-01-15 10:24:03 +00:00
Marc Bonnici
987f4ec4f1 wa: Rename update_results reference to update_output
To be consistant with the rename of output_processors, old references
to `update_results` have now been changed to `update_output`.
2018-01-15 10:24:03 +00:00
Marc Bonnici
538cd8d332 wa: Rename results_processors to output_processors
For clarity and to better reflect their purpose, rename
`results_processors` to `output_processors`.
2018-01-15 10:24:03 +00:00
Marc Bonnici
a688c9120e framework/configuration: Fix missing return statements for Jobspec properties 2018-01-12 17:28:56 +00:00
Marc Bonnici
bd9189f26a framework/getters: Fix missing return in Filer Getter 2018-01-12 17:28:56 +00:00
Marc Bonnici
58131c1a6d framework/plugin: Fix clearing of loaded plugin
`self.plugins` should be a dictionary and when being cleared was being
reinitialized as a list.
2018-01-12 17:28:56 +00:00
Marc Bonnici
4fc39dedbc framework/execution: Fix message parameter order 2018-01-12 17:28:56 +00:00
Marc Bonnici
937d417a42 framework/parsers: Fix whitespace 2018-01-12 17:28:56 +00:00
Marc Bonnici
ad530a8925 framework/parsers: Fix incorrect method name 2018-01-12 17:28:56 +00:00
Sergei Trofimov
fbb1a125fd framework/target: fix caseless runtime params
Fix the setting of runtime parameters when the casing of the parameter in
the agenda does not match the "canonical" casing.

To make the writing of agendas easier, the casing of the parameters is
supposed to be ignored. To achieve this, parameter names are converted to
caseless_string type before they are looked up.

caseless_string's do not work with dicts. Both __contains__ (used for
the "in" operator) and __getitem__ (used for the [] operator) implement
hash-based look up, and it is not possible to have a caseless_string
match against multiple hashes to cover the different casing
possibilities.

So instead, iterate over the items in parameters dict, comparing the
caseless_string name to the key, and returning the value if it matches.
2018-01-11 11:04:25 +00:00
Marc Bonnici
51794c99e9 framework/workload: Fix incorrect regex matching apk filepath
On some devices an installed apk filepath can contain an '=' character
which was previously used to end the regex match. Now match with the
package name as well to ensure the file path is extracted correctly.
2018-01-09 10:45:58 +00:00
Marc Bonnici
2dc1d2e54e confirguration/core: Add parameter default to MetaConfig
Uses os.eviron as the default environment for creation of a
MetaConfiguration object, this allows an instance to be created without
providing any parameters and fixes re-creating an instance from pod.
2018-01-08 10:36:57 +00:00
Marc Bonnici
6511db1559 framework/output: Fix RunOutput read_config
Fixes trying to deserialize wrong class type.
2018-01-08 10:36:57 +00:00
Sergei Trofimov
833c3dd0a0 framework/target: manager: use get_target_description
use the new get_target_description() when initializing the target inside
TargetManager, instead of manually searching through results of
list_target_descriptions().
2017-12-15 17:07:34 +00:00
Sergei Trofimov
d11165b703 framework/target: add get_target_descriptor()
Add a function to get a TargetDescriptor by its name.
2017-12-15 17:07:34 +00:00
Sergei Trofimov
0a67e94709 framework/target: rename get_target_descriptions
Rename get_target_descriptions to list_target_descriptions.
2017-12-15 17:07:34 +00:00
Marc Bonnici
7bd99637c1 Target/AndroidAssistant: Try to disable selinux
If the target is rooted, attempt to disable selinux as this can cause
issues during runtime.
2017-12-14 17:56:29 +00:00
Marc Bonnici
103368415b target/descriptor: Fix sudo_cmd parameter description 2017-12-14 09:50:26 +00:00
Marc Bonnici
346ce2fb15 framework/workload: Fix typo and formatting 2017-12-14 09:50:26 +00:00
Marc Bonnici
6df498b4bd framework/workload: Fix path of deployed asset
Get pushed file basename to ensure the assets deployed path is correct.
2017-12-14 08:15:39 +00:00
Sergei Trofimov
d1a42f6e64 framework/output: add discover_wa_outputs
Add a function that traverses a directory structure looking for WA
output directory and yielding corresponding RunOutput objects.
2017-12-13 15:55:38 +00:00
Sergei Trofimov
bb9f735cd4 framework/target: assistant: fix import
Import Parameter from the module that defines it rather than from wa to
avoid circular import errors.
2017-12-13 15:55:38 +00:00
Sergei Trofimov
c1319ce8cb framework/output: load jobs on RunOutput creation
Populate RunOutput's jobs attribute with job outputs on its creation.
2017-12-13 15:55:38 +00:00
Sergei Trofimov
4a239790d2 framework/output: add target_info to RunOutput
Make TargetInfo an attribute of run output, replacing the read/write
methods for the targetfile. Instead, always load it on creation, if
targetfile exists (useful for external scripts), and have a method to
set it after creation (uselful during WA run, where the output is
created before connecting to the target).
2017-12-13 15:55:38 +00:00
Sergei Trofimov
3ab0aa04de framework/target: fix TargetInfo deserialization
Fix TargetInfo.from_pod(). As part of creating the POD, some of devlib's
classes such as Cpuinfo, get serialized. Since they are from devlib,
they do not implement WA's serialization interface, and are instead
serialized by extracting their internal structures. They are then not
deserialized properly, since their __init__'s expect text rather than
parsed structures.

This implements deserialization functions for these classes which
initialize them properly from TargetInfo POD.
2017-12-13 15:55:38 +00:00
Sergei Trofimov
ed691a5335 framework/target: add shell_prompt target param.
Add a Parameter to specify a regex that matches the shell prompt on the
target. This used in establishing serial tty connections (e.g. VExpress
UART).
2017-12-12 10:07:04 +00:00
Marc Bonnici
0d45fa3b27 Revert "DO NOT MERGE: stub out broken method"
This reverts commit 7a3503775d.
2017-12-07 15:45:38 +00:00
Brendan Jackman
07243740d6 framework/plugin: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
56674aa3a9 framework/plugin: Satisfy pylint's nitpicks 2017-12-07 15:37:49 +00:00
Brendan Jackman
feabecd19d framework/workload: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
69f52f3359 framework/workload: Remove unused imports 2017-12-07 15:37:49 +00:00
Brendan Jackman
d8a44adde8 framework/run: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
c8d1b0e4fb framework/job: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
8032abc805 framework/run: Store RunInfo attributes from constructor 2017-12-07 15:37:49 +00:00
Brendan Jackman
3337cd477e framework/run: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
cf67a7784c framework/getters: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
4d7b76fd98 framework/getters: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
b05b0a2e4f framework/output: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
7a3503775d DO NOT MERGE: stub out broken method 2017-12-07 15:37:49 +00:00
Brendan Jackman
b5ac669994 framework/output: Fix checking artifact kind
Don't construct an ArtifactType in Output.Add_artifact, the Artifact
class does that for us.

Next, fix the use of a nonexistent attribute Artifact.valid_kinds
2017-12-07 15:37:49 +00:00
Brendan Jackman
3904096257 framework/plugin: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
6fc247a8fd framework/output: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
664f0a8905 framework/config: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
81d4587da3 framework/output: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
aec9a4b6fc framework/host: Add missing import 2017-12-07 15:37:49 +00:00
Brendan Jackman
6f59f8eb1b framework/agenda: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
5a373e6c7b framework/host: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
b861ca986a framework/host: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
33e5eae887 framework/entrypoint: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
f935c48d47 framework/execution: Add basic docstring for Runner class 2017-12-07 15:37:49 +00:00
Brendan Jackman
e3ae7fc906 framework/entrypoint: Whitespace 2017-12-07 15:37:49 +00:00
Brendan Jackman
79ea06044b framework/execution: Calm pylint 2017-12-07 15:37:49 +00:00
Brendan Jackman
20be13282b framework/execution: Remove more unused imports
autoflake didn't find these ones, I guess because of the brackets.
2017-12-07 15:37:49 +00:00
Brendan Jackman
aae4b465bb Remove trainling newlines
Calms pylint down
2017-12-07 15:37:49 +00:00
Brendan Jackman
aa2abd16a3 Run autoflake to remove unused imports & variables 2017-12-07 15:37:49 +00:00
Marc Bonnici
b85098d5b2 framework/entrypoint: Fix help information for subcommands
Previously only the top level help message would ever be displayed,
this was caused by 'parse_known_commands' automatically displaying the
default help message and exiting before any of the custom plugins are
loaded. Now ensure this flag is never passed into the method.
2017-12-06 17:05:52 +00:00
Sergei Trofimov
7796dabe90 framework: remove wa.framework.plugin.Artifact
Remove wa.framework.plugin.Artifact and associated references. The name
of the class clashes with the class from output and can potentially
cause confusion.

The original intention for this was to be an "expected artifact
descriptor" of sorts that plugins can specify for validation purposes,
but that functionality was never implemented. Given that the framework
has undergone significant changes since this was implemented, it's not
clear that this is the best way to go about the original goal.

Therefore remove this for now.
2017-12-06 16:25:54 +00:00
Marc Bonnici
698c61b0a5 framework/workload: Adds support for gui only workloads
Adds new workload types to allow for using a gui without requiring an
apk file.
2017-11-30 17:17:18 +00:00
Marc Bonnici
0f28a07476 framework/workload: Rename ReventWorkload to ApkReventWorkload
To reflect that the current implementation of ReventWorkload uses an APK
file on the device the name has been changed appropriately and the
relevant workloads have been updated.
2017-11-30 17:17:18 +00:00
Sergei Trofimov
0778766c8b framework/target: add adb_name alias
Add "adb_name" alias for "device" parameter of AdbConnection in order to
be backwards-compatible with WA2.
2017-11-29 13:19:39 +00:00
Sergei Trofimov
31e08bf797 framework/target: handle aliases
- Use get_config_point_map() to ensure aliases are parsed from
  configuraiton.
- Only set the default for a name only if it matches the corresponding
  param's name (i.e. don't set for alises).
2017-11-29 13:19:39 +00:00
Sergei Trofimov
a7a5d09dfa framework/config: plug_cache: handle aliases
Use get_config_point_map() when returning parameter mappings to ensure
aliases are included.
2017-11-29 13:19:39 +00:00
Sergei Trofimov
78a569a2d3 framework/config: add get_config_pont_map()
Add a function to generate a dict mapping names and aliases to
corresponding configuration point objects.
2017-11-29 13:19:39 +00:00
Marc Bonnici
6cc4021017 Framework/Uiauto: Add support for Applaunch 2017-11-28 08:12:31 +00:00
Marc Bonnici
ac5741136d Framework/Workload: Fix super propagation.
Previously super calls were not being propagated correctly so either
call the relevant super or remove the empty methods.
2017-11-28 08:12:31 +00:00
Marc Bonnici
74347b9e67 Framework/Workload: Change condition for pushing assets
Changes from checking if there are deployable assets to checking for
discovered assets. This allows for automated pushing of assets without
invoking the resource getters, for example pre discovered files.
2017-11-28 08:12:31 +00:00
Marc Bonnici
30da77a1cf Framework: Moved resource deployment to base workload 2017-11-28 08:12:31 +00:00
Sergei Trofimov
d7da24eb52 framework/config: fix section-level workload args
create a copy of the args dict before popping args off of it to make
sure it is intact when the next job is processed.
2017-11-24 16:35:07 +00:00
Sergei Trofimov
9a15b82997 framework/job: do not extract workload results on failure
Now that job output processing happens even on failure, this will result
in workload result processing also being triggered. We should not try to
extract failed workload results, so check the job status before
attempting to do so.
2017-11-24 14:30:09 +00:00
Sergei Trofimov
45d8be5228 framework/execution: ensure output is always processed
Ensure that job output is processed even if a workload fails. This is
because output processing includes things like extracting logs, which
we still want to happen on failure.

Job status is now also set correctly when an error occurs during output
processing rather than actual running of the workload. Previously, the
status would be correctly set to PARTIAL in the inner except clause,
but the exception is then re-raised, and the status was "upgraded" to
FAILED in the outer except clause.
2017-11-24 14:30:09 +00:00
Sergei Trofimov
7cd2982b31 workloads/uxperf: change message format
Do not concatenate start/stop markers to the action string with a '_',
delimit them with a space instead. This will make easier to parse the
message afterwards.
2017-11-24 14:30:09 +00:00
Marc Bonnici
2611579d29 Framework/Workload: Fix typo 2017-11-23 17:49:30 +00:00
Marc Bonnici
e9ae08e00b Framework/Workload: Utilize package_names during package resolution
Iterate through available package names when resolving an apk file from
the host.
2017-11-23 17:49:30 +00:00
Marc Bonnici
ff529462e0 Framework/Workload: Require package name for package resolution
Mandate that either a package name is specified or the owner defines
supported package names. This prevents any apk file in the correct
location incorrectly being resolved.
2017-11-23 17:49:30 +00:00
Sergei Trofimov
49eb40fec7 framework/workload: add markers_enabled param for uiauto workloads
Some workloads can insert timestamped markers into the logs if this
parameter is enabled in their automation. These markers can then be
used to look at measurements for specific parts of the execution.
2017-11-22 09:22:21 +00:00
Sergei Trofimov
73a530917a framework/workload: fix asset_files and deployed_assets
These two lists are updated by workloads as assets are discovered and
deployed. As these are defined as class attributes in the base Workload
class, the same two list instances are shared between all classes. This
results in subsequent workloads re-deploying assets from the previous
workloads, and may lead to mismatches with deployable_assets, which is
overwritten on per-workload basis.

Make these into instance attriutes to prevent these issues.
2017-11-22 09:22:21 +00:00
Sergei Trofimov
652f107d10 framework/workload: fix class attribute docs
Change from free-floating strings into comments and move above the
corresponding attribute.
2017-11-22 09:22:21 +00:00
Quentin Perret
be18157223 framework/target: wait 20 sec for network to show up
Some targets need more than 5 seconds to re-connect to a wifi network
after disabling airplane mode. To fix that, poll for network up to 20
seconds and show a warning if unreachable.
2017-11-17 07:53:16 +00:00
Marc Bonnici
8a913639bd Framework/Workload: Fix call to deploy assets 2017-11-15 15:59:27 +00:00
Marc Bonnici
1f5a34b9a8 Framework/Uiautomator: Adds Ui Swipe functions 2017-11-15 14:47:36 +00:00