1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-05 18:31:12 +01:00
Commit Graph

2105 Commits

Author SHA1 Message Date
Marc Bonnici
0e2a150170 fw/version: Bump release versions 2020-12-11 16:31:13 +00:00
Marc Bonnici
69378b0873 Dockerfile: Bump release version 2020-12-11 16:31:13 +00:00
Marc Bonnici
c543c49423 requirements.txt: Update to latest tested versions 2020-12-11 16:31:13 +00:00
Marc Bonnici
dd07d2ec43 workloads/speedometer: Fix markdown formatting in docstring 2020-12-11 16:26:49 +00:00
Marc Bonnici
94590e88ee docs/changes: Update changelog for v3.3 2020-12-11 16:26:49 +00:00
Marc Bonnici
c2725ffaa2 fw/uiauto: Update to handle additional permissions screen
On the latest version of android (currently Q) for applications that
are designed to run on older versions of android, an additional
screen asking to confirm the required permissions can popup.
Enable confirming of the granted permissions.
2020-12-10 15:57:30 +00:00
Marc Bonnici
751bbb19fe docs: Update Signal Dispatch diagram
Update the signal dispatch diagram to include the new optional reboot stage.
2020-12-09 07:48:20 +00:00
Marc Bonnici
ae1bc2c031 fw/config: Add additional run_completed reboot policy
Add an additional `run_completed` reboot policy for when a run
has finished.
This complements the `initial` reboot policy and aims to leave
the device in a fresh state after WA has finished executing.
2020-12-09 07:48:20 +00:00
Marc Bonnici
91b791665a workloads/googleplaybooks: Update to handle updated IDs
Resourceid and classes have been modified so update the
workload to handle these cases.
Additionally on some devices regex matches appear to fail
so workaround to match separately.
2020-11-29 19:42:30 +00:00
Marc Bonnici
62c4f3837c workloads/googleslides: Update to accommodate newer versions
Add support for newer version of the apk.
Also add support for differing screen sizes, on larger devices
the direction of swipe to change slide differs, perform both
horizontal and vertical swipes to satisfy both layouts.
2020-11-29 19:42:30 +00:00
Marc Bonnici
3c5bece01e workloads/aitutu: Improve reliability of results extraction
Wait for the device to become idle before attempting to extract
the test scores.
2020-11-29 19:42:30 +00:00
Marc Bonnici
cb51ef4d47 workloads/googlephotos: Update to handle new popup
Bump the minor known working version APK and handle a new "missing out"
popup.
2020-11-29 19:42:30 +00:00
Marc Bonnici
8e56a4c831 utils/doc: Fix output for lambda function
The "name" can be in the format "<class>.<lambda>" so
update to allow correct function with the updated format.
2020-11-13 16:27:39 +00:00
Marc Bonnici
76032c1d05 workloads/rt_app: Remove timeout in file transfer
Remove the explict timeout when pushing to the device.
Allow the polling mechanims to monitor the transfer if required.
2020-11-13 15:42:00 +00:00
Marc Bonnici
4c20fe814a workloads/exoplayer: Remove timeout in file transfer
Remove the explict timeout when pushing a media file to the device.
Allow the polling mechanims to monitor the transfer.
2020-11-13 15:42:00 +00:00
Marc Bonnici
92e253d838 workloads/aitutu: Handle additional popup on launch
Allow agreeing to an updated Terms agreement on launch
2020-11-13 11:31:50 +00:00
Marc Bonnici
18439e3b31 workloads/youtube: Update Youtube workload
The previous known working version of the youtube apk appears
to have stopped working. Update to support the new format.
2020-11-12 15:03:01 +00:00
Marc Bonnici
5cfe452a35 fw/version: Bump dev version.
We are relying on a newly available variable in devlib
so bump the version to remain in sync.
2020-11-09 17:56:16 +00:00
Marc Bonnici
f1aff6b5a8 fw/descriptor: Update sudo_cmd default
The WA default `sudo_cmd` is out of date compared to devlib.
Update the parameter to use the value directly from devlib
to prevent these from being out of sync in the future.
2020-11-09 17:56:16 +00:00
Marc Bonnici
5dd3abe564 fw/execution: Fix Typos 2020-11-04 18:27:34 +00:00
Marc Bonnici
e3ab798f6e wl/speedometer: Ensure test package is installed.
Check that the package specified for the test is installed on the
device.
2020-11-04 18:27:34 +00:00
Marc Bonnici
ed925938dc fw/version: Development version bump
Bump the development version to synchronise the parameters for transfer
polling.
2020-11-03 10:02:08 +00:00
Jonathan Paynter
ed4eb8af5d target/descriptor: Add connection config for polls
Adds parameters needed for WA to support file transfer polling.

``poll_transfers`` of type ``bool``, default ``True`` sets whether
transfers should be polled

``transfer_wait_no_poll`` controls the initial time in seconds that the
poller should wait for the transfer to complete before polling its
progress.
2020-11-03 10:01:52 +00:00
setrofim
a1bdb7de45 config/core: merge params from different files
Ensure that runtime and workload parameters specified across multiple
config files and the config section of the agenda are merged rather than
overwritten.
2020-10-30 12:05:30 +00:00
Marc Bonnici
fbe9460995 pylint: Remove uneccessary pass statements 2020-10-30 11:49:54 +00:00
Marc Bonnici
aa4df95a69 pep8: Ignore line break before binary operator
PEP8 has switched its guidance [1] for where a line break should occur
in relation to a binary operator, so don't raise this warning for
new code and update the code base to follow the new style.

[1] https://www.python.org/dev/peps/pep-0008/#should-a-line-break-before-or-after-a-binary-operator
2020-10-30 11:49:54 +00:00
Marc Bonnici
fbb84eca72 Pylint Fixes
Update our version of pylint to use the latest version and update the
codebase to comply with the majority of the updates.

For now disable the additional checks for `super-with-arguments`,
`useless-object-inheritance`, `raise-missing-from`, `no-else-raise`,
`no-else-break`, `no-else-continue` to be consistent with the existing
codebase.
2020-10-30 11:49:54 +00:00
Marc Bonnici
fbd6f4e90c fw/tm: Only finalize the assistant if instantiated 2020-10-30 11:47:56 +00:00
Marc Bonnici
1c08360263 fw/runtime_config: Fix case where no available gov for cpu
Ensure there is a default iterable value in the case there is no
governor entry for a particular cpu.
2020-10-09 08:29:20 +01:00
Vincent Donnefort
ff220dfb44 pcmark: do not clear on reset
The PCMark Work2.0 data-set is cleared and downloaded before each run. This
operation is time-consuming and pollutes the benchmark instrumentation.
Disabling clear_data_on_reset for the PCMark workload bypass this per-run
download.
2020-09-16 18:58:24 +01:00
Stephen Kyle
7489b487e1 workloads/speedometer: offline version of speedometer
This version replaces the previous uiauto version of Speedometer with a new
version.

* Supports both chrome and chromium again, this is selected with the
  chrome_package parameter.
* No longer needs internet access.
* Version 1.0 of Speedometer is no longer supported.
* Requires root:
  - sometimes uiautomator dump doesn't capture the score if not run as root
  - need to modify the browser's XML preferences file to bypass T&C acceptance
    screen
2020-09-16 18:58:01 +01:00
Jonathan Paynter
ba5a65aad7 target/runtime_config: Add support for stay-on
Adds runtime config support for the android setting
``stay_on_while_plugged_in``.
2020-09-10 15:53:03 +01:00
Jonathan Paynter
7bea3a69bb tests: Add tests to check job finalized
Adds tests to see if jobs are finalized after being skipped,
failed. Tests if jobs are labelled as skipped.
2020-09-10 15:52:20 +01:00
Jonathan Paynter
971289698b core,execution: Add run skipping on job failure
Add a global configuration parameter ``bail_on_job_failure`` that
allows all remaining jobs in a run to be skipped should a job fail its
initial execution and its retries. This is by default disabled.
2020-09-10 15:52:20 +01:00
Marc Bonnici
66e220d444 docs/installation: Rephrase dependency information
Reduce emphasis on the Android SDK requirements to prevent confusion when
running with Linux only targets.
2020-09-03 11:40:24 +01:00
Marc Bonnici
ae8a7bdfb5 docs/installation: Update WA's tested platform
Update WA's tested platform to a later version.
2020-09-03 11:40:24 +01:00
Marc Bonnici
b0355194bc docs/installation: Add note about python3 commands 2020-09-03 11:40:24 +01:00
Marc Bonnici
7817308bf7 docs/user_information: Fix formatting and typos 2020-09-03 11:40:24 +01:00
douglas-raillard-arm
ab9e29bdae framework/output: Speedup discover_wa_outputs()
Avoid recursing into subdirectory of folders containing __meta, since
they are not of interest and recursing can take a very large amount of
time if there are lot of files, like if there is a sysfs dump.
2020-08-05 13:17:15 +01:00
Jonathan Paynter
9edb6b20f0 postgres_schemas: Add rules for cascading deletes
Add cascading deletes to foreign keys as well as a rule to delete large
objects when artifacts are deleted.

Deleting a run entry should delete all dependent data of that run.
2020-07-17 13:52:50 +01:00
Marc Bonnici
879a491691 README: Update with more specific supported python version. 2020-07-16 12:20:02 +01:00
Jonathan Paynter
7086fa6b48 target: Force consistent logcat format
On some devices the default logcat format was inconsistent with what was
expected. This change explicitly sets the logcat format to be as
expected.
2020-07-16 11:38:06 +01:00
Jonathan Paynter
716e59daf5 framework/target: Add logcat buffer wrap detection
As WA currently supports either a single logcat dump after each job,
or fixed rate polling of logcat, it is possible that the fixed size
logcat buffer wraps around and overwrites data between each dump or
poll. This data may be used by output processors that should be
notified of the loss.

This change allows the detection of buffer wrapping by inserting a
known log entry into the buffer, although it cannot say how much data
was lost, and only applies to the "main" logcat buffer.

If buffer wrap is detected, a warning is logged by WA.
2020-07-16 11:38:06 +01:00
Marc Bonnici
08fcc7d30f fw/getters: Fix typo 2020-07-15 15:04:31 +01:00
Marc Bonnici
684121e2e7 fw: Replace usage of file locking with atomic writes
To prevent long timeouts occurring during to file locking on
both reads and writes replace locking with
atomic writes.

While this may results in cache entries being overwritten,
the amount of time used in duplicated retrievals will likely
be saved with the prevention of stalls due to waiting to
acquire the file lock.
2020-07-15 15:04:31 +01:00
Marc Bonnici
0c1229df8c utils/misc: Implement atomic writes
To simulate atomic writes, use a context manager to write to
a temporary file location and then rename over the original
file.
This is performed using the `safe_move` method which performs
this operation and handles cases where the source and destination
are on separate file systems.
2020-07-15 15:04:31 +01:00
Marc Bonnici
615cbbc94d fw/target_info: Prevent multiple parses of the target_info_cache
Instead of parsing the target_info_cache multiple times,allow for
it to be read it once and passed as a paramter to the coresponding
methods.
2020-07-15 15:04:31 +01:00
Marc Bonnici
1425a6f6c9 Implement caching of ApkInfo
Allow caching of ApkInfo to prevent the requirement of re-parsing
of APK files.
2020-07-15 15:04:31 +01:00
Marc Bonnici
4557da2f80 utils/android: Implement a Podable wrapper of ApkInfo
Add a Podable warpper to ApkInfo.
2020-07-15 15:04:31 +01:00
Jonathan Paynter
7cf5fbd8af framework, tests: Correct signal disconnection
While the Louie system operated on weakrefs for the callback
functions, the priority list wrapper did not. This difference led to
weakrefs to callback functions being compared to strong references in
list element operations within Louie's disconnect method, so that
handler methods were not disconnected from signals.

Converting the receiver to a weakref then allowed Louie to operate as
normal, which may include deleting and re-appending the handler method
to the receivers list. As ``append`` is a dummy method that allows the
priority list implementation, the handler method is then never added
back to the list of connected functions, so we must ``add`` it after
``connect`` is called.

Also included is a testcase to confirm the proper disconnection of
signals.
2020-07-14 17:31:38 +01:00