1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 10:51:13 +01:00
Commit Graph

1071 Commits

Author SHA1 Message Date
Jonathan Paynter
716e59daf5 framework/target: Add logcat buffer wrap detection
As WA currently supports either a single logcat dump after each job,
or fixed rate polling of logcat, it is possible that the fixed size
logcat buffer wraps around and overwrites data between each dump or
poll. This data may be used by output processors that should be
notified of the loss.

This change allows the detection of buffer wrapping by inserting a
known log entry into the buffer, although it cannot say how much data
was lost, and only applies to the "main" logcat buffer.

If buffer wrap is detected, a warning is logged by WA.
2020-07-16 11:38:06 +01:00
Marc Bonnici
08fcc7d30f fw/getters: Fix typo 2020-07-15 15:04:31 +01:00
Marc Bonnici
684121e2e7 fw: Replace usage of file locking with atomic writes
To prevent long timeouts occurring during to file locking on
both reads and writes replace locking with
atomic writes.

While this may results in cache entries being overwritten,
the amount of time used in duplicated retrievals will likely
be saved with the prevention of stalls due to waiting to
acquire the file lock.
2020-07-15 15:04:31 +01:00
Marc Bonnici
0c1229df8c utils/misc: Implement atomic writes
To simulate atomic writes, use a context manager to write to
a temporary file location and then rename over the original
file.
This is performed using the `safe_move` method which performs
this operation and handles cases where the source and destination
are on separate file systems.
2020-07-15 15:04:31 +01:00
Marc Bonnici
615cbbc94d fw/target_info: Prevent multiple parses of the target_info_cache
Instead of parsing the target_info_cache multiple times,allow for
it to be read it once and passed as a paramter to the coresponding
methods.
2020-07-15 15:04:31 +01:00
Marc Bonnici
1425a6f6c9 Implement caching of ApkInfo
Allow caching of ApkInfo to prevent the requirement of re-parsing
of APK files.
2020-07-15 15:04:31 +01:00
Marc Bonnici
4557da2f80 utils/android: Implement a Podable wrapper of ApkInfo
Add a Podable warpper to ApkInfo.
2020-07-15 15:04:31 +01:00
Jonathan Paynter
7cf5fbd8af framework, tests: Correct signal disconnection
While the Louie system operated on weakrefs for the callback
functions, the priority list wrapper did not. This difference led to
weakrefs to callback functions being compared to strong references in
list element operations within Louie's disconnect method, so that
handler methods were not disconnected from signals.

Converting the receiver to a weakref then allowed Louie to operate as
normal, which may include deleting and re-appending the handler method
to the receivers list. As ``append`` is a dummy method that allows the
priority list implementation, the handler method is then never added
back to the list of connected functions, so we must ``add`` it after
``connect`` is called.

Also included is a testcase to confirm the proper disconnection of
signals.
2020-07-14 17:31:38 +01:00
Jonathan Paynter
3f5a31de96 commands: Add report command
report provides a summary of a run and an optional list of all
jobs in the run, with any events that might have occurred during each
job and their current status.

report allows an output directory to be specified or will attempt to
discover possible output directories within the current directory.
2020-07-14 17:31:38 +01:00
Jonathan Paynter
7c6ebfb49c framework: Have Job hold its own JobState
JobState, previously handled by RunState, is now held in the
Job.

Changes and accesses to a Job's status access the Job's
JobState directly, so that there is only one place now that each Job's
state data is tracked.

This also means there is no use for update_job in RunState.
2020-07-14 17:31:38 +01:00
Jonathan Paynter
8640f4f69a framework: Add serializing Job status setter
When setting the job status through ExecutionContext, this change
should be accompanied by an update to the state file, so that the state
file accurately reflects execution state.

As Jobs should not be aware of the output, this method is added to
ExecutionContext, and couples setting job state with writing to the
state file.
2020-07-14 17:31:38 +01:00
Jonathan Paynter
460965363f framework: Fix serialized job retries set to 0
JobState serializations did not reflect the current state of
execution, as the 'retries' field was set to 0 instead of
JobState.retries.
2020-07-14 17:31:38 +01:00
Chris Redpath
37f4d33015 WA/Jankbench: Update Pandas function to remove deprecated .ix access
Pandas removed .ix as a way to iterate the index, .loc is the replacement
in most cases. Jankbench as a workload fails on a clean install due to
this call.

Replacing this works for me on a native install of Lisa with Ubuntu 20.04

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
2020-07-03 12:13:40 +01:00
Marc Bonnici
8c7320a1be workloads/gfxbench: Swtich to the home screen before run
Alter the element to check popups have closed to be present
on any main screen and ensure we switch to the homescreen of the
app before performing setup steps.
2020-06-30 16:51:11 +01:00
Marc Bonnici
0c2613c608 fw/execution: Fix missing parameter 2020-06-29 16:22:13 +01:00
Marc Bonnici
39121caf66 workloads/gfxbench: Fix using the correct scrollable element.
On smaller devices there can be multiple scrollable elements, ensure
we scroll the correct one to identify tests.
2020-06-19 11:24:46 +01:00
Marc Bonnici
83da20ce9f ouput_processor/postgres: Fix events sql command 2020-06-15 15:30:56 +01:00
Marc Bonnici
f664a00bdc config/core: Fix handling of depreciated parameters
Provide warning to user when attempting to set a depreciated
parameter instead of during validation and only raise the warning
if a value has been explicitly provided.
2020-06-12 09:24:51 +01:00
Marc Bonnici
443358f513 workloads/gfxbench: Rework score detection
Rework how the result matching is performed. Some tests from
gfxbench provide more than 1 score per test and
some provide their output in a different format to others.
Update the matching to perform more flexible matching as well
as dealing with entries that do not fit on a single results screen.
2020-06-10 11:10:26 +01:00
Marc Bonnici
58f3ea35ec workloads/gfxbench: Fix incorrect parameter name 2020-05-26 20:55:58 +01:00
Marc Bonnici
7fe334b467 workloads/gfxbench: Fix incorrect parameter name 2020-05-26 20:38:58 +01:00
Marc Bonnici
3967071a5e workloads/gfxbench: Fix incorrect parameter name 2020-05-26 20:13:53 +01:00
Marc Bonnici
cd6f4541ca workloads/gfxbench: Move results extraction to the extraction stage 2020-05-21 12:39:25 +01:00
Marc Bonnici
7e6eb089ab workloads/geekbench: Update result screen matching criteria
Update the element that is searched for as on some devices this can
match before all the tests are complete.
2020-05-21 12:39:25 +01:00
Marc Bonnici
7a085e586a workloads/gfxbench: Allow configuration of tests to be ran.
Allow the user to customise which tests are to be ran on the device.
2020-05-21 12:39:25 +01:00
Marc Bonnici
0f47002e4e fw/getters: Use the assets_repository as the default for the filer 2020-05-21 12:39:25 +01:00
Marc Bonnici
6ff5abdffe fw/config: Remove whitespace 2020-05-21 12:39:25 +01:00
Marc Bonnici
82d09612cb fw/config: Add default to `assets_repository' 2020-05-21 12:39:25 +01:00
Marc Bonnici
2d32d81acb utils/file_lock: Create lock files in system temp directory
Use the original file path to create a lock file in the system temp
directory. This prevents issues where we are attempting to lock a file
where wa does not have permission to create new files.
2020-05-19 17:55:40 +01:00
Marc Bonnici
b9d593e578 fw/version: Development version bump
Bump dev version to synchronise interface for SSHConnection with devlib.
2020-05-13 16:43:03 +01:00
Marc Bonnici
1f8be77331 Disable pep8 errors 2020-05-13 16:43:03 +01:00
Marc Bonnici
66f0edec5b descriptor/SSHConnection: Expose use_scp parameter
Allow specifying to use scp for file transfer rather than sftp as
this is not supported by all targets.
2020-05-13 16:43:03 +01:00
Marc Bonnici
e2489ea3a0 descriptor/ssh: Add note to password parameter for passwordless target
For a passwordless target the `password` parameter needs to be set to an
empty string to prevent attempting ssh key authentication.
2020-05-13 16:43:03 +01:00
Rob Freeman
16be8a70f5 Fix pcmark setup
* Pcmark sometimes auto installs without need for clicking button,
in such cases workload throws UiObjectNotFound exception.
* Added logic to check for installation button existence.
* Increased install wait time to 5 mins.
2020-04-28 09:57:41 +01:00
Rob Freeman
dce07e5095 Update gfxbench to log correct scores.
* Updated regex to reflect correct test name.

* Enabling/disabling tests on setup was missing tesselation on some devices
  so changed order of toggle.

* Sometimes whilst collecting scores the workload grabs the wrong score.
  Updated to check the name of the test before grabbing the score.

* Tested on mate20, xperia, s9-exynos, s10-exynos, pixel-4
2020-04-27 13:54:48 +01:00
Marc Bonnici
9b19f33186 target/descriptor: Fix overwriting variable
Ensure we don't overwrite `conn_params` in the inner for loop.
2020-04-17 13:03:27 +01:00
Marc Bonnici
53faf159e8 target/descriptor: Cosmetic fixes
Fix typo and choose more descriptive variable name.
2020-04-17 13:03:27 +01:00
Marc Bonnici
84a9526dd3 target/descriptor: Fix handling of custom Targets 2020-04-17 13:03:27 +01:00
Marc Bonnici
a3cf2e5650 descriptor: Fix overriding of parameters
Make sure we only override parameters that are present in the current
config. This allows for connection parameters to be supplied for a
platform but only overridden if required for the connection.
2020-04-16 09:44:17 +01:00
Marc Bonnici
607cff4c54 framework: Lock files which could be read/written to concurrently
Add file locking to files that could be read and written to concurrently
by separate wa processes causing race conditions.
2020-04-09 09:14:39 +01:00
Marc Bonnici
d56f0fbe20 utils/misc: Add file locking context manager
Enable automation locking and unlocking of a file path provided. Used to
prevent synchronisation issues between multiple wa processes.
2020-04-09 09:14:39 +01:00
Marc Bonnici
0f9c20dc69 target/descriptor: Add support for connection parameter overriding.
Allow for overriding connection parameters on a per platform basis, and
make the `host` parameter for `Juno` optional as this can be auto
detected via the serial connection.
2020-04-09 09:10:11 +01:00
Marc Bonnici
310bad3966 target/descriptor: Rework how parameter defaults are overridden.
Instead of supplying only the parameter name and value to be set as a
default, allow for replacing the entire parameter object as this allow
more control over what needs overriding for a particular platform.
2020-04-09 09:10:11 +01:00
Marc Bonnici
a8abf24db0 fw/descriptor: Add unsupported_platforms for a particular target
Allow for specifying a list of `Platforms` that a particular target does
not support, e.g. 'local_juno'
2020-04-09 09:10:11 +01:00
Marc Bonnici
dad0a28b5e logcat_parsing: Replace errors when decoding logcat output
Some devices print non standard characters to logcat. If an error
occurs when parsing the output, replace the offending character instead
of raising an error.
2020-04-07 14:15:24 +01:00
Rob Freeman
5049e3663b Force speedometer to use chrome and change to ApkUiAutoWorkload
* Workload was failing when chrome was not set as default broser so
  altered to use chrome every time.

* Changed workload to an ApkuiAutoWorkload since chrome is now a
  dependency.

* Refactored opening speedometer to new method.

* Added wait time for scores to show up when test finished.
2020-03-31 10:49:25 +01:00
Rob Freeman
c9ddee761a Update framework to wait for object before dismissing chrome popup
* Added wait for exist for google terms accept.

* Reduced wait time for device sync negative button to reduce workload run
  time.
2020-03-31 10:49:25 +01:00
scojac01
3be00b296d Androbench: Handle storage permissions prompt.
Updating the workload to handle the storage permissions that present themselves on certain devices.
2020-03-25 18:21:43 +00:00
scojac01
9a931f42ee Handle the common chrome browser popup messages.
The Chrome browser presents a number of popups when run for
the first time. This update handles those popup messages.
2020-03-25 16:35:04 +00:00
Marc Bonnici
06ba8409c1 target/descriptor: Make strict_host_check default to False
The majority of users will not find a benefit of the additional
check so make this parameter default to `False` instead.
2020-03-12 11:21:07 +00:00