1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

858 Commits

Author SHA1 Message Date
Brendan Jackman
016d68bfa0 PluginCache: Improve error message for ill-formed plugin config
Currently if you get confused and write a config with something like:

energy_measurement:
  acme_cape

Then you get an error when we try to 'iteritems' on the value
'acme_cape'. Instead, explicitly check for this case.
2017-10-06 18:00:38 +01:00
setrofim
12edabf753 Merge pull request #501 from bjackman/acme-multiple-instruments2
energy_measurement: Allow multiple IIO devices for ACME
2017-10-06 15:01:31 +01:00
setrofim
ae1363d573 Merge pull request #505 from bjackman/geekbench-fixes
geekbench: Fix use of WA3 APIs
2017-10-06 14:55:37 +01:00
setrofim
ad08cbfe37 Merge pull request #504 from setrofim/energy-backend-param-fix
framework/target: fix runtime param name resolution
2017-10-06 14:54:39 +01:00
Brendan Jackman
6d6fc25e40 geekbench: Fix use of WA3 APIs
Seem I cherry-picked the geekbench workload from my WIP branch
without testing properly, and missed a fixup commit. Here's the
content of that commit, which fixes some missing updates from WA2 ->
WA3 APIs.
2017-10-06 14:51:50 +01:00
Brendan Jackman
e672ea3faf energy_measurement: Allow multiple IIO devices for ACME
Devlib's ACME instrument uses iio-capture under the hood, which can
only capture data from one IIO device at a time. Devlib's instrument
API expects to produce a single CSV file for the Instrument, with a
single axis of sample timestamps. These two things cannot be
correctly reconciled without changing the devlib Instrument API -
get_data would need to be able to return two distinct sets of data.

Instead, where required WA will instantiate the ACME instrument
multiple times (once for each IIO device), producing two separate CSV
files. Aggregated energy info (as opposed to timestamped samples of
energy data) _can_ be meaningfully combined from multiple IIO
devices, so the derived stats are summed.

This requires altering the internal API of the energy_measurement
module - rather than a single instrument the super-instrument now has
a dictionary mapping 'device' keys (this is really an arbitrary key,
but I think that 'device' is a justifiable metaphor for whatever
might separate multiple instruments, plus it is the terminology used
by IIO/ACME) to Instrument objects.  Existing instruments simply
return a dict with a single entry, with None as the key.

It may in future be possible to solve this problem in devlib's ACME
instrument by avoiding the use of the iio-capture tool. In that case,
it should be possible to basically revert this patch and get the
required funcitonality from devlib.
2017-10-06 14:43:54 +01:00
Sergei Trofimov
bce66f4388 framework/target: fix runtime param name resolution
Runtime param names from config were matched against names available on
for the target using an overly-loose regex that resulted in the wrong
config point being returned.

Use caseless_string matching instead.
2017-10-06 14:21:23 +01:00
setrofim
1fe3ed9f96 Merge pull request #503 from setrofim/energy-backend-param-fix
energy instrument backends parameter validation fix
2017-10-06 13:56:46 +01:00
Sergei Trofimov
ce3064dd6f instrument/energy: correct backend param init
As paramters defined by energy_measurement instrument backends were not
used in instantiating the backends, but the underlying devlib
instrument, the values did not undergo the usual Plugin parameter
setting perocedure. In particuar, type conversion would not take place.

Make sure backend parameters are validated properly by setting them on
an obj_dict before passing them  to the devlib instrument.
2017-10-06 13:55:16 +01:00
Sergei Trofimov
109490ec3e utils/types: make list_of classes serializable 2017-10-06 12:55:25 +01:00
Sergei Trofimov
7d9b6d6dd7 framework/config: ensure correct type for param defaults
Call self.kind() on the default value of a parameter before assigning it
to make sure that the assigned value is of correct type.
2017-10-06 12:52:57 +01:00
Sergei Trofimov
6e0cb73a4e framework/config: make LoggingConfig serializable 2017-10-06 12:52:12 +01:00
setrofim
cd328c9a50 Merge pull request #502 from bjackman/youtube-playback
youtube_playback: Add youtube_playback workload
2017-10-06 10:33:32 +01:00
Brendan Jackman
b18db64be1 youtube_playback: Add youtube_playback workload
This is a very simple workload that just aims to allow collecting
instrumentation data on purely the video decoding aspect of Youtube.
2017-10-06 10:31:41 +01:00
setrofim
bec5b0982e Merge pull request #499 from bjackman/next
target/descriptor: Fix typo for 'telnet' param
2017-10-05 13:28:25 +01:00
Brendan Jackman
f767714cd4 target/descriptor: Fix typo for 'telnet' param 2017-10-05 13:27:08 +01:00
setrofim
85d23421b9 Merge pull request #497 from bjackman/phone-home
Add feature to mark workloads unsafe for confidential devices
2017-10-04 15:57:41 +01:00
setrofim
24d96c2397 Merge pull request #496 from bjackman/docstring-tweaks
Misc clarifications/cleanups
2017-10-04 15:56:58 +01:00
Brendan Jackman
73d45b69a5 geekbench: Mark as a workload that may phone home 2017-10-04 15:43:03 +01:00
Brendan Jackman
06b451d31b framework: Add feature to mark workloads unsafe for confidential devices
Some workloads, such as Geekbench, may phone home and report data about the
device they are running on. This poses a risk for users that are testing on
unreleased or otherwise confidential devices - perhaps they use a standard
agenda to run a large battery of tests, in which case they may forget to disable
these dangerous workloads.

This provides a mechanism to prevent running those workloads from running by
setting allow_phone_home=False in the user configuration.
2017-10-04 15:43:03 +01:00
Brendan Jackman
90e4cace06 target/descriptor: Add 'to false' to load_default_modules description 2017-10-04 15:42:18 +01:00
setrofim
5166f3e387 Merge pull request #495 from bjackman/geekbench-package-names
geekbench: Add missing package_names attribute
2017-10-04 13:49:34 +01:00
Brendan Jackman
188e5d752f framework/execution: Whitespace 2017-10-04 13:35:20 +01:00
Brendan Jackman
c4fe5b66e5 target/descriptor: Fix 'devlib' typo 2017-10-04 13:30:12 +01:00
Brendan Jackman
24dc8b2a65 target/descriptor: whitespace 2017-10-04 13:30:10 +01:00
Brendan Jackman
43f5cd793a PluginCache: Check plugin name before trying to iteritems
Currently if your config contains

typo_for_a_global_alias: 1

You will get an error where we try to call 'iteritems' on the value 1. This
commit re-orders the checks so that you instead get an error for the
unrecognised 'typo_for_a_global_alias'.
2017-10-04 13:28:53 +01:00
Brendan Jackman
8daae0752e PluginCache: whitespace 2017-10-04 13:28:52 +01:00
Brendan Jackman
ba01d011c5 geekbench: Add missing package_names attribute 2017-10-04 13:27:57 +01:00
setrofim
1bd19d9af3 Merge pull request #494 from marcbonnici/uiautomator
Uiautomator
2017-10-03 11:20:18 +01:00
Marc Bonnici
049530fe41 Youtube: Removes reference to UxPerf 2017-10-03 11:12:22 +01:00
Marc Bonnici
52b84b4030 Vellamo: Updated to automatically generate PackageID
Now generates packageID instead of using hardcoded values and removed
unused reference to UxPerf.
2017-10-03 11:12:22 +01:00
Marc Bonnici
3b7af8e52c Uiautomator: Adds missing support functionality
Adds initial implementation of ActionLogger class and adds method to
BaseUiAutomation to automatically generate packageID.
2017-10-03 11:12:22 +01:00
setrofim
a876d6b79a Merge pull request #493 from bjackman/geekbench-wa3-proper
Port Geekbench workload from WA2
2017-10-02 16:39:47 +01:00
Brendan Jackman
e4965096e6 geekbench: Error early if target isn't rooted 2017-10-02 16:24:26 +01:00
Brendan Jackman
84404da471 geekbench: Port Geekbench workload from WA2
Differences from original:
- Standard renaming and rearrangement of imports
- UiAutomation.java now uses the setup() and runWorkload() phases which weren't
  present before.
2017-10-02 16:24:23 +01:00
Brendan Jackman
e5f13076ac BaseUiAutomation: Port uiDeviceSwipe methods from WA2 2017-10-02 16:17:36 +01:00
Sergei Trofimov
a65cffb705 trace-cmd: add trace files as artificats
The binary and text trace files were not being added as artifacts in WA
output.
2017-10-02 13:55:58 +01:00
setrofim
66ba28e625 Merge pull request #492 from bjackman/trace-thermal-default
trace-cmd: Enable thermal trace events by default
2017-10-02 13:41:57 +01:00
Brendan Jackman
1eac74fa8f trace-cmd: Enable thermal trace events by default
Due to the nature of modern Android devices, in order to usefully
interpret the power* events, you more and more often need to know the
kernel's view of temperature too. Therefore I think enabling the
thermal event group by default makes sense.
2017-10-02 13:29:10 +01:00
setrofim
d03d950b8e Merge pull request #488 from setrofim/next-fps
Add FPS instrument
2017-09-29 15:33:21 +01:00
Sergei Trofimov
9cb7ea862c framework/workload: default view for ApkWorkload
- Add "activity" property to PackageManager exposing the underlying
  apk_info attribute.
- Add "view" class attribute to ApkWorkload, initialized to None
- If "view" is not set for a particular workload, set a default based on
  the package and activity of the underlying workload.
2017-09-29 14:22:22 +01:00
Sergei Trofimov
4e6ba64737 instrumentation: add FPS instrument.
Add an instrument to collect FPS (frames per second) and associated
rendering statics using corresponding devlib functionality.
2017-09-29 13:50:23 +01:00
setrofim
241b4a5a3a Merge pull request #490 from bjackman/import-apkworkload
wa: Add ApkWorkload to default imports
2017-09-29 13:49:58 +01:00
Sergei Trofimov
19f9998c2e tools/revent: further fixes
- Ensure that CLOCK_MONOTONIC is used for all timestamps, so that the
  timestamps collected by revent and those reported in the struct
  input_event wouild match.

- Do not attempt to free events array inside fini_revent_recroding() if
  no events have been recorded.

- Add missing \n in some debug messages.

- Add end of recording debug message.
2017-09-29 09:48:25 +01:00
Sergei Trofimov
28e339d3b2 tools/revent: timestamp adjustment fixes
- Make sure start/end timestamps are properly adjusted when there are
  zero events.
- Rename adjust_event_times --> adjust_timestamps, as the latter better
  fits its current function.
- Add a couple of debug messages.
2017-09-28 15:11:28 +01:00
Brendan Jackman
14eba69229 wa: Add ApkWorkload to default imports 2017-09-28 13:09:28 +01:00
Sergei Trofimov
831515ae8d instrumentation/energy_measurement: update for DerivedMetrics
DerivedMeasurements in devlib was changed to return DerivedMetrics
instead of Measurements. The former do not have "channel" attribute, so
just pass its name to WA's metrics.
2017-09-28 13:06:33 +01:00
Sergei Trofimov
7b1cc6fa4e utils/serializer: add OrderedDict to POD_TYPES
Allow OrderedDict to be serialized in the same way as regular dicts by
defining it as a Plain Old Data type.
2017-09-28 10:46:03 +01:00
setrofim
1df033fc22 Merge pull request #487 from setrofim/next
Various fixes.
2017-09-27 14:01:21 +01:00
Sergei Trofimov
ecd0dbd55b Add templerun2 workload 2017-09-27 10:34:50 +01:00