1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

873 Commits

Author SHA1 Message Date
Sergei Trofimov
5f7c64b089 framework/resource: add match_path method
This method is used to partially match a resource; its implementation
cannot rely on the resource file actually being present and must match
against the specified path alone.

match() implementation now defaults to match_path(), as for most
resource types, the path is sufficient to uniquely match a resource.
2017-10-10 11:39:53 +01:00
Sergei Trofimov
a6cb9eb6a4 instrument/energy: Fix backend parameter passing
- Convert parameter keys to identifiers before applying
- Raise an error if passed a param invalid for a backend.
2017-10-10 09:51:13 +01:00
setrofim
c3438b6814 Merge pull request #500 from setrofim/next
memcopy workload + fix gem5 support.
2017-10-10 08:58:24 +01:00
setrofim
590309b6be Merge pull request #508 from bjackman/instrumentation-typo
instrumentation: Fix typo
2017-10-10 08:40:25 +01:00
setrofim
bfa7e81996 Merge pull request #507 from bjackman/energy-measurement-check
energy_measurement: Check for failure of Instrument::get_data
2017-10-10 08:40:10 +01:00
setrofim
23b52a4b8e Merge pull request #506 from bjackman/next
PluginCache: Improve error message for ill-formed plugin config
2017-10-10 08:39:23 +01:00
Brendan Jackman
049e2df0dc instrumentation: Fix typo 2017-10-09 18:26:41 +01:00
Brendan Jackman
b4a608da95 energy_measurement: Check for failure of Instrument::get_data
Devlib's get_data API can return None when something goes wrong -
this currently results in us attempting to add a non-existent
artifact.

To avoid this confusion, check and use the return value from
get_data, instead of assuming that it will write data to its
'outfile' parameter.
2017-10-09 16:17:02 +01:00
Brendan Jackman
016d68bfa0 PluginCache: Improve error message for ill-formed plugin config
Currently if you get confused and write a config with something like:

energy_measurement:
  acme_cape

Then you get an error when we try to 'iteritems' on the value
'acme_cape'. Instead, explicitly check for this case.
2017-10-06 18:00:38 +01:00
Sergei Trofimov
287ab8d565 instrumentation/misc: fix sysfile_getter/cpufreq
- Fix start/stop methods of sysfile_getter, which where still using the
  WA2 mechanic of prefixing "slow_" to the name to indicate priority,
  and consequently weren't being invoked.
- Call super inside cpufreq's valide(() to make sure tempfs path is set.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
5ecc4703e4 framework/execution: fix TargetManager termination
- Make sure TargetManager.finalize() actually gets called at the end
  of the run.
- Overrule the "diconnect" parameter behavior for gem5 and make sure it
  always disconnects. This necessary for stats to be generated properly.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
05e90bd623 instrumentation/misc: fix interrupts instrument
When porting from WA2 interrupt instrument retained references to
"device" instead of "target".
2017-10-06 15:04:02 +01:00
Sergei Trofimov
0d54c5f070 workloads: add memcpy workload
Ported memcpy workload from WA2.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
c93b1a0939 utils/log: adjust level for CGgroups logger
devlib's CGgroups module logs things at INFO level. The information it
logs is too detailed for what WA considers "INFO", and should be logged
at DEBUG level instead.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
c89e249732 framework/target: add ability to pass additional platform params.
Gem5Platform requires a host output directory as one if it's
instantiation parameters. This is not something we want to expose a
configuration parameter to the user, as for WA, the standard output
directory ought to be used.

Up to this point, WA's target instatiation process assumed that all
parameters came from the user, and there was no way for WA itself to set
them. This commit adds extra_platform_parms argument to
instantiate_target, to remedi this.

extra_platform_parms is then used to set the host output directory for
gem5 appropriately.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
55a72002ca framework/target: fix gem5 target descriptor
(linux|android)_gem5 devices were being described as having gem5
platforms but standard linux/android connections. This commit fixes this
by making sure that the gem5 connection is used.
2017-10-06 15:04:02 +01:00
setrofim
12edabf753 Merge pull request #501 from bjackman/acme-multiple-instruments2
energy_measurement: Allow multiple IIO devices for ACME
2017-10-06 15:01:31 +01:00
setrofim
ae1363d573 Merge pull request #505 from bjackman/geekbench-fixes
geekbench: Fix use of WA3 APIs
2017-10-06 14:55:37 +01:00
setrofim
ad08cbfe37 Merge pull request #504 from setrofim/energy-backend-param-fix
framework/target: fix runtime param name resolution
2017-10-06 14:54:39 +01:00
Brendan Jackman
6d6fc25e40 geekbench: Fix use of WA3 APIs
Seem I cherry-picked the geekbench workload from my WIP branch
without testing properly, and missed a fixup commit. Here's the
content of that commit, which fixes some missing updates from WA2 ->
WA3 APIs.
2017-10-06 14:51:50 +01:00
Brendan Jackman
e672ea3faf energy_measurement: Allow multiple IIO devices for ACME
Devlib's ACME instrument uses iio-capture under the hood, which can
only capture data from one IIO device at a time. Devlib's instrument
API expects to produce a single CSV file for the Instrument, with a
single axis of sample timestamps. These two things cannot be
correctly reconciled without changing the devlib Instrument API -
get_data would need to be able to return two distinct sets of data.

Instead, where required WA will instantiate the ACME instrument
multiple times (once for each IIO device), producing two separate CSV
files. Aggregated energy info (as opposed to timestamped samples of
energy data) _can_ be meaningfully combined from multiple IIO
devices, so the derived stats are summed.

This requires altering the internal API of the energy_measurement
module - rather than a single instrument the super-instrument now has
a dictionary mapping 'device' keys (this is really an arbitrary key,
but I think that 'device' is a justifiable metaphor for whatever
might separate multiple instruments, plus it is the terminology used
by IIO/ACME) to Instrument objects.  Existing instruments simply
return a dict with a single entry, with None as the key.

It may in future be possible to solve this problem in devlib's ACME
instrument by avoiding the use of the iio-capture tool. In that case,
it should be possible to basically revert this patch and get the
required funcitonality from devlib.
2017-10-06 14:43:54 +01:00
Sergei Trofimov
bce66f4388 framework/target: fix runtime param name resolution
Runtime param names from config were matched against names available on
for the target using an overly-loose regex that resulted in the wrong
config point being returned.

Use caseless_string matching instead.
2017-10-06 14:21:23 +01:00
setrofim
1fe3ed9f96 Merge pull request #503 from setrofim/energy-backend-param-fix
energy instrument backends parameter validation fix
2017-10-06 13:56:46 +01:00
Sergei Trofimov
ce3064dd6f instrument/energy: correct backend param init
As paramters defined by energy_measurement instrument backends were not
used in instantiating the backends, but the underlying devlib
instrument, the values did not undergo the usual Plugin parameter
setting perocedure. In particuar, type conversion would not take place.

Make sure backend parameters are validated properly by setting them on
an obj_dict before passing them  to the devlib instrument.
2017-10-06 13:55:16 +01:00
Sergei Trofimov
109490ec3e utils/types: make list_of classes serializable 2017-10-06 12:55:25 +01:00
Sergei Trofimov
7d9b6d6dd7 framework/config: ensure correct type for param defaults
Call self.kind() on the default value of a parameter before assigning it
to make sure that the assigned value is of correct type.
2017-10-06 12:52:57 +01:00
Sergei Trofimov
6e0cb73a4e framework/config: make LoggingConfig serializable 2017-10-06 12:52:12 +01:00
setrofim
cd328c9a50 Merge pull request #502 from bjackman/youtube-playback
youtube_playback: Add youtube_playback workload
2017-10-06 10:33:32 +01:00
Brendan Jackman
b18db64be1 youtube_playback: Add youtube_playback workload
This is a very simple workload that just aims to allow collecting
instrumentation data on purely the video decoding aspect of Youtube.
2017-10-06 10:31:41 +01:00
setrofim
bec5b0982e Merge pull request #499 from bjackman/next
target/descriptor: Fix typo for 'telnet' param
2017-10-05 13:28:25 +01:00
Brendan Jackman
f767714cd4 target/descriptor: Fix typo for 'telnet' param 2017-10-05 13:27:08 +01:00
setrofim
85d23421b9 Merge pull request #497 from bjackman/phone-home
Add feature to mark workloads unsafe for confidential devices
2017-10-04 15:57:41 +01:00
setrofim
24d96c2397 Merge pull request #496 from bjackman/docstring-tweaks
Misc clarifications/cleanups
2017-10-04 15:56:58 +01:00
Brendan Jackman
73d45b69a5 geekbench: Mark as a workload that may phone home 2017-10-04 15:43:03 +01:00
Brendan Jackman
06b451d31b framework: Add feature to mark workloads unsafe for confidential devices
Some workloads, such as Geekbench, may phone home and report data about the
device they are running on. This poses a risk for users that are testing on
unreleased or otherwise confidential devices - perhaps they use a standard
agenda to run a large battery of tests, in which case they may forget to disable
these dangerous workloads.

This provides a mechanism to prevent running those workloads from running by
setting allow_phone_home=False in the user configuration.
2017-10-04 15:43:03 +01:00
Brendan Jackman
90e4cace06 target/descriptor: Add 'to false' to load_default_modules description 2017-10-04 15:42:18 +01:00
setrofim
5166f3e387 Merge pull request #495 from bjackman/geekbench-package-names
geekbench: Add missing package_names attribute
2017-10-04 13:49:34 +01:00
Brendan Jackman
188e5d752f framework/execution: Whitespace 2017-10-04 13:35:20 +01:00
Brendan Jackman
c4fe5b66e5 target/descriptor: Fix 'devlib' typo 2017-10-04 13:30:12 +01:00
Brendan Jackman
24dc8b2a65 target/descriptor: whitespace 2017-10-04 13:30:10 +01:00
Brendan Jackman
43f5cd793a PluginCache: Check plugin name before trying to iteritems
Currently if your config contains

typo_for_a_global_alias: 1

You will get an error where we try to call 'iteritems' on the value 1. This
commit re-orders the checks so that you instead get an error for the
unrecognised 'typo_for_a_global_alias'.
2017-10-04 13:28:53 +01:00
Brendan Jackman
8daae0752e PluginCache: whitespace 2017-10-04 13:28:52 +01:00
Brendan Jackman
ba01d011c5 geekbench: Add missing package_names attribute 2017-10-04 13:27:57 +01:00
setrofim
1bd19d9af3 Merge pull request #494 from marcbonnici/uiautomator
Uiautomator
2017-10-03 11:20:18 +01:00
Marc Bonnici
049530fe41 Youtube: Removes reference to UxPerf 2017-10-03 11:12:22 +01:00
Marc Bonnici
52b84b4030 Vellamo: Updated to automatically generate PackageID
Now generates packageID instead of using hardcoded values and removed
unused reference to UxPerf.
2017-10-03 11:12:22 +01:00
Marc Bonnici
3b7af8e52c Uiautomator: Adds missing support functionality
Adds initial implementation of ActionLogger class and adds method to
BaseUiAutomation to automatically generate packageID.
2017-10-03 11:12:22 +01:00
setrofim
a876d6b79a Merge pull request #493 from bjackman/geekbench-wa3-proper
Port Geekbench workload from WA2
2017-10-02 16:39:47 +01:00
Brendan Jackman
e4965096e6 geekbench: Error early if target isn't rooted 2017-10-02 16:24:26 +01:00
Brendan Jackman
84404da471 geekbench: Port Geekbench workload from WA2
Differences from original:
- Standard renaming and rearrangement of imports
- UiAutomation.java now uses the setup() and runWorkload() phases which weren't
  present before.
2017-10-02 16:24:23 +01:00