1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

835 Commits

Author SHA1 Message Date
Brendan Jackman
90e4cace06 target/descriptor: Add 'to false' to load_default_modules description 2017-10-04 15:42:18 +01:00
Brendan Jackman
c4fe5b66e5 target/descriptor: Fix 'devlib' typo 2017-10-04 13:30:12 +01:00
Brendan Jackman
24dc8b2a65 target/descriptor: whitespace 2017-10-04 13:30:10 +01:00
Brendan Jackman
43f5cd793a PluginCache: Check plugin name before trying to iteritems
Currently if your config contains

typo_for_a_global_alias: 1

You will get an error where we try to call 'iteritems' on the value 1. This
commit re-orders the checks so that you instead get an error for the
unrecognised 'typo_for_a_global_alias'.
2017-10-04 13:28:53 +01:00
Brendan Jackman
8daae0752e PluginCache: whitespace 2017-10-04 13:28:52 +01:00
setrofim
1bd19d9af3 Merge pull request #494 from marcbonnici/uiautomator
Uiautomator
2017-10-03 11:20:18 +01:00
Marc Bonnici
049530fe41 Youtube: Removes reference to UxPerf 2017-10-03 11:12:22 +01:00
Marc Bonnici
52b84b4030 Vellamo: Updated to automatically generate PackageID
Now generates packageID instead of using hardcoded values and removed
unused reference to UxPerf.
2017-10-03 11:12:22 +01:00
Marc Bonnici
3b7af8e52c Uiautomator: Adds missing support functionality
Adds initial implementation of ActionLogger class and adds method to
BaseUiAutomation to automatically generate packageID.
2017-10-03 11:12:22 +01:00
setrofim
a876d6b79a Merge pull request #493 from bjackman/geekbench-wa3-proper
Port Geekbench workload from WA2
2017-10-02 16:39:47 +01:00
Brendan Jackman
e4965096e6 geekbench: Error early if target isn't rooted 2017-10-02 16:24:26 +01:00
Brendan Jackman
84404da471 geekbench: Port Geekbench workload from WA2
Differences from original:
- Standard renaming and rearrangement of imports
- UiAutomation.java now uses the setup() and runWorkload() phases which weren't
  present before.
2017-10-02 16:24:23 +01:00
Brendan Jackman
e5f13076ac BaseUiAutomation: Port uiDeviceSwipe methods from WA2 2017-10-02 16:17:36 +01:00
Sergei Trofimov
a65cffb705 trace-cmd: add trace files as artificats
The binary and text trace files were not being added as artifacts in WA
output.
2017-10-02 13:55:58 +01:00
setrofim
66ba28e625 Merge pull request #492 from bjackman/trace-thermal-default
trace-cmd: Enable thermal trace events by default
2017-10-02 13:41:57 +01:00
Brendan Jackman
1eac74fa8f trace-cmd: Enable thermal trace events by default
Due to the nature of modern Android devices, in order to usefully
interpret the power* events, you more and more often need to know the
kernel's view of temperature too. Therefore I think enabling the
thermal event group by default makes sense.
2017-10-02 13:29:10 +01:00
setrofim
d03d950b8e Merge pull request #488 from setrofim/next-fps
Add FPS instrument
2017-09-29 15:33:21 +01:00
Sergei Trofimov
9cb7ea862c framework/workload: default view for ApkWorkload
- Add "activity" property to PackageManager exposing the underlying
  apk_info attribute.
- Add "view" class attribute to ApkWorkload, initialized to None
- If "view" is not set for a particular workload, set a default based on
  the package and activity of the underlying workload.
2017-09-29 14:22:22 +01:00
Sergei Trofimov
4e6ba64737 instrumentation: add FPS instrument.
Add an instrument to collect FPS (frames per second) and associated
rendering statics using corresponding devlib functionality.
2017-09-29 13:50:23 +01:00
setrofim
241b4a5a3a Merge pull request #490 from bjackman/import-apkworkload
wa: Add ApkWorkload to default imports
2017-09-29 13:49:58 +01:00
Sergei Trofimov
19f9998c2e tools/revent: further fixes
- Ensure that CLOCK_MONOTONIC is used for all timestamps, so that the
  timestamps collected by revent and those reported in the struct
  input_event wouild match.

- Do not attempt to free events array inside fini_revent_recroding() if
  no events have been recorded.

- Add missing \n in some debug messages.

- Add end of recording debug message.
2017-09-29 09:48:25 +01:00
Sergei Trofimov
28e339d3b2 tools/revent: timestamp adjustment fixes
- Make sure start/end timestamps are properly adjusted when there are
  zero events.
- Rename adjust_event_times --> adjust_timestamps, as the latter better
  fits its current function.
- Add a couple of debug messages.
2017-09-28 15:11:28 +01:00
Brendan Jackman
14eba69229 wa: Add ApkWorkload to default imports 2017-09-28 13:09:28 +01:00
Sergei Trofimov
831515ae8d instrumentation/energy_measurement: update for DerivedMetrics
DerivedMeasurements in devlib was changed to return DerivedMetrics
instead of Measurements. The former do not have "channel" attribute, so
just pass its name to WA's metrics.
2017-09-28 13:06:33 +01:00
Sergei Trofimov
7b1cc6fa4e utils/serializer: add OrderedDict to POD_TYPES
Allow OrderedDict to be serialized in the same way as regular dicts by
defining it as a Plain Old Data type.
2017-09-28 10:46:03 +01:00
setrofim
1df033fc22 Merge pull request #487 from setrofim/next
Various fixes.
2017-09-27 14:01:21 +01:00
Sergei Trofimov
ecd0dbd55b Add templerun2 workload 2017-09-27 10:34:50 +01:00
Sergei Trofimov
9d7f998511 framework/instrumentation: handle non-job errors in ManagedCallback
If an error occurs in a ManagedCallback that is invoked outside of a
job, re-raise rather than attempting to update the status of the
non-existent job.
2017-09-27 10:34:50 +01:00
Sergei Trofimov
7cdb32c7f8 doc: update revent documentation
Forward-port the updated revent documentation from WA2, including
recording format description.
2017-09-27 10:34:48 +01:00
Sergei Trofimov
0c40cdae18 tools/revent: improve recording + add prebuilts
The begining and end of recording timestamps are now stored as part of
the recording. This allows to correctly replay recordings where the
first event occurs some time after the recording started, or the last
event some time before recording ended.

Add pre-built revent binaries for the armeabi and arm64 architectures.
2017-09-27 10:34:00 +01:00
Sergei Trofimov
0934037e1b revent fixes
- do not attempt to replay setup if a setup recording has not been
  provided (perviously, setup was mandatory).
- update the apk initialization to the correct method inside the record
  command.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
335ed8198c instumentation/energy_management: update import to match devlib change 2017-09-27 10:32:41 +01:00
Sergei Trofimov
261417a9db framework/configuration: fix workload config with multiple sections
When an agenda contains multiple sections, the same workload entry might
be used in construction of multiple job specs. Job spec construction may
mangle the workload entry. To prevent this from impacting other
jobs, use a deep copy of the workload entry when constructing a job
spec.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
c96181bed7 framework/configruation: fix mandatory workload parameters
PluginCache.get_plugin_config assumes that no more configuration is to
be processed, and therefore config is final. As such, it is validating
that mandatory parameters are set. This assumption is invalid for
workload_parameters, however, as those need to be resolved on per-spec
basis, and cannot be globally cached.

This commit adds a prameter for get_plugin_config that indicates whether
or not it should consider the config to be final.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
aec89a077e framework/configuration: fix connection_settings configuration
Connection settings were not being properly extracted from device_config
and not used. Meaning it was impossible to e.g. specify the adb device
to use for a run.

This ensures that connection settings are extracted, validated, and used
properly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
04d5ec4277 framework/run: fix JobState deserialization
At some point, JobState __init__ was changed to take the iteration
number, but its from_pod was not updated accordingly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
7c7ffe3e77 framework/run: fix job status handling.
- Re-order Status entries so that higher severity entries have higher
  enum values.
- Add set_status() to Job that ensures that a status is only set if it
  is of higher severity (e.g. a Job that has been marked as PARTIAL by
  an instrument will not be overwritten as OK by the runner).
- Retry no generates a new job, rather than re-enqueuing the existing
  object; this ensures that the output status is tracked properly.
- Adjust ManagedCallback to set set job status to FAILED if it sees a
  WorkloadError, and to PARTIAL other wise. The idea being that
  instruments raise WorkloadError if they have a reason to believe
  workload did not execute properly and indicated failure even if the
  workload itself has failed to detect it (e.g. FPS instrument detecting
  crashed content, where the workload might lack any feedback regarding
  the crash). Other errors would indicate an issue with the instrument
  itself, and so the job is marked as PARTIAL, as there is no reason to
  suspect that the workload is at fault and the other results generated
  for this execution may be valid.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
fe53efcd49 framework/workload: ApkWorkload consistent package handling
Rename "package" parameter to "package_name" (to be cosnsitent with
package_names class attribute). "package" is now a property containting
the name of the actual package used after the APK is resolved.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
1e233d2104 framework/job: track run time.
Job now tracks how long it took to run the workload and save the
timedelta value in the run_time attribute.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
2ef4074c8b framework/execution: ExecutionContext enhancements
- Add workload property as the shortcut for accessing the workoad for
  the current job.
- Add set_status method that setts the status of the current job.
- Add get_metric method that searches for a metric in the output for the
  current job, or failing that, in the run output.
2017-09-27 10:32:19 +01:00
Sergei Trofimov
82b0b238c2 framework/output: add accessors for metrics and artifacts
- Add get_metric methods to Result and Output
- Add metrics and artifacts properties to Output
2017-09-27 10:32:19 +01:00
Sergei Trofimov
9b6d4d9fa6 utils/revent: stop recording fix.
It seems that occasionally an inclomplete recording file is pulled. Add
a one-second (target time) delay to allow SIGINT to be properly handled
and recording to be updated, followed by a sync to ensure the file has
been written.
2017-09-27 10:32:19 +01:00
Sergei Trofimov
c29b1c8b23 utils/log: log_error marks Exception as logged.
Mark the Exception is logged inside log_error, to prevent it form being
logged repeatedly if re-raised.
2017-09-27 10:32:19 +01:00
setrofim
67eca23ef7 Merge pull request #479 from bjackman/homescreen
Homescreen: Port homescreen workload to WA3
2017-09-15 21:47:12 +01:00
setrofim
5063ed6907 Merge pull request #480 from bjackman/remove-cpufreq-interactive
trace-cmd: remove cpufreq_interactive from default events
2017-09-15 21:45:06 +01:00
setrofim
9de5714b91 Merge pull request #481 from bjackman/fix-resolve-package
framework/workload: Fix resolve_packge_from_target
2017-09-15 21:44:46 +01:00
Brendan Jackman
4063ea0ba8 trace-cmd: remove cpufreq_interactive from default events
The interactive governor isn't standard any more (and was
Android-only anyway). Remove this default so you don't get errors for
kernels that don't support it.
2017-09-15 18:05:17 +01:00
Brendan Jackman
a79fb83fd1 framework/workload: Fix resolve_packge_from_target
This currently raises an error undonditionally when self.package is set, fix the
indentation so it only does that if it isn't installed on the target.
2017-09-15 18:04:11 +01:00
Sergei Trofimov
df01d8aa22 Homescreen: Port homescreen workload to WA3
Just did:  wlauto      -> wa
           self.device -> self.target
2017-09-15 18:03:24 +01:00
setrofim
0cc8ebbf69 Merge pull request #469 from marcbonnici/energy_measurement
Energy measurement
2017-08-21 08:50:28 +01:00