1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 10:51:13 +01:00
Commit Graph

163 Commits

Author SHA1 Message Date
Sergei Trofimov
ecd0dbd55b Add templerun2 workload 2017-09-27 10:34:50 +01:00
Sergei Trofimov
9d7f998511 framework/instrumentation: handle non-job errors in ManagedCallback
If an error occurs in a ManagedCallback that is invoked outside of a
job, re-raise rather than attempting to update the status of the
non-existent job.
2017-09-27 10:34:50 +01:00
Sergei Trofimov
0c40cdae18 tools/revent: improve recording + add prebuilts
The begining and end of recording timestamps are now stored as part of
the recording. This allows to correctly replay recordings where the
first event occurs some time after the recording started, or the last
event some time before recording ended.

Add pre-built revent binaries for the armeabi and arm64 architectures.
2017-09-27 10:34:00 +01:00
Sergei Trofimov
0934037e1b revent fixes
- do not attempt to replay setup if a setup recording has not been
  provided (perviously, setup was mandatory).
- update the apk initialization to the correct method inside the record
  command.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
335ed8198c instumentation/energy_management: update import to match devlib change 2017-09-27 10:32:41 +01:00
Sergei Trofimov
261417a9db framework/configuration: fix workload config with multiple sections
When an agenda contains multiple sections, the same workload entry might
be used in construction of multiple job specs. Job spec construction may
mangle the workload entry. To prevent this from impacting other
jobs, use a deep copy of the workload entry when constructing a job
spec.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
c96181bed7 framework/configruation: fix mandatory workload parameters
PluginCache.get_plugin_config assumes that no more configuration is to
be processed, and therefore config is final. As such, it is validating
that mandatory parameters are set. This assumption is invalid for
workload_parameters, however, as those need to be resolved on per-spec
basis, and cannot be globally cached.

This commit adds a prameter for get_plugin_config that indicates whether
or not it should consider the config to be final.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
aec89a077e framework/configuration: fix connection_settings configuration
Connection settings were not being properly extracted from device_config
and not used. Meaning it was impossible to e.g. specify the adb device
to use for a run.

This ensures that connection settings are extracted, validated, and used
properly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
04d5ec4277 framework/run: fix JobState deserialization
At some point, JobState __init__ was changed to take the iteration
number, but its from_pod was not updated accordingly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
7c7ffe3e77 framework/run: fix job status handling.
- Re-order Status entries so that higher severity entries have higher
  enum values.
- Add set_status() to Job that ensures that a status is only set if it
  is of higher severity (e.g. a Job that has been marked as PARTIAL by
  an instrument will not be overwritten as OK by the runner).
- Retry no generates a new job, rather than re-enqueuing the existing
  object; this ensures that the output status is tracked properly.
- Adjust ManagedCallback to set set job status to FAILED if it sees a
  WorkloadError, and to PARTIAL other wise. The idea being that
  instruments raise WorkloadError if they have a reason to believe
  workload did not execute properly and indicated failure even if the
  workload itself has failed to detect it (e.g. FPS instrument detecting
  crashed content, where the workload might lack any feedback regarding
  the crash). Other errors would indicate an issue with the instrument
  itself, and so the job is marked as PARTIAL, as there is no reason to
  suspect that the workload is at fault and the other results generated
  for this execution may be valid.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
fe53efcd49 framework/workload: ApkWorkload consistent package handling
Rename "package" parameter to "package_name" (to be cosnsitent with
package_names class attribute). "package" is now a property containting
the name of the actual package used after the APK is resolved.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
1e233d2104 framework/job: track run time.
Job now tracks how long it took to run the workload and save the
timedelta value in the run_time attribute.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
2ef4074c8b framework/execution: ExecutionContext enhancements
- Add workload property as the shortcut for accessing the workoad for
  the current job.
- Add set_status method that setts the status of the current job.
- Add get_metric method that searches for a metric in the output for the
  current job, or failing that, in the run output.
2017-09-27 10:32:19 +01:00
Sergei Trofimov
82b0b238c2 framework/output: add accessors for metrics and artifacts
- Add get_metric methods to Result and Output
- Add metrics and artifacts properties to Output
2017-09-27 10:32:19 +01:00
Sergei Trofimov
9b6d4d9fa6 utils/revent: stop recording fix.
It seems that occasionally an inclomplete recording file is pulled. Add
a one-second (target time) delay to allow SIGINT to be properly handled
and recording to be updated, followed by a sync to ensure the file has
been written.
2017-09-27 10:32:19 +01:00
Sergei Trofimov
c29b1c8b23 utils/log: log_error marks Exception as logged.
Mark the Exception is logged inside log_error, to prevent it form being
logged repeatedly if re-raised.
2017-09-27 10:32:19 +01:00
setrofim
67eca23ef7 Merge pull request #479 from bjackman/homescreen
Homescreen: Port homescreen workload to WA3
2017-09-15 21:47:12 +01:00
setrofim
5063ed6907 Merge pull request #480 from bjackman/remove-cpufreq-interactive
trace-cmd: remove cpufreq_interactive from default events
2017-09-15 21:45:06 +01:00
Brendan Jackman
4063ea0ba8 trace-cmd: remove cpufreq_interactive from default events
The interactive governor isn't standard any more (and was
Android-only anyway). Remove this default so you don't get errors for
kernels that don't support it.
2017-09-15 18:05:17 +01:00
Brendan Jackman
a79fb83fd1 framework/workload: Fix resolve_packge_from_target
This currently raises an error undonditionally when self.package is set, fix the
indentation so it only does that if it isn't installed on the target.
2017-09-15 18:04:11 +01:00
Sergei Trofimov
df01d8aa22 Homescreen: Port homescreen workload to WA3
Just did:  wlauto      -> wa
           self.device -> self.target
2017-09-15 18:03:24 +01:00
setrofim
0cc8ebbf69 Merge pull request #469 from marcbonnici/energy_measurement
Energy measurement
2017-08-21 08:50:28 +01:00
Marc Bonnici
bd126a430d Instruments/EnergyMeas: Adds support for DerivedMeasurement from devlib
Delvib now is capable of performing postprocessing of MeasurementCSV
files, instead of calculating additional metrics in WA this will be
performed externally. Currently support has been added for calculating
average power and cumulative energy.
2017-08-18 09:53:27 +01:00
Marc Bonnici
0f506dde17 RTConfig: Added support for specifying device initial screen state. 2017-08-16 17:35:19 +01:00
Marc Bonnici
51464165c1 RTConfig: Added android runtime config
Added a runtime config for android specific settings, currently
supported are screen brightness, rotation and airplane mode.
2017-08-16 17:35:19 +01:00
Marc Bonnici
c5b1caa276 Utils/Enum: Adds a __str__ method to display enum levels 2017-08-16 17:35:19 +01:00
Marc Bonnici
2cc60b77cb Utils/Serializer: Adds serializer support for level type 2017-08-16 11:10:46 +01:00
Marc Bonnici
5f919445d9 EnergyMeaseurement: Adds support for acme_cape 2017-08-11 10:30:19 +01:00
Marc Bonnici
b98b74f0be EnergyMeasurement: Removes parameter defaults
The instrument baseclass expects None for not defined parameters
instead of empty containers.
2017-08-11 10:30:19 +01:00
Marc Bonnici
de3301312f RTParams: Adds support for using default values for RT parameters 2017-08-09 16:01:39 +01:00
Marc Bonnici
bbfb802c00 RuntimeConfig: Makes setter_params optional
Now setter_params are optional when creating a Runtime Parameter to
allow them to be used for more purposes which don't require additional
settings.
2017-08-09 16:01:39 +01:00
Marc Bonnici
06be73f8e1 RuntimeConfig: Fixed typo 2017-08-09 16:01:39 +01:00
Sergei Trofimov
3f87f3ab07 utils/misc: remove enum_metaclass
It is unused and has been obsoleted by enum type in utils/types.
2017-08-09 15:59:20 +01:00
Sergei Trofimov
a9959550af utils/types: better enum class member setting
- What used to be enum.values is now enum.levels.
- Add enum.names and enum.values that are lists of enum's levels' names
  and values respectively.
- Add a check on creation to make sure that provided level names do not
  conflict with the atomatically created members.
2017-08-09 15:59:20 +01:00
Sergei Trofimov
5e0ff6aa51 cpufreq: rename get_domain_cpus
get_domain_cpus() got renamed to get_related_cpus() in devlib to reflect
the cpufreq nomenclature. This commit makes corresponding changes in WA.
2017-07-26 14:58:46 +01:00
setrofim
531dcfbb1b Merge pull request #447 from marcbonnici/energy_measurement
Energy measurement
2017-07-25 16:22:18 +01:00
Marc Bonnici
c455ad524f Parser: Fixes raising duplicate entry error. 2017-07-25 16:08:17 +01:00
Marc Bonnici
af49c5020f Instrumentation/Misc: Fixed using 'target' instead of 'device' 2017-07-25 16:08:17 +01:00
Marc Bonnici
e9a9f16032 EnergyMeasurement: Adds a wrapper for devlib instrumentation
The instrument allows for various devlib instruments (currently only
daq and energy_probe) to be used in WA to collect energy measurements.
2017-07-25 16:08:17 +01:00
Marc Bonnici
f8e4d34e60 APKResolution: Uses loose version matching
Allows for more flexible version matching  e.g. specifying a
version of 4, 4.3, 4.3.1 will all resolve to an apk version of 4.3.1
2017-07-20 14:30:52 +01:00
Marc Bonnici
c722a6a73c ApkResolution: Now takes into account apk abi when resolving.
Previously any apk found would be selected even if the abi did not match
the device, now an apk is only selected if it is compatible with the
device. Additionally the 'exact_abi' parameter has been added to allow only
selecting an apk if a devices primary abi's native code is present in
the apk, or there is no native code.
2017-07-20 14:30:52 +01:00
Marc Bonnici
5ef3d7673f Getters: Fixes raising error instead of returning. 2017-07-14 18:09:01 +01:00
Marc Bonnici
d76c78f3f6 trace-cmd: documentation fixes.
- Remove reference to default events from the overall workload
  documentation. It was, as of recently, outdated, and was also
  redundant, as the actual defaults will be in the parameter-specific
  documentation.
- Remove reference to Android-specific trace-cmd binary -- this was not
  true for a long time.
- Clarify that the on-host trace-cmd binary is now optional due to the
  report_on_target config.
2017-07-11 15:33:21 +01:00
Marc Bonnici
3382036d6e Vellamo: Ported workload to WA3 2017-07-10 11:18:32 +01:00
Marc Bonnici
b6cc845272 Youtube: Adds workload to WA3 2017-07-10 11:18:32 +01:00
Marc Bonnici
d015cce91e BenchmarkPi: Updated for uiauto2 refactor 2017-07-10 11:08:44 +01:00
Marc Bonnici
9284472d8b BaseUiAutomator: Ports additional functionality
Ports required functionality from WA2 for new workloads.
2017-07-10 11:08:44 +01:00
Marc Bonnici
d518328334 Uiauto: Removed reference to wlauto 2017-07-10 11:08:44 +01:00
Marc Bonnici
f9bc0261bf AndroidWorkload: Adds support for apk replacement/downgrading
Now tries to replace and downgrade an apk if already installed.
2017-07-10 11:08:44 +01:00
Marc Bonnici
4d397ab465 AndroidWorkload: Uninstall uiauto apks before reinstalling
Due to the fact that uiauto apk files built on different machines will
be signed with different keys, adb will fail to overwrite a previous
version even when set to replace. This commit now will uninstall the
previous uiauto apk file if present before attempting to install the new
version.
2017-07-10 11:08:43 +01:00