1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

125 Commits

Author SHA1 Message Date
setrofim
c3438b6814 Merge pull request #500 from setrofim/next
memcopy workload + fix gem5 support.
2017-10-10 08:58:24 +01:00
setrofim
590309b6be Merge pull request #508 from bjackman/instrumentation-typo
instrumentation: Fix typo
2017-10-10 08:40:25 +01:00
Brendan Jackman
049e2df0dc instrumentation: Fix typo 2017-10-09 18:26:41 +01:00
Brendan Jackman
016d68bfa0 PluginCache: Improve error message for ill-formed plugin config
Currently if you get confused and write a config with something like:

energy_measurement:
  acme_cape

Then you get an error when we try to 'iteritems' on the value
'acme_cape'. Instead, explicitly check for this case.
2017-10-06 18:00:38 +01:00
Sergei Trofimov
5ecc4703e4 framework/execution: fix TargetManager termination
- Make sure TargetManager.finalize() actually gets called at the end
  of the run.
- Overrule the "diconnect" parameter behavior for gem5 and make sure it
  always disconnects. This necessary for stats to be generated properly.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
c89e249732 framework/target: add ability to pass additional platform params.
Gem5Platform requires a host output directory as one if it's
instantiation parameters. This is not something we want to expose a
configuration parameter to the user, as for WA, the standard output
directory ought to be used.

Up to this point, WA's target instatiation process assumed that all
parameters came from the user, and there was no way for WA itself to set
them. This commit adds extra_platform_parms argument to
instantiate_target, to remedi this.

extra_platform_parms is then used to set the host output directory for
gem5 appropriately.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
55a72002ca framework/target: fix gem5 target descriptor
(linux|android)_gem5 devices were being described as having gem5
platforms but standard linux/android connections. This commit fixes this
by making sure that the gem5 connection is used.
2017-10-06 15:04:02 +01:00
Sergei Trofimov
bce66f4388 framework/target: fix runtime param name resolution
Runtime param names from config were matched against names available on
for the target using an overly-loose regex that resulted in the wrong
config point being returned.

Use caseless_string matching instead.
2017-10-06 14:21:23 +01:00
Sergei Trofimov
7d9b6d6dd7 framework/config: ensure correct type for param defaults
Call self.kind() on the default value of a parameter before assigning it
to make sure that the assigned value is of correct type.
2017-10-06 12:52:57 +01:00
Sergei Trofimov
6e0cb73a4e framework/config: make LoggingConfig serializable 2017-10-06 12:52:12 +01:00
Brendan Jackman
f767714cd4 target/descriptor: Fix typo for 'telnet' param 2017-10-05 13:27:08 +01:00
setrofim
85d23421b9 Merge pull request #497 from bjackman/phone-home
Add feature to mark workloads unsafe for confidential devices
2017-10-04 15:57:41 +01:00
Brendan Jackman
06b451d31b framework: Add feature to mark workloads unsafe for confidential devices
Some workloads, such as Geekbench, may phone home and report data about the
device they are running on. This poses a risk for users that are testing on
unreleased or otherwise confidential devices - perhaps they use a standard
agenda to run a large battery of tests, in which case they may forget to disable
these dangerous workloads.

This provides a mechanism to prevent running those workloads from running by
setting allow_phone_home=False in the user configuration.
2017-10-04 15:43:03 +01:00
Brendan Jackman
90e4cace06 target/descriptor: Add 'to false' to load_default_modules description 2017-10-04 15:42:18 +01:00
Brendan Jackman
188e5d752f framework/execution: Whitespace 2017-10-04 13:35:20 +01:00
Brendan Jackman
c4fe5b66e5 target/descriptor: Fix 'devlib' typo 2017-10-04 13:30:12 +01:00
Brendan Jackman
24dc8b2a65 target/descriptor: whitespace 2017-10-04 13:30:10 +01:00
Brendan Jackman
43f5cd793a PluginCache: Check plugin name before trying to iteritems
Currently if your config contains

typo_for_a_global_alias: 1

You will get an error where we try to call 'iteritems' on the value 1. This
commit re-orders the checks so that you instead get an error for the
unrecognised 'typo_for_a_global_alias'.
2017-10-04 13:28:53 +01:00
Brendan Jackman
8daae0752e PluginCache: whitespace 2017-10-04 13:28:52 +01:00
Marc Bonnici
3b7af8e52c Uiautomator: Adds missing support functionality
Adds initial implementation of ActionLogger class and adds method to
BaseUiAutomation to automatically generate packageID.
2017-10-03 11:12:22 +01:00
Brendan Jackman
e5f13076ac BaseUiAutomation: Port uiDeviceSwipe methods from WA2 2017-10-02 16:17:36 +01:00
Sergei Trofimov
9cb7ea862c framework/workload: default view for ApkWorkload
- Add "activity" property to PackageManager exposing the underlying
  apk_info attribute.
- Add "view" class attribute to ApkWorkload, initialized to None
- If "view" is not set for a particular workload, set a default based on
  the package and activity of the underlying workload.
2017-09-29 14:22:22 +01:00
Sergei Trofimov
9d7f998511 framework/instrumentation: handle non-job errors in ManagedCallback
If an error occurs in a ManagedCallback that is invoked outside of a
job, re-raise rather than attempting to update the status of the
non-existent job.
2017-09-27 10:34:50 +01:00
Sergei Trofimov
0934037e1b revent fixes
- do not attempt to replay setup if a setup recording has not been
  provided (perviously, setup was mandatory).
- update the apk initialization to the correct method inside the record
  command.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
261417a9db framework/configuration: fix workload config with multiple sections
When an agenda contains multiple sections, the same workload entry might
be used in construction of multiple job specs. Job spec construction may
mangle the workload entry. To prevent this from impacting other
jobs, use a deep copy of the workload entry when constructing a job
spec.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
c96181bed7 framework/configruation: fix mandatory workload parameters
PluginCache.get_plugin_config assumes that no more configuration is to
be processed, and therefore config is final. As such, it is validating
that mandatory parameters are set. This assumption is invalid for
workload_parameters, however, as those need to be resolved on per-spec
basis, and cannot be globally cached.

This commit adds a prameter for get_plugin_config that indicates whether
or not it should consider the config to be final.
2017-09-27 10:32:41 +01:00
Sergei Trofimov
aec89a077e framework/configuration: fix connection_settings configuration
Connection settings were not being properly extracted from device_config
and not used. Meaning it was impossible to e.g. specify the adb device
to use for a run.

This ensures that connection settings are extracted, validated, and used
properly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
04d5ec4277 framework/run: fix JobState deserialization
At some point, JobState __init__ was changed to take the iteration
number, but its from_pod was not updated accordingly.
2017-09-27 10:32:31 +01:00
Sergei Trofimov
7c7ffe3e77 framework/run: fix job status handling.
- Re-order Status entries so that higher severity entries have higher
  enum values.
- Add set_status() to Job that ensures that a status is only set if it
  is of higher severity (e.g. a Job that has been marked as PARTIAL by
  an instrument will not be overwritten as OK by the runner).
- Retry no generates a new job, rather than re-enqueuing the existing
  object; this ensures that the output status is tracked properly.
- Adjust ManagedCallback to set set job status to FAILED if it sees a
  WorkloadError, and to PARTIAL other wise. The idea being that
  instruments raise WorkloadError if they have a reason to believe
  workload did not execute properly and indicated failure even if the
  workload itself has failed to detect it (e.g. FPS instrument detecting
  crashed content, where the workload might lack any feedback regarding
  the crash). Other errors would indicate an issue with the instrument
  itself, and so the job is marked as PARTIAL, as there is no reason to
  suspect that the workload is at fault and the other results generated
  for this execution may be valid.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
fe53efcd49 framework/workload: ApkWorkload consistent package handling
Rename "package" parameter to "package_name" (to be cosnsitent with
package_names class attribute). "package" is now a property containting
the name of the actual package used after the APK is resolved.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
1e233d2104 framework/job: track run time.
Job now tracks how long it took to run the workload and save the
timedelta value in the run_time attribute.
2017-09-27 10:32:20 +01:00
Sergei Trofimov
2ef4074c8b framework/execution: ExecutionContext enhancements
- Add workload property as the shortcut for accessing the workoad for
  the current job.
- Add set_status method that setts the status of the current job.
- Add get_metric method that searches for a metric in the output for the
  current job, or failing that, in the run output.
2017-09-27 10:32:19 +01:00
Sergei Trofimov
82b0b238c2 framework/output: add accessors for metrics and artifacts
- Add get_metric methods to Result and Output
- Add metrics and artifacts properties to Output
2017-09-27 10:32:19 +01:00
Brendan Jackman
a79fb83fd1 framework/workload: Fix resolve_packge_from_target
This currently raises an error undonditionally when self.package is set, fix the
indentation so it only does that if it isn't installed on the target.
2017-09-15 18:04:11 +01:00
Marc Bonnici
0f506dde17 RTConfig: Added support for specifying device initial screen state. 2017-08-16 17:35:19 +01:00
Marc Bonnici
51464165c1 RTConfig: Added android runtime config
Added a runtime config for android specific settings, currently
supported are screen brightness, rotation and airplane mode.
2017-08-16 17:35:19 +01:00
Marc Bonnici
de3301312f RTParams: Adds support for using default values for RT parameters 2017-08-09 16:01:39 +01:00
Marc Bonnici
bbfb802c00 RuntimeConfig: Makes setter_params optional
Now setter_params are optional when creating a Runtime Parameter to
allow them to be used for more purposes which don't require additional
settings.
2017-08-09 16:01:39 +01:00
Marc Bonnici
06be73f8e1 RuntimeConfig: Fixed typo 2017-08-09 16:01:39 +01:00
Sergei Trofimov
a9959550af utils/types: better enum class member setting
- What used to be enum.values is now enum.levels.
- Add enum.names and enum.values that are lists of enum's levels' names
  and values respectively.
- Add a check on creation to make sure that provided level names do not
  conflict with the atomatically created members.
2017-08-09 15:59:20 +01:00
Sergei Trofimov
5e0ff6aa51 cpufreq: rename get_domain_cpus
get_domain_cpus() got renamed to get_related_cpus() in devlib to reflect
the cpufreq nomenclature. This commit makes corresponding changes in WA.
2017-07-26 14:58:46 +01:00
setrofim
531dcfbb1b Merge pull request #447 from marcbonnici/energy_measurement
Energy measurement
2017-07-25 16:22:18 +01:00
Marc Bonnici
c455ad524f Parser: Fixes raising duplicate entry error. 2017-07-25 16:08:17 +01:00
Marc Bonnici
f8e4d34e60 APKResolution: Uses loose version matching
Allows for more flexible version matching  e.g. specifying a
version of 4, 4.3, 4.3.1 will all resolve to an apk version of 4.3.1
2017-07-20 14:30:52 +01:00
Marc Bonnici
c722a6a73c ApkResolution: Now takes into account apk abi when resolving.
Previously any apk found would be selected even if the abi did not match
the device, now an apk is only selected if it is compatible with the
device. Additionally the 'exact_abi' parameter has been added to allow only
selecting an apk if a devices primary abi's native code is present in
the apk, or there is no native code.
2017-07-20 14:30:52 +01:00
Marc Bonnici
5ef3d7673f Getters: Fixes raising error instead of returning. 2017-07-14 18:09:01 +01:00
Marc Bonnici
9284472d8b BaseUiAutomator: Ports additional functionality
Ports required functionality from WA2 for new workloads.
2017-07-10 11:08:44 +01:00
Marc Bonnici
d518328334 Uiauto: Removed reference to wlauto 2017-07-10 11:08:44 +01:00
Marc Bonnici
f9bc0261bf AndroidWorkload: Adds support for apk replacement/downgrading
Now tries to replace and downgrade an apk if already installed.
2017-07-10 11:08:44 +01:00
Marc Bonnici
4d397ab465 AndroidWorkload: Uninstall uiauto apks before reinstalling
Due to the fact that uiauto apk files built on different machines will
be signed with different keys, adb will fail to overwrite a previous
version even when set to replace. This commit now will uninstall the
previous uiauto apk file if present before attempting to install the new
version.
2017-07-10 11:08:43 +01:00