1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 02:41:11 +01:00
Commit Graph

1146 Commits

Author SHA1 Message Date
Sergei Trofimov
a9329554d1 wa: fix exception exports 2018-03-08 11:18:12 +00:00
Sergei Trofimov
1b9ac9236a framework/exception: add ExecutionError
Add an exception class to represent an error from within the execution
framework (rather than a plugin).
2018-03-08 11:18:12 +00:00
Marc Bonnici
05b726db09 commands/create: Add support for creating packages to WA3 2018-03-08 10:36:29 +00:00
scott
b204e9c22a Antutu: New antutu workload for the 7.0.4 version 2018-03-08 10:17:13 +00:00
scott
7e0e7456a9 Androbench: Migrating to WA3 and modifying to extract results instead of taking a screenshot 2018-03-08 10:16:27 +00:00
scott
87541c647a Speedometer: Creating a wa3 speedometer workload 2018-03-08 10:16:02 +00:00
Sascha Bischoff
e82fa48826 rt-app: remove requirement for rooted device
We remove the requirement for rt-app to run on a rooted device as this
does not appear to be required. We make sure that we run the workload
from a directory where we have write permissions by first changing the
directory before executing the workload (in_directory argument of
invoke).

In the case that the device is rooted, we run rt-app as
root. Otherwise, we simply run the workload.
2018-03-01 15:42:20 +00:00
Sascha Bischoff
39925d3f79 rt-app: redirect stdout to stderr
By default rt-app writes to stderr, not to stdout. Therefore, if we
only sample the stdout, we get no output from the workload itself.

With this change, we redirect stderr to stdout to allow us to get the
workload output.
2018-03-01 15:42:20 +00:00
Sergei Trofimov
d44c2ff919 framework/execution: end_run() fix part 2
Missed some "output" references in the previous commit.
2018-02-23 12:38:50 +00:00
Sergei Trofimov
0728f35cbe framwork/target: parameterize disabling of SELinux
Add a parameter to optionally not attempt to disable SELinux as that can
cause problems on some platforms.
2018-02-22 11:20:35 +00:00
Sergei Trofimov
cec3eaa375 framework/execution: fix end_run() on crash
ExecutionContext.end_run() does final updates to the run info in the run
output (final status, run duration, etc). This was previously accessed
via self.output in the context. Typically, this would correctly resolve
to the run output, as there would be no current job. However, in the
event of a crash, current_job would be set, and this would resolve to
the job output itself, resulting in run info not being updated. Use
run_output to avoid this.
2018-02-22 11:20:35 +00:00
Sergei Trofimov
60fbd17fab doc/revent: add missing verbatim block markup 2018-02-22 08:41:04 +00:00
Sergei Trofimov
bbbe1c7814 workloads/hackbench: rename param duration -> timeout
Rename the parameter to better reflect its usage.
2018-02-22 08:41:04 +00:00
Sergei Trofimov
87667cd161 doc/revent: fix event field descriptions
Fix the field descriptions for replay event to reflect the correct
signedness and size.

(note: the layout diagrams were already correct, just the textual
descriptions needed to be adjusted)
2018-02-22 08:41:04 +00:00
Sergei Trofimov
12a33c8aeb framework: move instruments -> instrument
Use the singular to be consistent with other modules in wa.framework.
2018-02-22 08:41:04 +00:00
Marc Bonnici
b7aa7d68b5 config/plugin_cache: Add specific error for global and config entries
Add a special error for finding `global` or `config``  entries in
configuration to provide a more helpful error message.
2018-02-16 15:32:41 +00:00
Marc Bonnici
a52187a114 workloads/rt_app: Fix incorrect reference to context.result
Fix `context.result` to `context.output` leftover from WA2
2018-02-16 14:43:52 +00:00
Marc Bonnici
b103ae34ce configuration/core: Add missing initialization for classifiers 2018-02-16 14:43:52 +00:00
Sergei Trofimov
7313ed5cb5 framework/output: load job_specs
- Load job_specs from jobsfile on reload of RunOutput
- Attach the corresponding spec to a JobOutput
2018-02-16 13:56:03 +00:00
Sergei Trofimov
03328f0456 framework/output: improve access to config
Make it easier to access a run's configuration but automatically
deserializing it on reload of RunOutput, and exposing it via appropriate
settings.
2018-02-16 13:56:03 +00:00
Sergei Trofimov
e3b3799737 utils/misc: remove unused to_identifier
This has been moved to devlib, but, it appears, never removed from WA.
2018-02-16 13:56:03 +00:00
Sergei Trofimov
8466b77d5b framework/config: fix deserialization
- RebootPolicy: correctly handle being passed another instance of
  RebootPolicy on creation; this a requirement for it to be usable as a
  ConfigurationPoint kind.
- RebootPolicy: move static method to above __init__ (consistency).
- RunConfiguration: handle deserialization of 'device_config' entry.
- RunConfiguration: move static method to above __init__ (consistency).
2018-02-16 13:56:03 +00:00
Sergei Trofimov
1e21bd2ca8 framework/output: save classifiers in Result
Save classifiers at Result as well as Metric level. Reason: when
processing output, one might want to filter complete results, as well as
individual metrics. While it is in theory possible to get the
classifiers for a job by simply extracting the common classifiers
between all metrics, this fails when there are no metrics generated for
a job (note that one might still want to process the output in this
case, e.g. for the artifacts).
2018-02-16 13:56:03 +00:00
Marc Bonnici
c5659d51f1 commands/create: Update agenda sections to use augmentations 2018-02-15 13:42:48 +00:00
Marc Bonnici
4b6297e7ea configuration/plugin_cache: Fix incorrect parmeter name 2018-02-15 13:42:48 +00:00
Marc Bonnici
ed75764445 configuration/plugin_cache: Add target names as allowed plugin configs
Allow setting of device configuration by specifying target name.
2018-02-15 13:42:48 +00:00
Marc Bonnici
52dca17fef target/descriptor: Fix generate target default config
Was previously expecting a dictionary of parameter attributes however the
actual type is a list.
2018-02-15 13:42:48 +00:00
Sergei Trofimov
2158dd047e workloads/hackbench: fix target_binary
Set target_binary as a class, rather than instance, attribute. This
happens only only once per run, and setting it as instance attribute the
first time, makes it unavailable for subsequent instances of the same
workload.
2018-02-15 10:14:40 +00:00
Marc Bonnici
9efb95f4ff setup.py: Add work around for installing pandas
For some reason the automatic install of pandas will fail compilation
due to an issue with numpy. A workaround for this issue is to specify
numpy in `setup_requires` as mentioned in https://github.com/numpy/numpy/issues/2434
2018-02-14 19:20:13 +00:00
Marc Bonnici
4703dc616a framework/agenda: Remove unused code 2018-02-14 17:03:21 +00:00
Marc Bonnici
ad5fd0076c configuration/core: Add missing parameter alias for workload_parmeters 2018-02-14 17:03:21 +00:00
Sergei Trofimov
3dbba732f0 workloads/hackbench: fixes
- Only install/uninstall the executable once per run
- Add results file as a raw artifact
2018-02-14 11:48:07 +00:00
Sergei Trofimov
ea64630f57 workloads/rt-app: fix
Change "pull_file" to "pull" when exacting results. "pull_file" was WA2
API which somehow got missed.
2018-02-14 11:46:30 +00:00
Sergei Trofimov
2a712cd3bb workloads: add hackbench
Add hackbench kernel scheduler benchmark, which is part of the Linux
rt-tests suite. See:

	https://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rt-tests.git/tree/src/hackbench/hackbench.8
2018-02-14 11:13:05 +00:00
Sergei Trofimov
b1b379982c framework/output: fix job status
After loading (or attempting to load) job output, update the status from
the state, as that will be more accurate.
2018-02-14 11:13:05 +00:00
Sergei Trofimov
c87d016025 framework/output: handle missing/corrupt result files.
Do not explode if a result file for a job is missing when loading a
RunOutput. Specify job status as "UNKNOWN" and add the exception from
attempting to load the file to the events.
2018-02-14 11:13:05 +00:00
scott
d616d30ee1 Geekbench: Updating result processing to handle GB5 files 2018-02-14 09:29:56 +00:00
Marc Bonnici
0543982f29 framework/host: Generate default config.yaml upon first init 2018-02-14 09:13:04 +00:00
Marc Bonnici
dfce5ca7ba configuration/default: Update to use augmentation terminology 2018-02-14 09:13:04 +00:00
Marc Bonnici
73416d40c2 configuration/core: Remove unused RTParam code 2018-02-14 09:13:04 +00:00
Marc Bonnici
1ec1bd14a5 configuration/core: Remove unnecessary config point
`Augmentations` configuration point should be used instead of `output_processors`
entry which is already specified elsewhere.
2018-02-14 09:13:04 +00:00
Marc Bonnici
965f8e2218 configuration/core: Make generic_android default device 2018-02-14 09:13:04 +00:00
Marc Bonnici
d7f28da71e configuration/core: Add missing description to doc string 2018-02-14 09:13:04 +00:00
Sergei Trofimov
b33defbfea framework/workload: fix "no matching package" error
Fix an issue introduced by commit 42fb3eb

	framework/workload: make "no matching package" message more useful

PackageHandler was incorrectly passing self.name into the error message
instead of self.owner.name.
2018-02-12 11:45:03 +00:00
Sergei Trofimov
42fb3eb148 framework/workload: make "no matching package" message more useful
Make the error message reported when no matching Android app package
could be found more useful by including the workload name, and, if
applicable, sought version.
2018-02-09 17:45:04 +00:00
Sergei Trofimov
0d225dc128 workloads/exoplayer: add view
Add view for the workload, so that SurfaceFlinger frame data can be
collected for it.
2018-02-09 12:30:51 +00:00
scott
6e946a53dd Updating Geekbench workload to support the latest versions of both the standard and corporate application 2018-02-09 12:17:27 +00:00
Marc Bonnici
efc7dc9326 commands/create: Remove global section of generated agenda
Move the iterations parameter into `config`so the `global` section
can be removed.
2018-02-08 18:01:56 +00:00
Sergei Trofimov
ff5e5821d0 workloads/exoplayer: handle no orig rotation
Do not attempt to reset rotation in teardown if target did not specify
it before workload execution.
2018-02-08 15:01:03 +00:00
Marc Bonnici
fbf1bbb1cf workloads/exoplayer: Update supported versions
Adds '2.5' and '2.6' to the supported versions of the workload.
2018-02-08 14:58:39 +00:00