1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

266 Commits

Author SHA1 Message Date
Sergei Trofimov
13ebc8ad55 pep8: removed trailling spaces 2016-02-11 08:22:53 +00:00
Sergei Trofimov
759f8db1bc lmbench: adding taskset support
lmbench can now be run pinned to specific CPUs.
2016-02-11 08:22:39 +00:00
Sergei Trofimov
288aa764b3 dhrystone: fix busybox reference. 2016-02-10 17:28:33 +00:00
Sebastian Goscik
a32cc0f213 Merge pull request #99 from setrofim/master
Minor fixes.
2016-02-10 16:50:17 +00:00
Sergei Trofimov
9129a9d2d8 dhrystone: remove reference to sysbench from dhrystone doc. 2016-02-10 16:38:56 +00:00
ranjeet
0deb8fd7c6 Added GoogleMap : Navigation app by Google Inc. 2016-02-07 10:01:36 +05:30
Sebastian Goscik
68fefe8532 manual: Fixed trying to logcat on non linux devices 2016-02-01 15:24:32 +00:00
Sebastian Goscik
dc22856431 sysbench: Fixed binary installation 2016-02-01 11:13:57 +00:00
Sergei Trofimov
85fb5e3684 Pylint fixes
- apklaunch: ignore package (re-)assignment outside init.
- applaunch: factored out part of result processing into a separate
  method.
2016-01-22 12:19:39 +00:00
Sergei Trofimov
73ddc205fc geekbench: fixing root check
- negating the check (error if *not* rooted)
- do not check for version 2 (results are extracted differently and that
  does not require root).
2016-01-22 10:43:01 +00:00
Sebastian Goscik
f4c89644ff geekbench: Added check whether device is rooted 2016-01-22 09:39:49 +00:00
Sebastian Goscik
798a7befb8 pylint fixes 2016-01-22 09:39:29 +00:00
Sebastian Goscik
82df73278e recentfling: Fixed inequality 2016-01-20 16:31:27 +00:00
Sebastian Goscik
68a39d7fa1 bbench: Fix for web browser crash on latest Linaro release
Also fixes browser permissions issues on Android 6+
2016-01-20 16:29:38 +00:00
setrofim
120f0ff94f Merge pull request #78 from ep1cman/binary_install
BaseLinuxDevice: Tidied up the way binaries are handled
2016-01-19 10:52:54 +00:00
Sebastian Goscik
f47ba6fea6 ebizzy: changed os.path to device path 2016-01-19 10:45:09 +00:00
Sebastian Goscik
5f8da66322 antutu: Fixed runtime permissions
Antutu 6 lists corse_location as a requirement but also asks for
fine_location at runtime. So it is now manually beign granted.
2016-01-19 10:45:09 +00:00
Sebastian Goscik
7c35c604f4 BaseLinuxDevice: Tidied up the way binaries are handled
Added:
get_binary_path: Checks binary_directory for the wanted binary, if
                 if its not there, it will use which to find a
                 system one. returns the full path

install_if_needed: will install a binary only if it is not present.

Changes:
 - Busybox is now deployed to non-rooted devices
 - is_installed has now been removed as the new functions supersede it
 - binaries will now always be installed to `binaries_directory` and
   not system folders.
 - updated workloads to use these new functions
   - rt-app and sysbench might still need work
2016-01-19 10:45:09 +00:00
Chase Qi
bd826783cc applaunch: Added support for IO scheduler test
When IO is heavily loaded, the impact of IO schedulers on App launch
time varies. To measure the impact, added io_stress and io_scheduler two
parameters and related jinja2 blocks.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-01-15 02:16:35 -08:00
Brendan Jackman
6b3187c2c9 workloads: Add apklaunch workload
This is a workload to install and run an arbitrary .apk
2016-01-14 15:58:26 +00:00
Chase Qi
0998c18efd applaunch: pass cleanup argument to the template
Since cleanup test block is defined in the device_script.template, the
value of cleanup is needed to render the template properly.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-01-12 18:20:22 -08:00
Sebastian Goscik
09563bc01e antutu: Updated to support Antutu v6 2016-01-11 14:37:32 +00:00
Sebastian Goscik
585d8b2d7d recentfling: Added workload 2015-12-11 11:02:25 +00:00
Sebastian Goscik
bef8fb40ef Updated pylint for v1.5.1
Fixed WA for the latest version of pylint (1.5.1)
2015-12-09 16:52:39 +00:00
setrofim
3da58d9541 Merge pull request #64 from ep1cman/hwuitest
hwuitest: Added workload
2015-12-03 16:46:06 +00:00
Sebastian Goscik
065ebaac61 hwuitest: Added workload
This workload allow WA to use hwuitest from AOSP to test rendering
latency on Android devices.
2015-12-03 15:07:42 +00:00
Sergei Trofimov
f85ef61ce9 lmbench: add the output file as an artifiact 2015-12-03 10:54:32 +00:00
Sergei Trofimov
705ce9ae40 rt_app: added an option to force install.
Userful when the target device already has rt-app on it, but you want to
replace it with the version from your host.
2015-11-16 12:43:52 +00:00
Sergei Trofimov
90bfbf6022 Adding autotest workload wrapper. 2015-10-09 08:43:53 +01:00
Lisa Nguyen
2c5d51cb2a rt-app: Update use cases
Add video-long.json and video-short.json files and
update spreading-tasks.json. The originals can be
found in git.linaro.org/power/rt-app.git repo.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-08 17:46:20 -07:00
Sergei Trofimov
6159711a05 iozone: ensure test 0 always runs
Test 0 (write) creates a file that is used by subsequent tests.
Therefore if this test is not specified withen selecting which tests to
run, izone will fail with an error.

To avoid this, check the tests list specified by the user andd add test
0 if necessary.
2015-10-08 09:16:32 +01:00
Sergei Trofimov
208fdf4210 iozone: pylint fixes 2015-10-08 09:10:08 +01:00
setrofim
0fc602ecdb Merge pull request #43 from lisatn/iozone
Add iozone workload
2015-10-08 08:59:36 +01:00
Lisa Nguyen
5f6247cf8b iozone: Modify comments
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-01 14:57:35 -07:00
Lisa Nguyen
c96a50e081 iozone: Add description to enable classifiers
In order to show more detailed results for the iozone
workload, inform users to enable classifiers in their
agenda or config file.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-01 14:48:09 -07:00
Lisa Nguyen
812bbffab1 iozone: Rewrite parse_metrics() function
When users specify tests, the parse_metrics()
function doesn't capture the last report name and its
results during the parsing process. Fix the
parse_metrics() function to make sure the data for
all reports are captured.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-01 12:38:04 -07:00
Lisa Nguyen
361f1a0f0c iozone: Whitespace cleanup
Cleanup whitespace and reorganize code.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-01 11:52:51 -07:00
Lisa Nguyen
8e84e4a230 iozone: Add descriptions and refactor code
Add test descriptions, remove defaults, and
fix functions to parse non-thread related data.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-10-01 11:37:05 -07:00
Sergei Trofimov
dc01dd79ee telemetry: report errors. 2015-10-01 12:06:02 +01:00
Sergei Trofimov
f1d3ebc466 adding missing supported_platforms attributes.
Bbench only works on Android. It should advertise that fact by setting
supported_platforms to ['android'].
Telemetry is a Chrome browser workload that is only supported on
ChromeOS and Android.
2015-09-23 08:42:09 +01:00
Lisa Nguyen
18f4c3611c rt-app: Update binaries
A newer version of rt-app has been released. Built
binaries from git://git.linaro.org/power/rt-app.git.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-09-15 14:43:47 -07:00
Lisa Nguyen
100c6c0ac9 iozone: Add functions and rewrite update_result()
Added functions to parse thread-mode results and
non-thread mode results accordingly, in addition
to rewriting the update_result() function.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-09-13 20:50:17 -07:00
Sergei Trofimov
f430187b11 sysbench: fixed case where default timeout < max_time 2015-09-08 18:00:17 +01:00
Sergei Trofimov
9e12930a43 dhrystone: kill any running instances during setup (also handle CTRL-C). 2015-09-08 17:42:18 +01:00
Sergei Trofimov
2e35d4003f nenamark: made duration configurable. 2015-09-03 17:29:42 +01:00
Sergei Trofimov
0d076fe8ba andebench: added parameter to only run native portion of the benchmark. 2015-09-03 17:21:45 +01:00
Sergei Trofimov
9b7b57a4d4 androbench: removed trailing spaces from metric names. 2015-08-21 13:58:00 +01:00
Sergei Trofimov
5aae705172 androbench: added early check for root on device; updated description with link to website. 2015-08-21 09:37:53 +01:00
setrofim
b653beacd3 Merge pull request #45 from bathepawan/master
Androbench Storage Benchmark
2015-08-21 09:16:28 +01:00
Pawan Bathe
f44fd9df7a Change device pull to handle root,and renamed local file as well history.db from results.db 2015-08-20 21:25:55 +05:30
Pawan Bathe
8513304aeb python sqlite3 to remove host/DUT dependencies + other changes 2015-08-19 21:53:50 +05:30
Pawan Bathe
e38c87f258 Fix pylint errors 2015-08-18 01:28:09 +05:30
Pawan Bathe
03a9470007 Fixed issue reported by pep8 checkers 2015-08-18 01:09:45 +05:30
Pawan Bathe
2d8b8ba799 moved UiAutomator code to java, and removed not required python code for device interfacing 2015-08-18 00:46:48 +05:30
Pawan Bathe
e4ee496bc9 Use sqllite3 instead of sqllite to remove host dependencies and result unit in proper format 2015-08-17 23:20:30 +05:30
Pawan Bathe
1cf60c2615 Results format 2015-08-15 02:39:45 +05:30
Pawan Bathe
c3d8128ff3 Androbench Storage Benchmark Workload Addition 2015-08-15 02:10:08 +05:30
Sergei Trofimov
0703db05cf rt-app: relaxing timeouts. 2015-08-12 11:53:29 +01:00
Lisa Nguyen
25e53c2abc iozone: Add license information
Add license information for the iozone binaries.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-08-11 14:07:26 -07:00
Lisa Nguyen
0acbcc9f95 Add iozone workload
Initial commit of the iozone workload.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-08-10 13:13:17 -07:00
Sergei Trofimov
e38bd942a8 bbench: fixed to work when binaries_directory is not in path. 2015-07-17 15:38:47 +01:00
Sergei Trofimov
2d3be33bb0 Added target_config option to telemetry 2015-07-14 11:26:17 +01:00
Sergei Trofimov
a72ae92ece stream: pylint/pep8 fixes. 2015-07-09 08:23:20 +01:00
Lisa Nguyen
255e6c1545 stream: Add initialize and finalize functions
Add initalize and finalize functions in the stream workload
in addition to simplifying code.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-07-08 11:16:13 -07:00
Lisa Nguyen
c733ecad98 Add stream workload
Initial commit of the stream workload to measure
memory bandwidth.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-07-07 11:12:28 -07:00
Sergei Trofimov
a27768fe21 rt-app: do not unintall at then end by default
rt-app workload will no longer uninstall the executable at the
end of the run by default. A parameter can be used to re-enabed the
uninstall.
2015-06-30 15:54:58 +01:00
Bobby Batacharia
5c53f394cb Initial commit of lmbench workload
Remove unused includes
2015-06-29 22:55:56 +01:00
Sergei Trofimov
967f9570e2 manhattan: fixing syntax error introduced by previous commit 2015-06-29 17:57:41 +01:00
Sergei Trofimov
19d6ce5486 manhattan: chaning run_timemout into a parameter and upping default to 10mins 2015-06-29 17:55:43 +01:00
Sergei Trofimov
642da319d4 linpack-cli: setting run timeout based on array size 2015-06-19 09:47:49 +01:00
Sergei Trofimov
a6e9525264 Adding command line version of linpack benchmark. 2015-06-19 09:42:16 +01:00
Sergei Trofimov
c6ede56942 rt-app: added taskset_mask parameter 2015-06-18 15:21:12 +01:00
Sergei Trofimov
0b29bda206 rt-app: removed ftrace check
trace-cmd seems to work fine with "ftrace: true" in the config, so
removing the check that prevented both from being enabled.
2015-06-17 10:14:08 +01:00
Sergei Trofimov
042da24e7d antutu: updating result parsing to handle Android M logcat output
Looks like M formats logcat output with some extra ":"'s which was
screwing up the old parsing logic.
2015-06-17 09:46:11 +01:00
Sergei Trofimov
e7fae25821 list command: can now filter results by supported platform
Added -p option to the list command. This alows filtering results by
supported platforms, e.g.

	wa list workloads -p linux

Also adding missing supported_platforms attribute to various extensions.
If an extension does not have this attribute, the assumption is that it
is supported by all available platforms.
2015-06-16 12:49:07 +01:00
Sergei Trofimov
15e854b8f1 telemetry: fix the doc so they no longer say it must be installed
WA will now fetch it automatically
2015-06-16 11:10:50 +01:00
Sergei Trofimov
d775be25f7 Removing total_tasks "metric" form rt-app, as reported as a classifier. 2015-06-15 13:07:49 +01:00
Sergei Trofimov
8dc4321deb Adding rt-app workload 2015-06-15 12:04:00 +01:00
Sergei Trofimov
67896dfd86 energy_model: adding dhrystone support
- updated energy_model to accept dhrystone as well as sysbench as
  the workload
- added "threads" parameter to sysbench (basically, an alias for
  "num_threads") to be consistent with dhrystone
- added "taskset_mask" parameter to dhrystone to allow pinning
  it to specific cores.
2015-06-11 10:10:36 +01:00
Vasilis Flouris
88ba8e3ba7 Fixes result result processing bug in sysbench 2015-06-09 18:25:30 +01:00
Vasilis Flouris
88c304292f comment correction 2015-06-05 13:00:59 +01:00
Sergei Trofimov
f59da723fb list_or: chaniging how list_or_* functions work and adding a generic list_or
list_or_* functions (e.g. list_or_string) will now always return a list,
however will accept lists or indivitual values. Also added a list_or()
generator function, similar to what already exists for list_of().
2015-06-01 15:31:25 +01:00
Sergei Trofimov
a9ab67990a sysbench: updating to work on unrooted Android devices. 2015-06-01 12:21:51 +01:00
Sergei Trofimov
578dfb3d99 telemetry: fixing "test" parameter description. 2015-06-01 10:08:26 +01:00
Sergei Trofimov
506ed57ca6 fix: telemetry: ignore all return codes
Telemetry seems to return random values as return code, so completely
ignore them and don't treat any values as errors.
2015-05-27 17:25:17 +01:00
Sergei Trofimov
260616711f removing unused variable. 2015-05-26 09:29:59 +01:00
Sergei Trofimov
b9a8f6155c telemetry: remove obsolete metrics. 2015-05-15 14:09:53 +01:00
Sergei Trofimov
512bacc1be Adding classifiers to metrics and updating csv and telemetry to take advantage of them
- Adding "classifiers" field to Metric objects. This is a dict mapping
  classifier names (arbitrary strings) to corresponding values for that
  specific metrics. This is to allow extensions to add
  extension-specific annotations to metric that could be handled in a
  generic way (e.g. by result processors).
- Updating telemetry workload to add classifiers for the url and internal
  iteration (or "time") for a particular result.
- Updating csv result processor with the option to use classifiers to
  add columns to results.csv (either using all classifiers found, or
  only for the specific ones listed).
2015-05-14 15:15:32 +01:00
Sergei Trofimov
ee63cbde62 telemetry: automatically download if necessary
If run_benchmark_path isn't specified, Telemetry zip archive will be
downloaded and extracted into dependencies.
2015-05-13 14:33:13 +01:00
Sergei Trofimov
e7b58c72ac telemetry: added fps extraction
Also fixed text that mistakenly referred to "run_benchmarks" (plural) as
the name of the script.
2015-05-13 14:05:15 +01:00
Sergei Trofimov
48cced2ff9 antutu: robustness fixes for v5 multi-times executions
- added slight delays to ensure more precise navigation
- handle dialogs that occasionally pop up when re-running tests
2015-05-12 14:29:24 +01:00
Sergei Trofimov
57d3b80e85 power_loadtest: kill any already-running autotest instances during setup 2015-05-12 14:27:04 +01:00
Sergei Trofimov
cbd2c6727f Adding ChromeOS power_LoadTest 2015-05-12 11:34:30 +01:00
Sergei Trofimov
a4bff161aa idle: updated to work on Linux devices. 2015-05-11 12:12:41 +01:00
Sergei Trofimov
62593987f4 sysbench workload updates.
- added the ability to run based on time, rather than number of
  requests.
- added a parameter to taskset to specific core(s).

Conflicts:
	wlauto/workloads/sysbench/__init__.py
2015-05-11 12:12:40 +01:00
Sergei Trofimov
57972a56af geekbench: upping run time out.
Run timeout was been hit on very slow systems.
2015-05-07 11:55:37 +01:00
Sergei Trofimov
67ad4a63e4 antutu: multi-times playback fix [part 2]
A "please rate me" dialog occasionally pops but when returning to the
initial screen (when re-running the test). The check to dismiss it
wasn't being done at the right time, so it was still preventing
mutli-times execution. This commit resolves that issue.
2015-05-07 10:48:26 +01:00
Vasilis Flouris
f504fc8791 A fix for spec2000 to align with device API changes 2015-05-06 17:02:48 +01:00
Sergei Trofimov
09d0736d3b antutu: fixing multi-times playback for v5
"times" parameter didn't work properly for version 5 because an extra
back button press was required due to UI changes from previous versions.
This commit adds the button press.
2015-05-06 12:56:58 +01:00
Sergei Trofimov
6824f045fd telemetry: adding support for Android devices
Also, adding missing copyright header.
2015-05-05 12:24:53 +01:00
Sergei Trofimov
1993007d49 telemetry: relaxing the initial validation check
It seems valid install may return values other than 0xff00. Relaxing the
check to consider anything above 0xff to be valid.
2015-05-05 08:17:43 +01:00
Sergei Trofimov
50c8c4da34 pep8: minor fixes in hackbench and ebizzy 2015-04-28 08:30:43 +01:00
setrofim
dcea921907 Merge pull request #17 from lisatn/ebizzy-workload
Add ebizzy workload
2015-04-28 08:27:08 +01:00
setrofim
0beb3fc3a2 Merge pull request #18 from lisatn/hackbench_workload
Hackbench workload
2015-04-28 08:26:58 +01:00
Lisa Nguyen
1633d91331 hackbench: Clean regex code and add run_timeout parameter
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-04-27 14:42:25 -07:00
Lisa Nguyen
d126d56e98 Add ebizzy workload
Add ebizzy to allow users to run a workload resembling common
web server application workloads.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-04-27 12:53:17 -07:00
Sergei Trofimov
698240b6e0 telemetry: copy generated trace files into WA output
If trace files have been generated during a telemetry run (e.g.
--profiler=trace was enabled), copy them into wa_output and extract
them.
2015-04-27 15:50:59 +01:00
Lisa Nguyen
4cff2a52c0 Add hackbench workload
Add hackbench to run tests on the Linux scheduler.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-04-26 13:12:08 -07:00
Sergei Trofimov
aef7f52f96 telemetry: ignore errors in dividuation subtests.
check_output will ignore error code 1 returned by telemetry execution,
as this happens when individiual sub-tests and partial results may, and
should, still be extracted.
2015-04-21 15:12:48 +01:00
Sergei Trofimov
399c9f82c3 telemetry: handle scalar values correctly.
The result regex in telemetry workload has been updated to capture lines
reproting single value results.
2015-04-21 13:19:21 +01:00
Sergei Trofimov
c0bb30e12d Fixing overall score generation in telemetry
Updating overall score generation function to handle zero values.
2015-04-16 13:00:59 +01:00
Sergei Trofimov
ee626fd36e Adding telemetry workload.
This workload executes tests from Google's Telemetry browser test
framework. Currently, only ChromeOS devices are supported.
2015-04-15 18:40:30 +01:00
Lisa Nguyen
e37bf90da7 sysbench: Fix spelling, descriptions, and error message
Make the WorkloadError() message clearer if the sysbench
binary cannot be found on the device and host.

Also make trivial fixes to improve descriptions and fix
spelling errors.

Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
2015-04-10 08:22:01 +01:00
Sergei Trofimov
678efa72e1 Fixing sysbench workload
- Updated sysbench binary to a statically linked verison
- Added missing LICENSE file for the sysbench binary
- Removed Android browser launch and shutdown from workload (now runs on both
  Linux and Android)
- Updated update_result to work with the new binary
- Added missing descriptions for parameters
- Added file_test_mode parameter -- this is a mandatory argumet if test is
  fileio
- Added cmd_params parameter to pass options directily to sysbench invocation
2015-03-25 10:31:02 +00:00
Sergei Trofimov
3e0c8aa83b glb_corporate: clear logcat in Monitor's __init__ to prevent getting results from previous run. 2015-03-16 09:36:32 +00:00
Sergei Trofimov
a747ec7e4c Initial commit of open source Workload Automation. 2015-03-10 13:09:31 +00:00