1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

266 Commits

Author SHA1 Message Date
Marc Bonnici
e4fdf0bdb9 glbenchmark: Updated version numbers from X.X.X to X.X.
Apks contain version number in format X.X therefore when trying to
match for versions using X.X.X the apk is never found.
2016-12-14 15:37:07 +00:00
setrofim
d493b1e790 Merge pull request #307 from marcbonnici/exact_abi
Modified how apks containing native code are matched to a target devices supported abi(s).
2016-12-14 10:32:56 +00:00
setrofim
47e31765b4 Merge pull request #312 from marcbonnici/facebook
Facebook
2016-12-14 08:13:54 +00:00
setrofim
79faec120e Merge pull request #311 from marcbonnici/playbooks
GooglePlayWorkloads: Updated for new version.
2016-12-14 08:13:12 +00:00
Marc Bonnici
66dbe7a508 GooglePlayWorkloads: Updated for new version. 2016-12-13 16:33:50 +00:00
Marc Bonnici
80a780dcfe FacebookWorkload: Fixed UI automation and speificed maximum vesrion.
Fixed the workload UI automation and specified the version of
facebook that the workload is designed to work with.
2016-12-12 18:12:34 +00:00
Marc Bonnici
27b08bade0 PeacekeeperWorkload: Updated to work with new APK versions
Updated peacekeeper UI automation to work with newer version of
chrome and firefox.
2016-12-12 16:49:54 +00:00
Marc Bonnici
1477a89ee4 AndroidDevice: Renamed 'supported_eabis' property to 'supported_abis'
Renamed android device property from 'supported_eabis' to 'supported_abis' to be consistent with linux device.

Updated dex2oat workload to use new property name.
2016-12-09 15:23:58 +00:00
Michael McGeagh
8cbf189029 skype: longer wait time for signin and search 2016-12-09 13:45:35 +00:00
Marc Bonnici
b6f770cfc5 AdobeReader: Updated workload for new app version
Updated the workload to work with the latest version (v16.4)
of Adobe Reader app.
2016-12-08 18:14:16 +00:00
setrofim
c7de8cabd6 Merge pull request #294 from jimboatarm/improve-geekbench-timeouts
geekbench: Improve UiAutomator timeouts for slower devices
2016-12-02 17:04:01 +00:00
setrofim
fd7df36a5a Merge pull request #301 from jimboatarm/skype-hangup
skype: now hangs up at the end. Better duration handling
2016-12-02 17:03:13 +00:00
Michael McGeagh
ae4ae3da5e skype: now hangs up at the end. Better duration handling 2016-12-02 11:30:30 +00:00
Michael McGeagh
393abc267f GooglePhotos: Navigate up only when needed. Use baseclass' selectGalleryFolder 2016-12-02 11:28:38 +00:00
James Hartley
088709f290 geekbench: Improve UiAutomator timeouts for slow devices, fixes for rooted devices
Tested by running Mate 8 on the lowest cluster at the lowest available frequency

On rooted devices, skip attempting to pull log files from device.  This allows
the benchmark to run to completion without failure.

GB log files are stored in /data/data/com.primatelabs.geekbench/files and
not accessible without root.  On Chromebooks ( which have no adb root
rights) it is possible to copy from this folder manually after the test run.
2016-12-02 07:23:44 +00:00
Sergei Trofimov
850fcb24ab pylint/pep8 fixes 2016-12-01 15:01:45 +00:00
Michael McGeagh
cb53fe9ec8 Gmail: A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them.
Moved broadcast to super. Mandatory and Default are XOR

Added a longer wait for sync to finish. Increases reliability on certain phones

Changed recipient to not mandatory and a default set

Wait for sync when launching gmail from the sharing feature

Fix: cornercase where image viewer already points to working directory. Refactored code due to duplication

Added new function to BaseUiAutomation class to find a folder in the gallery
2016-12-01 13:21:49 +00:00
James Hartley
f69e4c5b18 Fix octaned8 results capture
Ensure the device log file is deleted between runs and that the host file is not kept as an
artefact.  Both can lead to errors appending data from the next iteration/run.
2016-11-24 14:54:19 +00:00
setrofim
40d281b336 Merge pull request #283 from jimboatarm/octaned8
Add Octane d8 workload
2016-11-11 17:32:53 +00:00
James Hartley
642d757066 Add Octane d8 workload
A workload to run the binary (non-browser) version of Octane
2016-11-11 17:04:25 +00:00
setrofim
969201968e Merge pull request #282 from jimboatarm/min-apk-version-fix
Support for Geekbench 4
2016-11-11 11:23:07 +00:00
James Hartley
46b78d35be Geekbench 4 is similar in terms of operation but runs significantly different tests and hence
produces a different results output.

- Add UiAutomator and results parsing support for GB4
- Remove the version checking in init.py and rely instead on the ApkWorkload version
  checking.  The required version in the agenda is checked against that on the host or target

Tested with GB3.4.1 and GB4.0.0 on a Galaxy S7.  Older variants of the APK were not available.
2016-11-10 18:56:22 +00:00
Michael McGeagh
6d173f2f3f Change min apk ver for reader 2016-11-10 15:48:50 +00:00
Leo Yan
d3c59e2f74 rt-app: add user case for camera
Add camera-long.json and camera-short.json files to simulate camera
 recorder.

Signed-off-by: Leo Yan <leo.yan@linaro.org>
2016-11-10 12:08:02 +08:00
setrofim
27df426c0d Merge pull request #263 from jimboatarm/books-codetidy
GooglePlayBooks: Code tidy to conform.
2016-11-08 13:19:27 +00:00
Michael McGeagh
f8966bf324 GooglePlayBooks: Code tidy to conform. Additional parameter added for when the search name differs to the library name. Defaults changed due to Hamlet dissapearing from the store.
Added cornercase of accidentially triggering a card popup

New version includes a new dialogue and name for search. Added support for this

If device has more than one account associated, a new popup appears. Handle that with an optional parameter listing specific account to select, if ommitted the first account in the list is selected
2016-11-08 12:31:27 +00:00
James Hartley
359d9d3e5f Fix incorrect min_apk_version setting in adobereader and googleplaybooks 2016-11-07 15:42:38 +00:00
Sergei Trofimov
41b52178bb pylint fixes. 2016-10-17 11:02:41 +01:00
Michael McGeagh
4649fa13db AdobeReader: Fails on S7 due to search icons being a different class. Removed the class specifier 2016-10-11 13:25:35 +01:00
Sergei Trofimov
edc26fe75c vellamo: fixing capitalization in automation
Capitalization for the "LET'S ROLL" button text was incorrect in the
automation code. This has not caused issues up to this point, but it
seems the button is no longer being found in the latest AOSP. This
commit corrects the capitalization.
2016-10-07 17:21:56 +01:00
setrofim
6aecf1b35e Merge pull request #264 from jimboatarm/reader-workload
Reader Workload: The Adobe Reader workflow carries out typical…
2016-10-05 08:30:57 +01:00
Michael McGeagh
4c5e008609 Reader Workload: The Adobe Reader workflow carries out typical productivity tasks.
Conform to code style of other workloads.

Changed search strings to be a list. Allows user the ability to search for 1 or more search terms.

Renamed Reader to AdobeReader. Removed sleep and now detect and wait for searchicon
2016-10-04 16:59:22 +01:00
setrofim
5bed658045 Merge pull request #262 from jimboatarm/youtube-codetidy
Youtube: Codetidy to conform with the other workloads
2016-10-03 14:33:08 +01:00
setrofim
8ac5657993 Merge pull request #260 from jimboatarm/slides-codetidy
GoogleSlides: Code tidy, to conform with the other workloads.
2016-10-03 14:09:42 +01:00
setrofim
fc26daecfc Merge pull request #261 from jimboatarm/skype-codetidy
Skype CodeTidy to conform with the other workloads
2016-10-03 13:44:02 +01:00
Michael McGeagh
4e94ff9ed7 Skype CodeTidy to conform with the other workloads
Dont double click if the confirm icon appears
2016-09-29 17:29:21 +01:00
Michael McGeagh
c47ae5cfcf Youtube: Codetidy to conform with the other workloads 2016-09-28 14:39:53 +01:00
Michael McGeagh
ce11b94f28 GoogleSlides: Code tidy, to conform with the other workloads. 2016-09-28 13:02:49 +01:00
Michael McGeagh
765fdd7cbb GooglePhotos: Change step value for crop operation to speed up workload. 500 is too slow and not realistic of real user behaviour.
Image order not guaranteed. Changed workflow to use subfolders to guarantee ordering and have same action be performed on the same image. As a result, swipe gestures are no longer applicable

Colour change was using clicks. Now uses drags. This is more reliable across devices
2016-09-28 09:58:22 +01:00
Marc Bonnici
e69aea4e69 Skype: Added check for update popup.
Since there has been an update to the skype app, there was now a
update prompt preventing test from running. Now checks for the
prompt on older versions of skype and dismisses it if present.
2016-09-23 11:18:40 +01:00
Michael McGeagh
5738d19114 Skype: Codetidy, no functional changes. Make voice action default in code, move private function to bottom. 2016-09-22 16:52:39 +01:00
Sebastian Goscik
5dbf7e7d38 Merge pull request #251 from jimboatarm/googlephotos-codetidy
GooglePhotos: Reorder functions and relabel ActionLogger tags
2016-09-20 12:48:45 +01:00
Michael McGeagh
a2945d58cb Reorder functions and relabel ActionLogger tags. No functional changes made to actual workload run 2016-09-20 11:15:40 +01:00
Sebastian Goscik
8727fe514a Merge pull request #249 from jimboatarm/googleplaybooks-gesturefix
GooglePlayBooks: Reorder so that the gesture test happens after selec…
2016-09-20 10:55:57 +01:00
setrofim
6465e732fd Merge pull request #250 from ep1cman/apk_ver_fixes
Redone APK file resolution
2016-09-20 10:50:29 +01:00
Sebastian Goscik
f9ec869c7b Revert "Updated workload versions to match APK files"
This reverts commit b426e00f2f.
2016-09-20 10:47:57 +01:00
Sebastian Goscik
1bbd3ef87a Revert "workloads: Fixed versions to be backward compatible"
This reverts commit 8608c3b747.
2016-09-20 10:47:57 +01:00
Michael McGeagh
5018a1ec94 GooglePlayBooks: Reorder so that the gesture test happens after selecting a known chapter in the book. Previously the gestures could happen on any random page thus making the results not neccessarily comparable to another run. 2016-09-20 10:10:21 +01:00
Sebastian Goscik
fe58245843 Merge pull request #248 from jimboatarm/youtube-actionlogger-modified
Youtube: Changed actionlogger to measure different more relevant acti…
2016-09-19 17:36:39 +01:00
Michael McGeagh
19569816d3 Youtube: Changed actionlogger to measure different more relevant actions. Also rearranged if statement for video_source so that the default is at the top. 2016-09-19 16:46:03 +01:00
Michael McGeagh
1dfbaf4ebe GooglePlayBooks: Modified ActionLogger tags and capture points 2016-09-19 14:21:03 +01:00
Michael McGeagh
b5dc5b8648 GoogleSlides workload did not do slide_count number of swipes in edit mode. This fixes that. 2016-09-19 13:26:21 +01:00
Sebastian Goscik
8608c3b747 workloads: Fixed versions to be backward compatible
In a recent commit workload versions were changed to match their APK versions.
This commit adds the old versions to the allowed versions and automatically
maps them onto the new values.
2016-09-13 11:07:18 +01:00
muendelezaji
d279cc7453 Add Google Slides workload 2016-09-12 17:57:41 +01:00
Sebastian Goscik
ff2f88fbd7 Merge pull request #225 from jimboatarm/googlephotos_uxperf
Add googlephotos workload
2016-09-09 14:26:46 +01:00
John Richardson
96f4ade874 Add googlephotos workload 2016-09-09 09:56:15 +01:00
setrofim
ac0256e377 Merge pull request #239 from jimboatarm/googleplaybooks_fix
Updates and fixes for googleplaybooks workload
2016-09-08 15:50:17 +01:00
John Richardson
5ef7d2dd44 Remove skypevideo workload
The original skypevideo workload has now been replaced with the newer
skype workload.
2016-09-07 16:51:40 +01:00
John Richardson
cf8cb5bfab Add Skype workload 2016-09-07 16:45:11 +01:00
John Richardson
e18366b3f8 Add support for chrome books to googleplaybooks workload
Accommodate layout changes in googleplaybooks on chrome book devices
when accessing the action bar menu in the app.
2016-09-05 17:15:49 +01:00
muendelezaji
441ba974b7 Youtube workload 2016-09-05 15:03:53 +01:00
John Richardson
a33df50ce8 Use new AndroidUxPerformance class in googleplaybooks
Utilise new convenience method for checking for an valid internet
connection and set the min_apk_version within the AndroidUxPerformance
class.
2016-09-05 14:56:14 +01:00
John Richardson
52d4635fe8 Improve logic for scrolling during book search 2016-09-02 15:29:28 +01:00
John Richardson
14924ec6f4 Remove hardcoded package name from googleplaybooks workload 2016-09-02 15:29:28 +01:00
Sebastian Goscik
9d4aa4983a antutu: Fixed setting permissions
It has been observed on some devices that the FINE_LOCATION premissions is required
for antutu to run without asking for permissions at run time but this was not listed
in the APK manifest. This caused issues on devices were only the permissions in the
manifest can be granted. This commit sliences any error when trying to set this permission
as well as only trying only on Android 6+
2016-08-24 14:36:00 +01:00
Sebastian Goscik
b426e00f2f Updated workload versions to match APK files
Some workloads presented a different version than what was in the APK file.
With the changes introduced in bb33123 several workloads broke.
2016-08-24 14:33:18 +01:00
John Richardson
0f579e18b3 Add googleplaybooks workload 2016-08-09 12:08:20 +01:00
John Richardson
51e4e71931 Upgrade Android API level from 17 to 18
Upgrade API level for BaseUiAutomation from 17 -> 18 and update
dependent workloads accordingly.
2016-07-26 10:30:15 +01:00
Sebastian Goscik
9707aa6237 Pylint fixes
Pylint now checks for trailing new lines, this commit fixes them.
2016-07-21 16:40:26 +01:00
Michele Di Giorgio
2dd3a2ba4d camerarecord: add possibility to select slow_motion recording mode
Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2016-07-13 16:40:39 +01:00
muendelezaji
bb33123b17 Check APK version and ABI when installing
- Check the APK's versionName property against the workload's
  expected version if specified
- If workload specifies check_abi param, try to get APK from
  ABI-specific path on the host
- Add variant_name param to APK resource-getter for backwards
  compatibility of dex2oat and peacekeeper
2016-07-12 17:02:17 +01:00
Michele Di Giorgio
f27b500028 camerarecord: add frame stats collection through dumpsys gfxinfo
Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2016-07-11 13:53:38 +01:00
Michele Di Giorgio
d3dd9c849a geekbench: fix output files listing
After running the benchmark, when collecting the output files the execution
fails because the split('\n') call, used for creating a list of output files,
returns an empty string as last element. The empty string makes the pull command
fail because file '' doesn't exist on the target device.

Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2016-07-08 11:59:51 +01:00
Sergei Trofimov
c8a735e298 sysbench: adding arm64 binary 2016-07-01 17:35:19 +01:00
Ranjeet Kumar
88b18dda07 Permission granted for generic browser 2016-06-29 17:57:25 +05:30
Sergei Trofimov
8660d0f488 hwuitest: invoke executable via full path
Previously, the workload was invoking the executable via its name,
assuming that it will be in PATH. As WA's executables directory is not
in path, the invocation was failing. This commit saves the full path to
the installed executable and uses that instead.
2016-06-17 08:03:50 +01:00
Sergei Trofimov
be7aa3d379 recentfling: fixing uninstalling of binaries
Inconsistently, while install() for Android devices automatically
handles both APKs and executables appropriately, uninstall() only works
for packages. Changing to use uninstall_executable() for the scripts
deployed by recentfling.
2016-06-15 09:00:49 +01:00
Sergei Trofimov
b48e5ce58a recentfling: script deployment and PID file fixes
- Scripts are now deployed during via install() ensuring that they are
  executable.
- Handle the case where the PID file is delted before getting to
  process_results.
- Exposed the option to not start any apps before flinging via a
  parameter.
2016-06-14 10:21:06 +01:00
Sergei Trofimov
f33d6f4729 bbench: handle lack of results in logcat
- make sure results_list is always instatiated even if no metrics found;
  this would previously cause a "used before declaration" error
- Detect that no metrics were extracted from the log and raise a
  WorkloadError
2016-06-14 10:20:51 +01:00
Sebastian Goscik
23b3b165d5 docs: Change log & updates 2016-06-10 13:17:10 +01:00
Sebastian Goscik
7ccac87b93 cameracapture & camerarecord: Fixed parameters
Parameters were not being passed to the UI automation properly
2016-05-25 09:49:21 +01:00
setrofim
24a2afb5b9 Merge pull request #168 from ep1cman/vellamo-update
Vellamo update
2016-05-24 13:01:31 +01:00
Sebastian Goscik
9652801cce vellamo: Fixed geting values from logcat
The previous method of getting results out of logcat does not work
if the format of logcat changes.
2016-05-24 13:00:10 +01:00
Sebastian Goscik
f02b6d5fd9 vellamo: Added support for v3.2.4 2016-05-24 09:57:38 +01:00
Sebastian Goscik
eaf4d02aea Merge pull request #162 from chase-qi/add-blogbench-workload
workloads: add blogbench workload
2016-05-24 09:55:37 +01:00
Chase Qi
56a4d52995 workloads: add blogbench workload
Blogbench is a portable filesystem benchmark that tries to reproduce the
load of a real-world busy file server.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-05-24 16:49:19 +08:00
Sebastian Goscik
ec5c149df5 Merge pull request #165 from chase-qi/add-stress-ng-workload
workloads: add stress_ng workload
2016-05-24 09:45:35 +01:00
Sebastian Goscik
5a1c8c7a7e cameracapture & camerarecord: Updated workloads to work with Android M+
The stock camera app as of Android M has changed. This commit updates
the ui automation to work with this new app. As part of this change
it was required to bump the API level of the ui automation to 18.

Also made the teardown of the capture workload close the app like the
record workload.
2016-05-16 17:25:50 +01:00
Chase Qi
5ad75dd0b8 workloads: add stress_ng workload
stress-ng will stress test a computer system in various selectable ways.
It was designed to exercise various physical subsystems of a computer as
well as the various operating system kernel interfaces.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-05-13 19:35:26 +08:00
Sergei Trofimov
44a49db04d glbcorp: pep8 fix
Added a missing blank line between method declaration and class
attribute definitions.
2016-04-15 16:39:24 +01:00
Sebastian Goscik
73aa590056 glbench: renamed start_activity to launch_package
To match changes made in: ff5f48b7e7
2016-04-14 16:36:37 +01:00
Sebastian Goscik
985b249a24 glbench: Fixed ending regex
Updated the regex that detected the end of the benchmark to match the new
logcat format.
2016-04-14 16:36:37 +01:00
Sebastian Goscik
9a301175b0 glbench: Fixed updated logcat format
The old results looked like:
I/TfwActivity(30824):    "description": "",
I/TfwActivity(30824):    "elapsed_time": 62070,
I/TfwActivity(30824):    "error": "NOERROR",

The new format is:
04-04 11:38:04.144  1410  1410 I TfwActivity:    "description": "",
04-04 11:38:04.144  1410  1410 I TfwActivity:    "elapsed_time": 62009,
04-04 11:38:04.144  1410  1410 I TfwActivity:    "error": "NOERROR",
2016-04-04 17:33:48 +01:00
Sebastian Goscik
50353d0b8f Merge pull request #122 from Sticklyman1936/lmbench_update
lmbench: Tidied up the code and improved stability
2016-03-24 16:26:52 +00:00
Sascha Bischoff
2eca77fb02 sysbench: use device busybox binary
Use the full path to busybox on the target device as opposed to
assuming it is found on the path.
2016-03-24 16:21:01 +00:00
Sascha Bischoff
3de5b5fe0b lmbench: Tidied up the code and improved stability
This patch tidies up the benchmark code to bring it in line with the
style used in Workload Automation in general. Additionally, the
results from sub-benchmarks are now directly written to a file on the
device as opposed to processing the standard output/error from the
benchmark, which was error prone.
2016-03-24 10:20:32 +00:00
Sergei Trofimov
3043506d86 applaunch: pass the location of busybox into the script
applaunch creates and deploys an auxilary script in order to collect
precise timings. This script invoked busybox with the assumption that it
is in PATH.

Since recent changes mean that it is no longer deployed to /system/bin,
the busybox in not found. With this commit, the full path to busybox
will be passed into the script's template.
2016-03-23 16:28:18 +00:00
Sebastian Goscik
3531dd6d07 dhyrstone: Fixed arm64 binary
It was dynamically linked, its is now statically linked
2016-03-15 14:38:18 +00:00
Sebastian Goscik
11184750ec dhrystone: Updated executable resolution
Previously it was just using the binary in the dhrystone folder.
Now it uses WA's resource resolution to use the correct ABI.
2016-03-09 14:54:39 +00:00
Steve Bannister
08663209d6 Fix up lmbench commandline 2016-02-11 17:40:31 +00:00