1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

860 Commits

Author SHA1 Message Date
jummp01
79f58e74b9 Googleplaybooks: implement ApplaunchInterface
Adapt googleplaybooks to support applaunch workload by implementing the methods
required by the interface.

Tested on Huawei Mate8 and Samsung S7.
2017-02-05 12:18:31 +00:00
jummp01
7fe699eb41 Youtube: implement ApplaunchInterface
Adapt youtube to support applaunch workload by implementing the required methods of the interface.

Tested on Huwaei Mate8 and Samsung S7.
2017-02-05 11:52:20 +00:00
Marc Bonnici
2bd77f0833 UiAutomator: Removed unnecessary class files.
Removed stale classes from the repository.
2017-02-03 17:14:33 +00:00
Marc Bonnici
7b7eb05e7f AdobeReader: Updated workload to work with latest version.
The 'My Documents' element that the workload used to check for completed
setup has been removed in the latest version, now checks for the
text itself which is also present in previous versions.
2017-02-03 15:42:53 +00:00
jummp01
75b56e462c Adobereader: implement ApplaunchInterface
Adapts adobereader to support uxperfapplaunch workload by implementing the required methods
of the interface.

Tested on Huwaei Mate8 and Samsung S7.
2017-02-03 15:15:05 +00:00
jummp01
d8438c5ae8 Gmail: implement ApplaunchInterface
Adapts gmail workload to support applaunch workload by implementing the
required methods of the interface.

Tested on Huawei Mate8 and Samsung S7.
2017-02-03 15:14:56 +00:00
jummp01
ca7a1abe78 Add new applaunch workload.
The workload supports launch time measurement of other uxperf workloads that
implement the ApplicationlaunchInterface. It takes a uxperf workload as a parameter
and helps to instrument the application launch time in two modes.

a)launch_from_background
b)launch_from_long_idle

The workload executes in two major phases.
1- Setup phase: clears initial dialogues on the first launch of an application.
2- Run phase: Runs multiple iterations of the application launch and measures
the time taken for launch. Iteration number can be specified as parameter applaunch_iterations.

Applaunch measurements are captured in the logcat file.
2017-02-03 15:14:47 +00:00
jummp01
bd37973442 Remove existing applaunch workload
The applaunch workload is deprecated to be replaced by another implementation.
2017-02-03 15:14:37 +00:00
jummp01
c38dc287ab Move UxperfParser into utils
UxperfParser class is moved from the UxperfResultProcessor class into a new
python module in utils. This will help to use the UxperfParser even when
the result procesor is not configured.
2017-02-03 15:14:27 +00:00
jummp01
3feb702898 fps: move VSYNC_INTERVAL into utils
It is not anything to do with instrument, but a generic ocnstant,
and this way it can be used by the other parts of the code line.
2017-02-03 15:14:16 +00:00
jummp01
0f57dee6bf Add ApplaunchInterface class
Inorder to support application launch workload, a new interface is created
which has methods that need to be implemented by all workloads that
support application launch time instrumentation.
2017-02-03 15:14:03 +00:00
jummp01
99b46b4630 Add UiAutoUtils class
UiAutoUtility class added to support utility functions used for UiAutomation.
This class can help in refactoring some existing fucntionalities into utility functions.

Launch command generation is added as a utility class method which can be used
by workloads to construct their launch command from parameters passed.
2017-02-03 15:13:50 +00:00
jummp01
4ce20e2d3f Move package parameters to UxperfUiAutomation
All the uxperf workloads get some common package parameters.
These are moved to the parent class and a new method is introduced to fill
these parameter values. All the uxperf workloads can call this method to resolve
the package parameters.
2017-02-03 15:13:32 +00:00
jummp01
65aa00483b Add pressHome() method in the Base class
This function Presses Home button on the Android User Interface.
2017-02-03 15:12:35 +00:00
setrofim
da7a3276ed Merge pull request #339 from marcbonnici/revent_fix
Revent fix / improvments
2017-02-01 18:10:32 +00:00
Marc Bonnici
b89d9bbc4b StateDetection: Moved check for missing phase definition earlier.
Now checks to see if a phase is correctly defined before image matching
so that it is only performed if required.
2017-02-01 18:04:05 +00:00
Marc Bonnici
54bffb45ab Revent: Fixed uninitialized variable. 2017-02-01 18:04:05 +00:00
Marc Bonnici
e7a47f602d Revent: Removed check for 'wait_for_stdin'.
Revent is terminated from WA via a 'SIGINT', therefore this means
that in order for revent to receive the signal and deal with it
accordingly, revent always needs to be listening on STDIN regardless of
the 'wait_for_stdin' flag.
2017-02-01 18:03:28 +00:00
Sergei Trofimov
490dd404ae AndroidDevice: write "dumpsys window" output on host
On Android targets, WA collects device display information by running
"dumpsys window" during run initialisation. Previously, this was
redirectied into on-device file (under the working directory) and then
pulled from the target.

It looks like on Android-on-ChromeOS devices the redirect leads to an
"Unknown Error" and the resulting file is empty. To get around that,
this commit modfies the dumpsys command so that the output is collected
directly from the shell's stdout and then writen on the host.
2017-01-24 17:52:51 +00:00
setrofim
60f52c2187 Merge pull request #265 from jimboatarm/multiapp-workload
Multiapp Workload: Workload to test how responsive a device is when…
2017-01-16 16:41:35 +00:00
setrofim
fbb9908298 Merge pull request #330 from jimboatarm/break_setup
Splits ApkWorkload setup() into short methods.
2017-01-13 15:39:11 +00:00
jummp01
01fa0a3571 Splits ApkWorkload setup() into short methods.
Apkworkload setup phase performs many functionalities in
a single method that is broken down into short methods.
The split short methods can be called individually when
relevant use cases arise.
2017-01-13 11:56:39 +00:00
setrofim
be2b14a575 Merge pull request #331 from jimboatarm/skype_fix
Create a function for launching skype application.
2017-01-13 05:28:00 +00:00
setrofim
dab2bbb1c7 Merge pull request #329 from jimboatarm/epochtime
Changes Action Logger to give epoch time
2017-01-13 05:26:28 +00:00
Siddhanathan Shanmugam
e64dc8bc26 Fix typo
Instrumention -> Instrumentation
2017-01-12 12:13:05 -08:00
Sergei Trofimov
cb48ece77f daq: fixing energy metric calculation
The summary energy metric was being calculated incorrectly. Instead of
dividing power by the sampling rate, it was being multiplied by it and
divided by a million for some reason.
2017-01-12 11:29:44 +00:00
jummp01
8f67b7f94b Create a function for launching skype application.
Skype has a unique launch command which is called in the setup
phase of the workload. The launch command is split into a stand alone
method which can be called as a separate method if required.

This can be used as common method if more applications in future
require their own customized launch command.
2017-01-12 11:27:42 +00:00
jummp01
fa553ee430 Changes Action Logger to give epoch time
Log time changed to produce epoch time in milli seconds.
Nano to milli second conversion done in uxperf result
processor is removed.

Tested on Mate8 and time obtained is verified.
2017-01-12 10:30:07 +00:00
Sergei Trofimov
01c9c88e79 perf: do not force root on Android in stop()
perf instrument was forcing killall() to run as root on Android devices.
This constraint was preventing perf from being used on unrooted devices.
However, it appears that it is possible for killall() to succeed on at
least some devices as a regular user.

This commit removes the constraint. Since killall() will default to
running as root whenever possible, the instrument will still behave
correctly on rooted Android devices where root is required.
2017-01-11 13:44:55 +00:00
setrofim
0c32e39ce0 Merge pull request #317 from jimboatarm/minapkremoved
Removed min_apk_version from UXperf workloads
2017-01-09 15:13:04 +00:00
Marc Bonnici
1364ec05e8 Version bump 2016-12-23 14:14:43 +00:00
Marc Bonnici
30e9b553ff get_assets: Updated Remote Assets URL
Updated URL to new location of WA assets as previous URL was no longer found.
2016-12-23 10:39:51 +00:00
Marc Bonnici
13cbe2f059 Renamed `get-assests.py to get_assests.py`
``get-assests.py`` is not a valid python module name therefore it has been
renamed to ``get_assests.py``, the command however remains with the same name.
2016-12-23 10:39:51 +00:00
Marc Bonnici
53b173c55f Documentation Update
Updated documentation to conform with pylint and sphinx.
2016-12-23 10:39:51 +00:00
marcbonnici
f598c60514 Merge pull request #324 from marcbonnici/spec2000
Spec2000
2016-12-23 10:38:43 +00:00
marcbonnici
ceda8e74bf Merge pull request #323 from marcbonnici/poller
Poller
2016-12-23 10:35:44 +00:00
Marc Bonnici
173c71b867 FilePoller: Updated 32 bit binary.
Recompiled the 32 bit binary as previous version would fail
due to missing applet.
2016-12-22 17:38:20 +00:00
Marc Bonnici
d88d35be26 FilePoller: Made files a mandatory parameter.
Changed the `files` parameter to be mandatory and added check to
ensure that at least one file is passed in configuration otherwise
the instrument crashes.
2016-12-22 17:34:50 +00:00
marcbonnici
599452d41f Merge pull request #320 from marcbonnici/perf
Perf: Recompiled 32bit binary as little endian.
2016-12-22 11:04:14 +00:00
marcbonnici
33dae51536 Merge pull request #322 from marcbonnici/big.Little
big.Little Sample Agenda: Fixed workload name and typo.
2016-12-21 17:10:33 +00:00
marcbonnici
f8950dea33 Merge pull request #318 from marcbonnici/googlephotos
GooglePhotos: Updated to work with the latest version (v2.6.0).
2016-12-21 17:09:20 +00:00
Marc Bonnici
136d1fef0f GooglePhotos: Updated to work with the latest version (v2.6.0).
Dismisses backup confirmation in newer versions.
Updated for new way of editing photo colours and lack of confirmation.
2016-12-21 17:05:09 +00:00
marcbonnici
5204383582 Merge pull request #321 from marcbonnici/netstat
NetStat: Updated Regex to match logcat format in android 6.0
2016-12-21 16:53:55 +00:00
marcbonnici
bfa1d8dd62 Merge pull request #319 from marcbonnici/dex2oat
Dex2oat: Now uses root if available.
2016-12-21 16:53:12 +00:00
Marc Bonnici
b75fdf85d0 big.Little Sample Agenda: Fixed workload name and typo.
One of the workloads was listed as sqlite which is an instrument,
corrected the workload name to sqlitebm.

Added missing word in description.
2016-12-21 12:08:51 +00:00
Marc Bonnici
fcbb83f5ac Spec2000 Workload: Spelling Corrected 2016-12-20 16:34:07 +00:00
Marc Bonnici
807003128e Spec2000 Workload: Updated to use busybox provided from WA.
Commands have been updated to use the busybox executable that
is installed via WA rather than relying on it already being in
PATH on the device as this would fail for devices that it was
not installed on.
2016-12-20 16:20:59 +00:00
Marc Bonnici
3e4d068eff NetStat: Updated Regex to match logcat format in android 6.0
Android marshmallow uses a different default view for logcat
therefore prevented output from being extracted. The regex
expression has been updated to include matching the new format.
2016-12-20 09:56:26 +00:00
Marc Bonnici
a3936afb4c Perf: Recompiled 32bit binary for little endian.
Compiled 32bit perf binary as little endian instead of big
endian to support more devices.
2016-12-16 18:13:43 +00:00
Marc Bonnici
24000a21df Dex2oat: Now uses root if available.
Command fails to execute with permission error on some devices therefore
now attempts to use root if available.
2016-12-16 11:20:37 +00:00
Michael McGeagh
51f07c4473 Multiapp Workload: Workload to test how responsive a device is when context switching between appplication tasks. It combines workflows from googlephotos, gmail and skype.
Added tagName to give unique logger name for multiple share attempts. Turn off markers_enabled for subclasses

Renamed multiapp to appshare. Description is now more accurate

Changed appshare to use class instances for each sub workload. This allows APKs to be setup correctly on the device

Photos changed function name
2016-12-15 16:19:19 +00:00
Michael McGeagh
ffde7401ef Removed min_apk_version from UXperf workloads. The known working version is now part of the description instead. 2016-12-15 16:13:58 +00:00
James Hartley
b4026ae390 Fix inverted swipe up/down gestures
Tested on Chromebook and Mate 8
2016-12-15 15:28:23 +00:00
James Hartley
f76c00dd99 Fix adobereader on S7
Tested on S7
2016-12-15 15:04:07 +00:00
setrofim
b1e375a676 Merge pull request #314 from marcbonnici/skype
SkypeWorkload: Workload Fixes
2016-12-15 11:18:24 +00:00
Marc Bonnici
1102ba1679 SkypeWorkload: Workload Fixes
Fixed contact not being found when mistakenly selecting search for
bots instead of contact.

Changed element used to determine when contact has been
selected from status indicator to search icon as with other other
contacts in view the element is never removed.

Now throws error if end call button is not found otherwise workload
could fail to start the call and incorrectly report overall success.
2016-12-15 10:30:34 +00:00
setrofim
6d999301f3 Merge pull request #313 from marcbonnici/glbenchmark
glbenchmark: Updated version numbers from X.X.X to X.X.
2016-12-14 16:42:52 +00:00
Marc Bonnici
e4fdf0bdb9 glbenchmark: Updated version numbers from X.X.X to X.X.
Apks contain version number in format X.X therefore when trying to
match for versions using X.X.X the apk is never found.
2016-12-14 15:37:07 +00:00
setrofim
d493b1e790 Merge pull request #307 from marcbonnici/exact_abi
Modified how apks containing native code are matched to a target devices supported abi(s).
2016-12-14 10:32:56 +00:00
setrofim
47e31765b4 Merge pull request #312 from marcbonnici/facebook
Facebook
2016-12-14 08:13:54 +00:00
setrofim
79faec120e Merge pull request #311 from marcbonnici/playbooks
GooglePlayWorkloads: Updated for new version.
2016-12-14 08:13:12 +00:00
Marc Bonnici
66dbe7a508 GooglePlayWorkloads: Updated for new version. 2016-12-13 16:33:50 +00:00
Marc Bonnici
80a780dcfe FacebookWorkload: Fixed UI automation and speificed maximum vesrion.
Fixed the workload UI automation and specified the version of
facebook that the workload is designed to work with.
2016-12-12 18:12:34 +00:00
Marc Bonnici
694d51ffb6 AndroidWorkload: Fixed validate version error messages.
Fixed typos in ``validate_version`` method which caused incorrect reporting
of maximum apk version.
2016-12-12 18:09:40 +00:00
Marc Bonnici
27b08bade0 PeacekeeperWorkload: Updated to work with new APK versions
Updated peacekeeper UI automation to work with newer version of
chrome and firefox.
2016-12-12 16:49:54 +00:00
Marc Bonnici
1477a89ee4 AndroidDevice: Renamed 'supported_eabis' property to 'supported_abis'
Renamed android device property from 'supported_eabis' to 'supported_abis' to be consistent with linux device.

Updated dex2oat workload to use new property name.
2016-12-09 15:23:58 +00:00
Marc Bonnici
0dfbbae7b6 Renamed 'check_abi' parameter to 'exact_abi' 2016-12-09 14:11:16 +00:00
Marc Bonnici
a8a8d21de6 AndroidWorkload: Modified workload to properly check for an apks abi
Previously when retrieving apks only it's name would be used to choose
an apk. Now the native code reported by the apk is used to determine
the correct version to run for the specific device. It tries to
match the primary abi of device with native code before falling back to
using a compatible apk.

When using the check_abi parameter it no longer relies on naming convention
and only allows apks with native code supporting a devices primary abi to be
used.

Updated the relevant documentation.
2016-12-09 14:11:16 +00:00
Marc Bonnici
f467f6f991 AndroidDevice: Added method to retrive primary abi of installed package
Tries to retireve the primary abi of a currently installed package on
device.
2016-12-09 14:11:16 +00:00
Marc Bonnici
4352e02806 APK Info: Added property to extract native code from an apk
Looks for any native code present in an apk and stores the
mapped abi(s) for use when selecting appropriate apks.
2016-12-09 14:11:16 +00:00
Marc Bonnici
693d0544a3 ABI_MAP: Added armeabi-v7a as armeabi 2016-12-09 14:11:16 +00:00
Marc Bonnici
eb239c65d0 LinuxDevice: Added supported eabi property.
Added property to linux device to return list of supported abis
to be consistent with android devices. Currently only returns a
list containing the primary abi.
2016-12-09 14:11:16 +00:00
Michael McGeagh
8cbf189029 skype: longer wait time for signin and search 2016-12-09 13:45:35 +00:00
Marc Bonnici
b6f770cfc5 AdobeReader: Updated workload for new app version
Updated the workload to work with the latest version (v16.4)
of Adobe Reader app.
2016-12-08 18:14:16 +00:00
setrofim
c7de8cabd6 Merge pull request #294 from jimboatarm/improve-geekbench-timeouts
geekbench: Improve UiAutomator timeouts for slower devices
2016-12-02 17:04:01 +00:00
setrofim
fd7df36a5a Merge pull request #301 from jimboatarm/skype-hangup
skype: now hangs up at the end. Better duration handling
2016-12-02 17:03:13 +00:00
setrofim
607187ad5b Merge pull request #306 from marcbonnici/revent_BC
Revent: Fixed backwards compatibility with v1
2016-12-02 17:02:06 +00:00
setrofim
b4036c5f15 Merge pull request #304 from jimboatarm/uxperf-fix
UXPERF: handle error when only one timestamp is logged
2016-12-02 16:58:33 +00:00
Marc Bonnici
b6e077c06b Revent: Fixed backwards compatibility with v1.
Fixed new gamemode not being set for compatibility mode.
Fixed input buffer not being flushed causing replay to fail.
Fixed infinte loop if revent recording did not end correctly.
Updated revent binaries.
2016-12-02 16:58:09 +00:00
Michael McGeagh
8b82451230 util-fps: handle divide by zero possibility 2016-12-02 16:34:32 +00:00
Michael McGeagh
64c352fab6 UXPERF: handle error when only one timestamp is logged 2016-12-02 16:31:45 +00:00
Michael McGeagh
ae4ae3da5e skype: now hangs up at the end. Better duration handling 2016-12-02 11:30:30 +00:00
Michael McGeagh
393abc267f GooglePhotos: Navigate up only when needed. Use baseclass' selectGalleryFolder 2016-12-02 11:28:38 +00:00
setrofim
751970f991 Merge pull request #302 from marcbonnici/master
Revent: Added updated revent binaries.
2016-12-02 10:31:03 +00:00
Marc Bonnici
254e9fff38 Revent: Added updated revent binaries.
Added updated revent binaries for armabi and arm64 for the fixes
in commit 7a19046645
2016-12-02 09:55:19 +00:00
James Hartley
088709f290 geekbench: Improve UiAutomator timeouts for slow devices, fixes for rooted devices
Tested by running Mate 8 on the lowest cluster at the lowest available frequency

On rooted devices, skip attempting to pull log files from device.  This allows
the benchmark to run to completion without failure.

GB log files are stored in /data/data/com.primatelabs.geekbench/files and
not accessible without root.  On Chromebooks ( which have no adb root
rights) it is possible to copy from this folder manually after the test run.
2016-12-02 07:23:44 +00:00
Sergei Trofimov
850fcb24ab pylint/pep8 fixes 2016-12-01 15:01:45 +00:00
Michael McGeagh
cb53fe9ec8 Gmail: A workload to perform standard productivity tasks within Gmail. The workload carries out various tasks, such as creating new emails, attaching images and sending them.
Moved broadcast to super. Mandatory and Default are XOR

Added a longer wait for sync to finish. Increases reliability on certain phones

Changed recipient to not mandatory and a default set

Wait for sync when launching gmail from the sharing feature

Fix: cornercase where image viewer already points to working directory. Refactored code due to duplication

Added new function to BaseUiAutomation class to find a folder in the gallery
2016-12-01 13:21:49 +00:00
setrofim
4e161127e1 Merge pull request #300 from setrofim/master
A couple of fixes for trace CPU power state post-processing and energy model scripts
2016-11-30 09:38:05 +00:00
setrofim
bf43bf93bc Merge pull request #298 from marcbonnici/revent_fixes
Revent: Fixes
2016-11-29 17:34:17 +00:00
Marc Bonnici
7a19046645 Revent: Fixes
Changed termination signal to interrupt signal to prevent code exiting too early.
Added exit handler to ensure revent exits correctly as previously was
crashing and therefore not running final code.
Fixed error in writing input event where half of timestamp seconds was missing.
Fixed typo in documentation for revent file structure.
2016-11-29 17:29:33 +00:00
Sergei Trofimov
5db11462be utils/power: handle devlib frequency reporting
When collecting ftrace events, the instrument will insert frequencies
into the trace to make it possible reconstruct power states when there
were no frequency transitions during the measured period.

The format in which frequencies are inserted is different in devlib.
Since post-processing scripts may be run on traces collected by devlib
as well as WA, it needs to support both formats.
2016-11-29 10:23:32 +00:00
Sergei Trofimov
7bf0e3c344 energy_model: only set "ui" parameter for ChomeOS
Fixed a bug, where "ui" runtime parameter was being set regardless of
the underlying platforms for "freq" iterations, causing the to fail on
non-ChomeOS devices.
2016-11-29 10:23:32 +00:00
setrofim
d16d8bf62d Merge pull request #297 from marcbonnici/octaned
ResourceGetters: Updated executable discovery to use http_getter
2016-11-28 11:45:46 +00:00
Marc Bonnici
c93cc81aac ResourceGetters: Updated executable discovery to use http_getter
Executables were not previously automatically downloaded from a
provided remote_assets_url. The http_getter is now called to attempt to
find and download executable files.
2016-11-28 11:38:29 +00:00
setrofim
a172c8f624 Merge pull request #295 from jimboatarm/fix-octaned8-log-appending
Fix octaned8 results capture
2016-11-28 11:04:42 +00:00
James Hartley
f69e4c5b18 Fix octaned8 results capture
Ensure the device log file is deleted between runs and that the host file is not kept as an
artefact.  Both can lead to errors appending data from the next iteration/run.
2016-11-24 14:54:19 +00:00
James Hartley
5b543d2edf Fix exact_apk_version check 2016-11-24 12:47:57 +00:00
setrofim
40d281b336 Merge pull request #283 from jimboatarm/octaned8
Add Octane d8 workload
2016-11-11 17:32:53 +00:00
Michael McGeagh
8edce40301 FPS percentiles: Ignore ValueError when NaN
Fixed: if index isnt zero based, drop(0) will fail
2016-11-11 17:21:47 +00:00