1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

636 Commits

Author SHA1 Message Date
John Richardson
5ef7d2dd44 Remove skypevideo workload
The original skypevideo workload has now been replaced with the newer
skype workload.
2016-09-07 16:51:40 +01:00
John Richardson
cf8cb5bfab Add Skype workload 2016-09-07 16:45:11 +01:00
muendelezaji
9376c6875b ApkWorkload extension support
- Allow disabling main activity launch in setup (required for some apps)
- Parameterise clear data on reset (default behaviour unchanged)
2016-09-07 16:45:07 +01:00
setrofim
a6347f5833 Merge pull request #240 from ep1cman/master
Workloads: Fixed issues with calling super
2016-09-06 10:16:08 +01:00
Sebastian Goscik
38a7e01e83 Workloads: Fixed issues with calling super
Super works by having a iterator of parent classes to ensure each is called
once and only once. WA calls some parent methods in classes with multiple
inheritenceconditionally so calls them directly instread. This breaks super
which ends up calling some methods multiple times.

To work around this until workloads are reworked to use composition rather than
inheritance, all classes that that subclass `Workload` directly no longer use
`super`.
2016-09-06 10:11:09 +01:00
muendelezaji
441ba974b7 Youtube workload 2016-09-05 15:03:53 +01:00
setrofim
3d610788a3 Merge pull request #238 from jimboatarm/check_app_version_uxperf
Add check_app_version method to ApkWorkload
2016-09-02 11:59:08 +01:00
John Richardson
1986511ae8 Add check_app_version method to ApkWorkload
Implement new method to enforce that a valid apk version is used to run
the workload. Based on a min and max range of apk versions tested
during development.
2016-09-02 11:56:40 +01:00
muendelezaji
392a3f1600 Add network check methods to Device and Workload classes
- Device subclasses should provide their own implementation.
    Default behaviour is to raise a `NotImplementedError`
  - Workload subclasses can set `requires_network` to `True` and
    network connectivity check will be performed during `setup()`
2016-09-02 10:33:18 +01:00
John Richardson
502b0ed4b3 Add package name to uiautomator params in AndroidUxPerfWorkload
The package name is common to all workloads that inherit from this
class and is used when locating UI elements.
2016-08-31 14:44:21 +01:00
John Richardson
cab9d918ab Move uiautomator params in AndroidUxPerfWorkload
uiautomator parameters are set per instance of a workload and not per
iteration. Move uiautomator parameter assignment from setup() to
validate().
2016-08-31 14:43:33 +01:00
John Richardson
b510b31052 Add broadcast_media_mounted method to android device
New method to force a re-index of the mediaserver cache for the
specified directory. Used in workloads that require external media files
as dependencies.
2016-08-31 10:24:06 +01:00
Sebastian Goscik
7713f02252 Execution: Added a clean_up global config to delete WA files from devices
Adds a WA configuration point `clean_up` that will delete the WA binaries
directory  and the WA working directory from a device at the end of a WA run.
2016-08-26 17:25:11 +01:00
Sebastian Goscik
0a2afdfd84 AndroidDevice: Added -rf to delete_files
This allows it to delete folders and makes it consistend with LinuxDevice.
2016-08-26 17:13:21 +01:00
John Richardson
67f418f79f Add new AndroidUxPerfWorkload class
Create a new workload class to encapsulate functionality common to all
uxperf workloads.
2016-08-26 12:22:39 +01:00
Sebastian Goscik
f57dd83d1a pylint fixes 2016-08-25 14:20:10 +01:00
Sebastian Goscik
9d4aa4983a antutu: Fixed setting permissions
It has been observed on some devices that the FINE_LOCATION premissions is required
for antutu to run without asking for permissions at run time but this was not listed
in the APK manifest. This caused issues on devices were only the permissions in the
manifest can be granted. This commit sliences any error when trying to set this permission
as well as only trying only on Android 6+
2016-08-24 14:36:00 +01:00
Sebastian Goscik
b426e00f2f Updated workload versions to match APK files
Some workloads presented a different version than what was in the APK file.
With the changes introduced in bb33123 several workloads broke.
2016-08-24 14:33:18 +01:00
Sebastian Goscik
07d34e5615 ApkWorkload: Moved APK resolution into setup
Previously if you had multiple versions of the same workload in one
agenda only the first one would work, the others would fail to find
their APK.
2016-08-24 14:28:53 +01:00
Sergei Trofimov
4ea4bc8631 ssh: fixing rasing of CalledProcessErrorWithStderr
CalledProcessErrorWithStderr is a subclass of CalledProcessError that
also takes stderr output. Both that and normal output must be passed as
keyword arguments. They were being passed as keyword arguments inside
_scp() of SshConnection, causing cryptic errors to appear.

Additionally, "output" was not being properly poped off before invoking
super's init.
2016-08-23 17:22:02 +01:00
John Richardson
86f3066f56 Add unsupported package check to AndroidUiAutoBenchmark
Add method to automatically check against a dictionary of known package
versions that don't work well with AndroidUiAutoBenchmark workloads.
Raises an exception if found.
2016-08-09 12:08:28 +01:00
John Richardson
0f579e18b3 Add googleplaybooks workload 2016-08-09 12:08:20 +01:00
John Richardson
25172fb027 Add UxPerfUiAutomation class
UxPerfUiAutomation contains methods specific to UX performance testing.
2016-08-09 09:56:04 +01:00
John Richardson
550a0db61a Rename dumpsys_enabled parameter to markers_enabled
Change parameter name for enabling markers to better reflect its
purpose. The old name was a misnomer.
2016-08-09 09:56:04 +01:00
John Richardson
73c2609a72 Fix message regex in uxperf result processor
The logcat output differs between devices. Modify the regex pattern to
accommodate different output formats when matching UX_PERF markers.
2016-08-09 09:56:04 +01:00
setrofim
1ec7961b0e Merge pull request #210 from drcef/master
Implemented visual state detection functionality for revent workloads
2016-08-05 11:57:59 +01:00
George Psimenos
01f2a5f412 Implemented visual state detection functionality for revent workloads
- Added statedetect.py in utils which is a standalone module that
    contains all the methods needed for state detection

  - Modified the setup() and run() methods of the GameWorkload class
    in common/android/workload.py to have a parameter that enables
    state checks and run the check after setup and run if requested.

State detection uses the template matching method available in
OpenCV to determine the state of the workload by detecting
predefined unique elements on a screenshot from the device.
2016-08-05 10:09:18 +01:00
setrofim
480a054860 Merge pull request #220 from ep1cman/master
RunConfiguration: Fixed disabling of instruments in workload specs
2016-08-04 16:18:52 +01:00
Sebastian Goscik
e9ba9352a6 RunConfiguration: Fixed disabling of instruments in workload specs 2016-08-04 16:16:53 +01:00
muendelezaji
75cc5854bf Add script to get external assets for workloads 2016-08-04 13:59:12 +01:00
John Richardson
0945dd6ba4 Dump hierarchy view on error
Dump window hierarchy view from uiautomator to a file when WA fails
during execution. Note: the xml file are pre-formatted after dump.
Implementation specific to android.device.
2016-08-02 16:47:24 +01:00
muendelezaji
efae2e8c32 Android permission grant for API 23 and above
Issue: On some devices, _grant_requested_permissions may throw an
error when trying to grant permissions that were already granted
at install time using 'adb install -g'.

Fix: Surround permission grant method for API 23+ with try/except

Issue: Currently, _grant_requested_permissions skips all lines after
the first line not starting with 'android.permission'. This causes
a problem for apps where some required permissions appear after a
non-matching line e.g. Google Slides.

Fix: Don't break on non-matching line until end of section is reached
2016-07-29 16:27:29 +01:00
Sebastian Goscik
59874b862d Merge pull request #212 from jimboatarm/fps_util_uxperf
Additional changes to FpsProcessor, fps instrument and uxperf result processor
2016-07-27 14:56:14 +01:00
John Richardson
da19859c25 Add logging and exception handling to uxperf result processor
The uxperf result processor now provides warnings for unmatched UX_PERF
markers when running the fps instrument. Previously unmatched markers
resulted in an exception being thrown. Includes additional logging for
debugging purposes.
2016-07-27 14:50:24 +01:00
Sebastian Goscik
d87e425c24 Merge pull request #214 from jimboatarm/network_check_uxperf
Add network connectivity check for devices
2016-07-27 11:59:57 +01:00
John Richardson
2872080d1a Add network connectivity check for devices
Add a general check that pings a IP address rather than checking the
status of dumpsys wifi due to the fact that not all devices are
connected via wifi. Intended for workloads that require an internet
connection to operate.
2016-07-26 16:30:06 +01:00
John Richardson
625a3a39a5 Add dumpsys_period parameter to fps instrument
Add a new parameter to fps instrument to specify the time period between
calls to ``dumpsys SurfaceFlinger --latency`` in seconds when collecting
frame data. A lower value improves the granularity of timings when
recording actions for UX Performance metrics.
2016-07-26 11:30:34 +01:00
John Richardson
aa2d187c4d Fix for FpsProcessor logic in utils module
- Add requirement on filtered_vsyncs_to_compose for total_vsync metric
- Remove misleading comment in class description
2016-07-26 11:29:51 +01:00
John Richardson
2208d45bfb Add further convenience methods to BaseUiAutomation
Convenience methods to make it easier to get UiObjects and perform
UiDevice based gestures and operations.
2016-07-26 10:34:04 +01:00
John Richardson
51e4e71931 Upgrade Android API level from 17 to 18
Upgrade API level for BaseUiAutomation from 17 -> 18 and update
dependent workloads accordingly.
2016-07-26 10:30:15 +01:00
John Richardson
0388fa6f36 Minor code maintenance for base class BaseUiAutomation
- Rename inconsistently named variables
- Avoid long wrapped lines where possible
- Fix whitespace around operators
2016-07-26 08:59:17 +01:00
Sebastian Goscik
9707aa6237 Pylint fixes
Pylint now checks for trailing new lines, this commit fixes them.
2016-07-21 16:40:26 +01:00
Sebastian Goscik
873bdf0bc7 revent: replay and record fix + reorganisation
The two commands now always copy over revent.

Reoraganised the commands to use a common base class.
2016-07-21 15:42:12 +01:00
Sebastian Goscik
54c409ce6f revent: Fixed magic check
A null character was not being added to the end of the string.
2016-07-21 15:42:11 +01:00
Sebastian Goscik
dd61f99785 revent: Updated resource getters to check if a file is actually a recording 2016-07-19 16:48:13 +01:00
Sebastian Goscik
164f207084 revent: Timeout is now based on recording duration
The revent file is now parsed and the duration of the recording is calculated.
This duration + 30 seconds is now used for the timeout for revent.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
cb01b0c9a9 utils/revent: Added revent recording parser
revent binary recordings can now be parsed and used within WA.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
139a0698c9 revent: Added "magic" and file version to revent files.
revent files should now start with "REVENT" followed by the file format
version.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
259b813a96 revent: Various fixes.
revent:
    - Fixed 32-bit/64-bit compatibility by no longer "long" for timestamps
    - Removed superfluous code
    - SIGTERM is now handled only while waiting for a file not while processing one
    - Added '-s' to docs
    - Fixed path_buff size

Record Command:

    - Removed timeout in command as -s is specified.
    - Previously the command would send SIGTERM to revent but not wait for it to terminate.
      This would result in the pulled recording missing its send. This has now been fixed.

Replay Command:
    - Added more logging
2016-07-19 16:11:42 +01:00
Sergei Trofimov
ece33c1d68 resource getters: weaken File to also resove to directories
Because UNIX tells us that "everything is a file".
2016-07-15 15:05:19 +01:00