1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

947 Commits

Author SHA1 Message Date
setrofim
3782a33060 Merge pull request #226 from ep1cman/version_check_fixes
Version check fixes
2016-08-24 16:01:18 +01:00
Sebastian Goscik
8e27794124 Added a script to check APK/Workload versions for inconsistencies 2016-08-24 15:48:22 +01:00
Sebastian Goscik
9d4aa4983a antutu: Fixed setting permissions
It has been observed on some devices that the FINE_LOCATION premissions is required
for antutu to run without asking for permissions at run time but this was not listed
in the APK manifest. This caused issues on devices were only the permissions in the
manifest can be granted. This commit sliences any error when trying to set this permission
as well as only trying only on Android 6+
2016-08-24 14:36:00 +01:00
Sebastian Goscik
b426e00f2f Updated workload versions to match APK files
Some workloads presented a different version than what was in the APK file.
With the changes introduced in bb33123 several workloads broke.
2016-08-24 14:33:18 +01:00
Sebastian Goscik
07d34e5615 ApkWorkload: Moved APK resolution into setup
Previously if you had multiple versions of the same workload in one
agenda only the first one would work, the others would fail to find
their APK.
2016-08-24 14:28:53 +01:00
Sebastian Goscik
b1ae5a5465 Merge pull request #224 from setrofim/master
ssh: fixing rasing of CalledProcessErrorWithStderr
2016-08-23 17:26:07 +01:00
Sergei Trofimov
4ea4bc8631 ssh: fixing rasing of CalledProcessErrorWithStderr
CalledProcessErrorWithStderr is a subclass of CalledProcessError that
also takes stderr output. Both that and normal output must be passed as
keyword arguments. They were being passed as keyword arguments inside
_scp() of SshConnection, causing cryptic errors to appear.

Additionally, "output" was not being properly poped off before invoking
super's init.
2016-08-23 17:22:02 +01:00
Sebastian Goscik
fe259dca05 Merge pull request #216 from jimboatarm/googleplay_uxperf
Add UxPerfUiAutomation and googleplaybooks workload
2016-08-09 13:49:02 +01:00
John Richardson
86f3066f56 Add unsupported package check to AndroidUiAutoBenchmark
Add method to automatically check against a dictionary of known package
versions that don't work well with AndroidUiAutoBenchmark workloads.
Raises an exception if found.
2016-08-09 12:08:28 +01:00
John Richardson
0f579e18b3 Add googleplaybooks workload 2016-08-09 12:08:20 +01:00
John Richardson
25172fb027 Add UxPerfUiAutomation class
UxPerfUiAutomation contains methods specific to UX performance testing.
2016-08-09 09:56:04 +01:00
John Richardson
550a0db61a Rename dumpsys_enabled parameter to markers_enabled
Change parameter name for enabling markers to better reflect its
purpose. The old name was a misnomer.
2016-08-09 09:56:04 +01:00
John Richardson
73c2609a72 Fix message regex in uxperf result processor
The logcat output differs between devices. Modify the regex pattern to
accommodate different output formats when matching UX_PERF markers.
2016-08-09 09:56:04 +01:00
setrofim
1ec7961b0e Merge pull request #210 from drcef/master
Implemented visual state detection functionality for revent workloads
2016-08-05 11:57:59 +01:00
George Psimenos
01f2a5f412 Implemented visual state detection functionality for revent workloads
- Added statedetect.py in utils which is a standalone module that
    contains all the methods needed for state detection

  - Modified the setup() and run() methods of the GameWorkload class
    in common/android/workload.py to have a parameter that enables
    state checks and run the check after setup and run if requested.

State detection uses the template matching method available in
OpenCV to determine the state of the workload by detecting
predefined unique elements on a screenshot from the device.
2016-08-05 10:09:18 +01:00
setrofim
480a054860 Merge pull request #220 from ep1cman/master
RunConfiguration: Fixed disabling of instruments in workload specs
2016-08-04 16:18:52 +01:00
Sebastian Goscik
e9ba9352a6 RunConfiguration: Fixed disabling of instruments in workload specs 2016-08-04 16:16:53 +01:00
Sebastian Goscik
0a3ff099c0 Merge pull request #219 from jimboatarm/mdzj/upstream-get-assets
Add script to get external assets for workloads
2016-08-04 15:46:16 +01:00
muendelezaji
75cc5854bf Add script to get external assets for workloads 2016-08-04 13:59:12 +01:00
Sebastian Goscik
77aaa0b849 Merge pull request #217 from jimboatarm/dump_hierarchy_uxperf
Dump hierarchy view on error
2016-08-03 09:57:09 +01:00
John Richardson
0945dd6ba4 Dump hierarchy view on error
Dump window hierarchy view from uiautomator to a file when WA fails
during execution. Note: the xml file are pre-formatted after dump.
Implementation specific to android.device.
2016-08-02 16:47:24 +01:00
setrofim
4c94ba43ac Merge pull request #215 from jimboatarm/remove-old-apk-permissions
permission grant for API 23 and above
2016-07-29 16:33:59 +01:00
muendelezaji
efae2e8c32 Android permission grant for API 23 and above
Issue: On some devices, _grant_requested_permissions may throw an
error when trying to grant permissions that were already granted
at install time using 'adb install -g'.

Fix: Surround permission grant method for API 23+ with try/except

Issue: Currently, _grant_requested_permissions skips all lines after
the first line not starting with 'android.permission'. This causes
a problem for apps where some required permissions appear after a
non-matching line e.g. Google Slides.

Fix: Don't break on non-matching line until end of section is reached
2016-07-29 16:27:29 +01:00
Sebastian Goscik
59874b862d Merge pull request #212 from jimboatarm/fps_util_uxperf
Additional changes to FpsProcessor, fps instrument and uxperf result processor
2016-07-27 14:56:14 +01:00
John Richardson
da19859c25 Add logging and exception handling to uxperf result processor
The uxperf result processor now provides warnings for unmatched UX_PERF
markers when running the fps instrument. Previously unmatched markers
resulted in an exception being thrown. Includes additional logging for
debugging purposes.
2016-07-27 14:50:24 +01:00
Sebastian Goscik
d87e425c24 Merge pull request #214 from jimboatarm/network_check_uxperf
Add network connectivity check for devices
2016-07-27 11:59:57 +01:00
John Richardson
2872080d1a Add network connectivity check for devices
Add a general check that pings a IP address rather than checking the
status of dumpsys wifi due to the fact that not all devices are
connected via wifi. Intended for workloads that require an internet
connection to operate.
2016-07-26 16:30:06 +01:00
John Richardson
625a3a39a5 Add dumpsys_period parameter to fps instrument
Add a new parameter to fps instrument to specify the time period between
calls to ``dumpsys SurfaceFlinger --latency`` in seconds when collecting
frame data. A lower value improves the granularity of timings when
recording actions for UX Performance metrics.
2016-07-26 11:30:34 +01:00
John Richardson
aa2d187c4d Fix for FpsProcessor logic in utils module
- Add requirement on filtered_vsyncs_to_compose for total_vsync metric
- Remove misleading comment in class description
2016-07-26 11:29:51 +01:00
Sebastian Goscik
b80e5dc52e Merge pull request #213 from jimboatarm/baselib_uxperf
Extensions to baselib for UX Performance
2016-07-26 11:28:57 +01:00
John Richardson
2208d45bfb Add further convenience methods to BaseUiAutomation
Convenience methods to make it easier to get UiObjects and perform
UiDevice based gestures and operations.
2016-07-26 10:34:04 +01:00
John Richardson
51e4e71931 Upgrade Android API level from 17 to 18
Upgrade API level for BaseUiAutomation from 17 -> 18 and update
dependent workloads accordingly.
2016-07-26 10:30:15 +01:00
John Richardson
0388fa6f36 Minor code maintenance for base class BaseUiAutomation
- Rename inconsistently named variables
- Avoid long wrapped lines where possible
- Fix whitespace around operators
2016-07-26 08:59:17 +01:00
setrofim
ee7c04a568 Merge pull request #209 from ep1cman/revent_fixes
Pylint fixes
2016-07-21 16:47:14 +01:00
Sebastian Goscik
9707aa6237 Pylint fixes
Pylint now checks for trailing new lines, this commit fixes them.
2016-07-21 16:40:26 +01:00
setrofim
019ee34c0d Merge pull request #207 from ep1cman/revent_fixes
Revent fixes
2016-07-21 15:43:55 +01:00
Sebastian Goscik
873bdf0bc7 revent: replay and record fix + reorganisation
The two commands now always copy over revent.

Reoraganised the commands to use a common base class.
2016-07-21 15:42:12 +01:00
Sebastian Goscik
54c409ce6f revent: Fixed magic check
A null character was not being added to the end of the string.
2016-07-21 15:42:11 +01:00
setrofim
a2d0747b4c Merge pull request #200 from ep1cman/revent_fixes
revent: Fixed dump command segfault
2016-07-19 16:54:53 +01:00
Sebastian Goscik
25eac432c9 revent: Added revent file structure to documentation 2016-07-19 16:52:12 +01:00
Sebastian Goscik
dd61f99785 revent: Updated resource getters to check if a file is actually a recording 2016-07-19 16:48:13 +01:00
Sebastian Goscik
164f207084 revent: Timeout is now based on recording duration
The revent file is now parsed and the duration of the recording is calculated.
This duration + 30 seconds is now used for the timeout for revent.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
cb01b0c9a9 utils/revent: Added revent recording parser
revent binary recordings can now be parsed and used within WA.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
139a0698c9 revent: Added "magic" and file version to revent files.
revent files should now start with "REVENT" followed by the file format
version.
2016-07-19 16:48:13 +01:00
Sebastian Goscik
259b813a96 revent: Various fixes.
revent:
    - Fixed 32-bit/64-bit compatibility by no longer "long" for timestamps
    - Removed superfluous code
    - SIGTERM is now handled only while waiting for a file not while processing one
    - Added '-s' to docs
    - Fixed path_buff size

Record Command:

    - Removed timeout in command as -s is specified.
    - Previously the command would send SIGTERM to revent but not wait for it to terminate.
      This would result in the pulled recording missing its send. This has now been fixed.

Replay Command:
    - Added more logging
2016-07-19 16:11:42 +01:00
Sebastian Goscik
299b28b3c1 Merge pull request #203 from setrofim/master
resource getters: weaken File to also resove to directories
2016-07-15 16:07:49 +02:00
Sergei Trofimov
ece33c1d68 resource getters: weaken File to also resove to directories
Because UNIX tells us that "everything is a file".
2016-07-15 15:05:19 +01:00
Sebastian Goscik
f68cf4e317 Merge pull request #201 from jimboatarm/marker_api
Add per-action instrumentation for UX performance
2016-07-15 11:15:46 +02:00
John Richardson
c49c5c4121 Add per-action instrumentation for UX performance
- Implement a new Marker API in BaseUiAutomation so workload can
  generate start and end markers with string name. Outputs to logcat.

- Document the Marker output log format in the WA documentation

- Create a results processor to take existing instrument fps logs and
  parse them based on the workload markers. Produce per-action fps
  metrics.

- Add simple timing results based on the workload markers
2016-07-14 13:49:39 +01:00
John Richardson
b8d7956d4c Move processing logic in fps instrument to utility file
Processing logic for frame statistics can be moved out of fps instrument
to a new utility file. This will allow result processors to use the same
logic to produce frame statistics on a subsection of the data
produced by the fps instrument.
2016-07-14 13:49:39 +01:00