mirror of
https://github.com/ARM-software/workload-automation.git
synced 2025-01-31 02:01:16 +00:00
Documentation: Updated change log for new version.
This commit is contained in:
Marc Bonnici
parent
30e9b553ff
commit
d6ab68bffc
@ -1,6 +1,228 @@
|
|||||||
=================================
|
=================================
|
||||||
What's New in Workload Automation
|
What's New in Workload Automation
|
||||||
=================================
|
=================================
|
||||||
|
-------------
|
||||||
|
Version 2.6.0
|
||||||
|
-------------
|
||||||
|
|
||||||
|
.. note:: Users who are currently using the GitHub master version of WA should
|
||||||
|
uninstall the existing version before upgrading to avoid potential issues.
|
||||||
|
|
||||||
|
Additions:
|
||||||
|
##########
|
||||||
|
|
||||||
|
Workloads
|
||||||
|
~~~~~~~~~
|
||||||
|
- ``AdobeReader``: A workload that carries out following typical productivity
|
||||||
|
tasks. These include opening a file, performing various gestures and
|
||||||
|
zooms on screen and searching for a predefined set of strings.
|
||||||
|
- ``octaned8``: A workload to run the binary (non-browser) version of the JS
|
||||||
|
benchmark Octane.
|
||||||
|
- ``GooglePlayBooks``: A workload to perform standard productivity tasks with
|
||||||
|
Google Play Books. This workload performs various tasks, such as searching
|
||||||
|
for a book title online, browsing through a book, adding and removing notes,
|
||||||
|
word searching, and querying information about the book.
|
||||||
|
- ``GooglePhotos``: A workload to perform standard productivity tasks with
|
||||||
|
Google Photos. Carries out various tasks, such as browsing images,
|
||||||
|
performing zooms, and post-processing the image.
|
||||||
|
- ``GoogleSlides``: Carries out various tasks, such as creating a new
|
||||||
|
presentation, adding text, images, and shapes, as well as basic editing and
|
||||||
|
playing a slideshow.
|
||||||
|
- ``Youtube``: The workload plays a video, determined by the ``video_source``
|
||||||
|
parameter. While the video is playing, some common actions such as video
|
||||||
|
seeking, pausing playback and navigating the comments section are performed.
|
||||||
|
- ``Skype``: Replacement for the ``skypevideo`` workload. Logs into Skype
|
||||||
|
and initiates a voice or video call with a contact.
|
||||||
|
|
||||||
|
Framework
|
||||||
|
~~~~~~~~~
|
||||||
|
- ``AndroidUxPerfWorkload``: Added a new workload class to encapsulate
|
||||||
|
functionality common to all uxperf workloads.
|
||||||
|
- ``UxPerfUiAutomation``: Added class which contains methods specific to
|
||||||
|
UX performance
|
||||||
|
testing.
|
||||||
|
- ``get-assets``: Added new script and command to retrieve external assets
|
||||||
|
for workloads
|
||||||
|
|
||||||
|
Results Processors
|
||||||
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
- ``uxperf``: Parses device logcat for `UX_PERF` markers to produce performance
|
||||||
|
metrics for workload actions using specified instrumentation.
|
||||||
|
|
||||||
|
Other
|
||||||
|
~~~~~
|
||||||
|
- ``State Detection``: Added feature to use visual state detection to
|
||||||
|
verify the state of a workload after setup and run.
|
||||||
|
|
||||||
|
|
||||||
|
Fixes/Improvements:
|
||||||
|
###################
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
- ``Revent``: Added file structure to the documentation.
|
||||||
|
- Clarified documentation regarding binary dependencies.
|
||||||
|
- Updated documentation with ``create`` and ``get-assets`` commands.
|
||||||
|
|
||||||
|
Instruments
|
||||||
|
~~~~~~~~~~~~
|
||||||
|
- ``sysfs_extractor``: Fixed error when `tar.gz` file already existed on device,
|
||||||
|
now overwrites.
|
||||||
|
- ``cpufreq``: Fixed error when `tar.gz` file already existed on device, now
|
||||||
|
overwrites.
|
||||||
|
- ``file-poller``:
|
||||||
|
- Improved csv output.
|
||||||
|
- Added error checking and reporting.
|
||||||
|
- Changed ``files`` to be a mandatory parameter.
|
||||||
|
- ``fps``:
|
||||||
|
- Added a new parameter to fps instrument to specify the time period between
|
||||||
|
calls to ``dumpsys SurfaceFlinger --latency`` when collecting frame data.
|
||||||
|
- Added gfxinfo methods to obtain fps stats. Auto detects and uses appropriate
|
||||||
|
method via android version of device.
|
||||||
|
- Fixed issue with regex.
|
||||||
|
- Now handles empty frames correctly.
|
||||||
|
- ``energy_model``: Ensures that the ``ui`` runtime parameter is only set for
|
||||||
|
ChromeOS devices.
|
||||||
|
- ``ftrace``: Added support to handle traces collected by both WA and devlib.
|
||||||
|
- ``Perf``: Updated 32bit binary file for little endian devices.
|
||||||
|
|
||||||
|
Resource Getters
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
- ``http_getter``: Now used to try and find executables files from a
|
||||||
|
provided ``remove_assets_url``.
|
||||||
|
|
||||||
|
Result Processors
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
- ``cpu_states``: Fixes using stand-alone script with timeline option.
|
||||||
|
|
||||||
|
Workloads
|
||||||
|
~~~~~~~~~
|
||||||
|
- ``antutu``: Fixed setting permissions of ``FINE_LOCATION`` on some devices.
|
||||||
|
- ``bbench`` Fixed handling of missing results.
|
||||||
|
- ``camerarecord``:
|
||||||
|
- Added frame stats collection through dumpsys gfxinfo.
|
||||||
|
- Added possibility to select slow_motion recording mode.
|
||||||
|
- ``Geekbench``:
|
||||||
|
- Fixed output file listing causing pull failure.
|
||||||
|
- Added support for Geekbench 4.
|
||||||
|
- ``recentfling``:
|
||||||
|
- Fixed issue when binaries were not uninstalled correctly.
|
||||||
|
- Scripts are now deployed via ``install()`` to ensure they are executable.
|
||||||
|
- Fixed handling of when a PID file is deleted before reaching processing
|
||||||
|
results stage.
|
||||||
|
- Added parameter to not start any apps before flinging.
|
||||||
|
- ``rt-app``: Added camera recorder simulation.
|
||||||
|
- ``sysbench``: Added arm64 binary.
|
||||||
|
- ``Vellamo``: Fixed capitalization in part of UIAutomation to prevent
|
||||||
|
potential issues.
|
||||||
|
- ``Spec2000``: Now uses WA deployed version of busybox.
|
||||||
|
- ``NetStat``: Updated to support new default logcat format in Android 6.
|
||||||
|
- ``Dex2oat``: Now uses root if available.
|
||||||
|
|
||||||
|
Framework
|
||||||
|
~~~~~~~~~
|
||||||
|
- ``adb_shell``:
|
||||||
|
- Fixed issue when using single quoted command with ``adb_shell``.
|
||||||
|
- Correctly forward stderror to the caller for newer version of adb.
|
||||||
|
- ``revent``
|
||||||
|
- Added ``-S`` argument to "record" command to automatically record a
|
||||||
|
screen capture after a recording is completed.
|
||||||
|
- Fixed issue with multiple iterations of a revent workload.
|
||||||
|
- Added ``-s`` option to executable to allow waiting on stdin.
|
||||||
|
- Removed timeout in command as ``-s`` is specified.
|
||||||
|
- Revent recordings can now be parsed and used within WA.
|
||||||
|
- Fixed issue when some recordings wouldn't be retrieved correctly.
|
||||||
|
- Timeout is now based on recording duration.
|
||||||
|
- Added `magic` and file version to revent files. Revent files should now
|
||||||
|
start with ``REVENT`` followed by the file format version.
|
||||||
|
- Added support for gamepad recording. This type of recording contains
|
||||||
|
only the events from a gamepad device (which is automatically
|
||||||
|
identified).
|
||||||
|
- A ``mode`` field has been added to the recording format to help
|
||||||
|
distinguish between the normal and gamepad recording types.
|
||||||
|
- Added ``-g`` option to ``record`` command to expose the gamepad recording
|
||||||
|
mode.
|
||||||
|
- The structure of revent code has undergone a major overhaul to improve
|
||||||
|
maintainability and robustness.
|
||||||
|
- More detailed ``info`` command output.
|
||||||
|
- Updated Makefile to support debug/production builds.
|
||||||
|
- ``Android API``: Upgraded Android API level from 17 to 18.
|
||||||
|
- ``uiautomator``: The window hierarchy is now dumped to a file when WA fails
|
||||||
|
on android devices.
|
||||||
|
- ``AndroidDevice``:
|
||||||
|
- Added support for downgrading when installing an APK.
|
||||||
|
- Added a ``broadcast_media_mounted`` method to force a re-index of the
|
||||||
|
mediaserver cache for a specified directory.
|
||||||
|
- Now correctly handles ``None`` output for ``get_pids_of()`` when there are no
|
||||||
|
running processes with the specified name.
|
||||||
|
- Renamed the capture method from ``capture_view_hierachy`` to
|
||||||
|
``capture_ui_hierarchy``.
|
||||||
|
- Changed the file extension of the capture file to ``.uix``
|
||||||
|
- Added ``-rf`` to delete_files to be consistent with ``LinuxDevice``.
|
||||||
|
- ``LinuxDevice``: Now ensures output from both stdout and etderr is propagated in
|
||||||
|
the event of a DeviceError.
|
||||||
|
- ``APKWorkload``:
|
||||||
|
- Now ensure APKs are replaced properly when reinstalling.
|
||||||
|
- Now checks APK version and ABI when installing.
|
||||||
|
- Fixed error on some devices when trying to grant permissions that were
|
||||||
|
already granted.
|
||||||
|
- Fixed some permissions not being granted.
|
||||||
|
- Now allows disabling the main activity launch in setup (required for some
|
||||||
|
apps).
|
||||||
|
- Added parameter to clear data on reset (default behaviour unchanged).
|
||||||
|
- Ignores exception for non-fatal permission grant failure.
|
||||||
|
- Fixed issue of multiple versions of the same workload failing to find their APK.
|
||||||
|
- Added method to ensure a valid apk version is used within a workload.
|
||||||
|
- Updated how APK resolution is performed to maximise likelihood of
|
||||||
|
a workload running.
|
||||||
|
- When ``check_apk`` is ``True`` will prefer host APK and if no suitable APK
|
||||||
|
is found, will use target APK if the correct version is present. When ``False``
|
||||||
|
will prefer target apk if it is a valid version otherwise will fallback to
|
||||||
|
host APK.
|
||||||
|
- ``RunConfiguration``: Fixed disabling of instruments in workload specs.
|
||||||
|
- ``Devices``:
|
||||||
|
- Added network connectivity check for devices.
|
||||||
|
- Subclasses can now set ``requires_network`` to ``True`` and network
|
||||||
|
connectivity check will be performed during ``setup()``.
|
||||||
|
- ``Workloads``:
|
||||||
|
- Added network check methods.
|
||||||
|
- Fixed versions to be backwards compatible.
|
||||||
|
- Updated workload versions to match APK files.
|
||||||
|
- Fixed issues with calling super.
|
||||||
|
- ``Assets``: Added script to retrieve external assets for workloads.
|
||||||
|
- ``Execution``: Added a ``clean_up`` global config option to delete WA files from
|
||||||
|
devices.
|
||||||
|
- ``Runner``: No longer takes a screenshot or dump of UI hierarchy for some errors when
|
||||||
|
unnecessary, e.g. host errors.
|
||||||
|
- ``core``: Constraints and allowed values are now checked when set instead of
|
||||||
|
when validating.
|
||||||
|
- ``FpsProcessor``:
|
||||||
|
- Added requirement on ``filtered_vsyncs_to_compose`` for ``total_vsync metric``.
|
||||||
|
- Removed misleading comment in class description.
|
||||||
|
- ``BaseUiAutomation``: Added new Marker API so workloads generate start and end
|
||||||
|
markers with a string name.
|
||||||
|
- ``AndroidUiAutoBenchmark``: Automatically checks for known package versions
|
||||||
|
that don't work well with AndroidUiAutoBenchmark workloads.
|
||||||
|
|
||||||
|
Other
|
||||||
|
~~~~~
|
||||||
|
- Updated setup.py url to be a valid URI.
|
||||||
|
- Fixed workload name in big.Little sample agenda.
|
||||||
|
|
||||||
|
Incompatible changes
|
||||||
|
####################
|
||||||
|
|
||||||
|
Framework
|
||||||
|
~~~~~~~~~
|
||||||
|
- ``check_abi``: Now renamed to ``exact_abi``, is used to ensure that if enabled,
|
||||||
|
only an apk containing no native code or code designed for the devices primary
|
||||||
|
abi is use.
|
||||||
|
- ``AndroidDevice``: Renamed ``supported_eabis`` property to ``supported_abis``
|
||||||
|
to be consistent with linux devices.
|
||||||
|
|
||||||
|
Workloads
|
||||||
|
~~~~~~~~~~
|
||||||
|
- ``skypevideo``: Workload removed and replaced with ``skype`` workload.
|
||||||
|
|
||||||
-------------
|
-------------
|
||||||
Version 2.5.0
|
Version 2.5.0
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user