1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 19:01:15 +01:00
Commit Graph

619 Commits

Author SHA1 Message Date
Sebastian Goscik
77a44f11c6 AndroidDevice & BaseLinuxDevice: minor parameter changes
Moved ``working_directory`` parameter to BaseLinuxDevice.

Changed the default ``binaries_directory`` for AndroidDevice to allow
WA binaries to be easily separated.
2016-06-21 14:55:02 +01:00
Sergei Trofimov
afeb726d53 energy_model: only set "ui" runtime parameter for ChromeOS
energy_model instrument generates job specs during the run. One of the
things it does is set "ui" runtime parameter to "off". This parameter
only exists for ChromeOS devices. This commit ensure that the parameter
is not set when running on any other device.
2016-06-21 09:14:12 +01:00
setrofim
7904e6b562 Merge pull request #182 from ep1cman/fixes
sysfs_extractor & cpufreq: Fixed error when tar.gz file already existed
2016-06-20 10:13:38 +01:00
Sebastian Goscik
224b973ace sysfs_extractor & cpufreq: Fixed error when tar.gz file already existed
If the tar.gz already existed on the target device the instruments would
fail. This fix adds the "-f" option to gzip to force it to overwrite the
file.
2016-06-20 10:12:00 +01:00
Sergei Trofimov
8660d0f488 hwuitest: invoke executable via full path
Previously, the workload was invoking the executable via its name,
assuming that it will be in PATH. As WA's executables directory is not
in path, the invocation was failing. This commit saves the full path to
the installed executable and uses that instead.
2016-06-17 08:03:50 +01:00
Sergei Trofimov
be7aa3d379 recentfling: fixing uninstalling of binaries
Inconsistently, while install() for Android devices automatically
handles both APKs and executables appropriately, uninstall() only works
for packages. Changing to use uninstall_executable() for the scripts
deployed by recentfling.
2016-06-15 09:00:49 +01:00
Sergei Trofimov
b48e5ce58a recentfling: script deployment and PID file fixes
- Scripts are now deployed during via install() ensuring that they are
  executable.
- Handle the case where the PID file is delted before getting to
  process_results.
- Exposed the option to not start any apps before flinging via a
  parameter.
2016-06-14 10:21:06 +01:00
Sergei Trofimov
f33d6f4729 bbench: handle lack of results in logcat
- make sure results_list is always instatiated even if no metrics found;
  this would previously cause a "used before declaration" error
- Detect that no metrics were extracted from the log and raise a
  WorkloadError
2016-06-14 10:20:51 +01:00
Sebastian Goscik
a826b661f4 Version bump 2016-06-10 14:26:32 +01:00
Sebastian Goscik
23b3b165d5 docs: Change log & updates 2016-06-10 13:17:10 +01:00
Sebastian Goscik
2f87e126f0 apk_version: Removed instrument
APK versions are now added as result classifiers:
48259d872b
2016-06-09 13:55:27 +01:00
Sebastian Goscik
7b92f355c8 netstat: Changed exception type & typo fix 2016-06-08 11:13:35 +01:00
Sebastian Goscik
982069be32 servo_power: Added check for device platform.
Now checks to see if the device is running chromeOS.
2016-06-08 11:10:53 +01:00
Sebastian Goscik
f276d4e39f cpustates: Added the ability to configure how a missing start marker is handled.
cpustates can now handle the lack of a start marker in three ways:

 - try: If the start marker is present only the correct section of the trace
        will be used, if its not the whole trace will be used.
 - error: An error will be raised if the start marker is missing
 - ignore: The markers are ignored and the whole trace is always used.
2016-06-06 17:09:48 +01:00
Sebastian Goscik
1811a8b733 PowerStateProcessor: Added a warning when no stop marker is encountered
PowerStateProcessor will now stop itrerating over events when it finds
a stop marker. If it does not find a stop marker it will log a warning.
2016-06-06 17:03:56 +01:00
Sebastian Goscik
0ae03e2c54 PowerStateProcessor: Exceptions no longer stop processing
If an exception is raised inside a generator it cannot be continued.
To get around this exceptions are now caught and later output via the
logger.

Also added logger setup when running cpustates as a standalone script
2016-06-06 16:28:07 +01:00
Sebastian Goscik
c423a8b4bc Utils.misc: Added memoised function decorator
This allows the return value of a function to be cached so that
when it is called in the future the function does not need to
run.

Borrowed from: https://github.com/ARM-software/devlib
2016-06-06 16:28:07 +01:00
Sebastian Goscik
c207a34872 cpustates: Now shows a warning when it fails to nudge a core.
Before WA would raise a error message that wasn't very clear.
Now when cpustates tries to nudge cores and and error occurs it
will only show a warning (which promts users to check if the cpu is
hot plugged out) and keep going with the reset of the run without
causing errors in other WA extensions.
2016-06-02 15:14:03 +01:00
Sebastian Goscik
18d1f9f649 ReventWorkload: Now kills all revent instances on teardown
Previously revent would be left running if a run was aborted.
2016-06-01 16:47:01 +01:00
Sebastian Goscik
17ce8d0fe9 Revent: Device model name is now used when searching for revent files
Previously the WA device name was used when searching for revent files.
Since most were `generic_android` this made it difficult to keep revent
files for multiple android devices. Now it the device model is used instead.

If a file with the device model is not found it will fall back to the WA
device name.
2016-06-01 16:47:01 +01:00
Sebastian Goscik
8bdffe6f9c LinuxDevice: Removed has_root method
Was not used anywhere and is_rooted should be used instead
2016-06-01 14:13:37 +01:00
Sebastian Goscik
2ff13089fd LinuxDevice: kick_off & killall will now run as root on rooted devices by default
kick_off has been changed to behave the same as AndroidDevice.

Said changes caused kill all to fail on rooted devices. Killall will now
behave in the same way as kick_off, if specifically told to (or not to)
run as root it will. Otherwise it will run as root if the device is rooted
2016-06-01 13:50:59 +01:00
setrofim
772346507c Merge pull request #144 from ep1cman/servo
servo_power: Added support for chromebook servo boards
2016-05-27 16:16:49 +01:00
Sebastian Goscik
0fc88a84be servo_power: Added support for chromebook servo boards
Servo is a debug board used for Chromium OS test and development. Among other uses, it allows
access to the built in power monitors (if present) of a Chrome OS device. More information on
Servo board can be found in the link bellow:

 https://www.chromium.org/chromium-os/servo

based on: 03ede10739
and: 9a0dc55b55
2016-05-27 16:09:08 +01:00
setrofim
6e4f6af942 Merge pull request #164 from ep1cman/poller
Poller: Added an instrument to poll files and output a csv of their v…
2016-05-26 16:33:59 +01:00
Sebastian Goscik
c87daa510e Poller: Added an instrument to poll files and output a csv of their values 2016-05-26 16:32:58 +01:00
Sergei Trofimov
a9a42164a3 list_or_string: ensure that elements of a list are always strings 2016-05-26 16:05:43 +01:00
Sebastian Goscik
0d50fe9b77 AndroidDevice: kick-off no longer requires root
kick off will now use root if the device is rooted or if manually
specified otherwise its run without root.
2016-05-26 10:29:21 +01:00
Sebastian Goscik
7ccac87b93 cameracapture & camerarecord: Fixed parameters
Parameters were not being passed to the UI automation properly
2016-05-25 09:49:21 +01:00
setrofim
24a2afb5b9 Merge pull request #168 from ep1cman/vellamo-update
Vellamo update
2016-05-24 13:01:31 +01:00
Sebastian Goscik
9652801cce vellamo: Fixed geting values from logcat
The previous method of getting results out of logcat does not work
if the format of logcat changes.
2016-05-24 13:00:10 +01:00
Sebastian Goscik
17fe6c9a5b AndroidDevice: Improved gathering of build props
These are now gathered via `getprop` rather than trying to parse the
build.prop file directly.

This fixes issues with build.prop files that have imports.
2016-05-24 12:55:33 +01:00
Sebastian Goscik
f02b6d5fd9 vellamo: Added support for v3.2.4 2016-05-24 09:57:38 +01:00
Sebastian Goscik
eaf4d02aea Merge pull request #162 from chase-qi/add-blogbench-workload
workloads: add blogbench workload
2016-05-24 09:55:37 +01:00
Chase Qi
56a4d52995 workloads: add blogbench workload
Blogbench is a portable filesystem benchmark that tries to reproduce the
load of a real-world busy file server.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-05-24 16:49:19 +08:00
Sebastian Goscik
ec5c149df5 Merge pull request #165 from chase-qi/add-stress-ng-workload
workloads: add stress_ng workload
2016-05-24 09:45:35 +01:00
Sebastian Goscik
5a1c8c7a7e cameracapture & camerarecord: Updated workloads to work with Android M+
The stock camera app as of Android M has changed. This commit updates
the ui automation to work with this new app. As part of this change
it was required to bump the API level of the ui automation to 18.

Also made the teardown of the capture workload close the app like the
record workload.
2016-05-16 17:25:50 +01:00
Sebastian Goscik
46cd26e774 BaseUiAutomation: Added functions for checking version strings
Added splitVersion and compareVersions functions allow versions strings
like "3.2.045" to be compared.

Also fixed the build script to now copy to the correct folder
2016-05-16 17:22:09 +01:00
Sebastian Goscik
544c498eb6 UiAutomatorWorkload: Added quotes around uiautomator parameters
Some characters would be interpreted by the shell thus breaking the
command. Adding quotes around the parameters solved this.

N.B Space still needs to be replaced.
2016-05-16 16:19:57 +01:00
Chase Qi
5ad75dd0b8 workloads: add stress_ng workload
stress-ng will stress test a computer system in various selectable ways.
It was designed to exercise various physical subsystems of a computer as
well as the various operating system kernel interfaces.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-05-13 19:35:26 +08:00
setrofim
b2248413b7 Merge pull request #148 from ep1cman/master
cpustates: Fix for error when trying to use cpustates with hotplugged…
2016-05-13 11:35:45 +01:00
Sebastian Goscik
8abf39762d hwmon: Fixed sensor naming
Previously the sensor name was just appeneded to the end of the
previous sensors name.

Now the hwmon name is added as a classifier of the metric.
If the hwmon sensor has a label, the metric will use this for its name,
if it does not then the sensors kind and ID will be used e.g. temp3
2016-05-10 09:27:42 +01:00
Sebastian Goscik
87cbce4244 hwmon: Added allowed values to sensors parameter
Previously the sensor name was just appeneded to the end of the
previous sensors name.
2016-05-10 09:27:42 +01:00
Sebastian Goscik
ef61f16896 AndroidDevice: Fixed screen lock disable
Due to the previous commits, this command no longer works properly.

It turns out there is an issue with using multiple levels of escaping.
It seems that bash handles the backslashes and single quotes separately
incorrectly processing our escaping. To get around this we are writing the
sqlite command to a shell script file and running that.

This seems to be the only case in WA at the moment that requires this,
if more show up/when WA moves to devlib it should use the devlib shutil
mechanism.
2016-05-10 09:27:42 +01:00
Sebastian Goscik
e96450d226 adb_shell: Fixed getting return codes
They way we were attempting to get return codes before always gave
us a return code of the previous echo, therefore always `0`.

This commit adds the newline into the last echo.
2016-05-10 09:12:54 +01:00
Sebastian Goscik
59cfd7c757 AndroidDevice: WA now pushes its own sqlite3 binary
Some device have the sqlite3 binary removed. WA will now check for
this and push its own binary if necessary.
2016-05-09 17:31:09 +01:00
Sebastian Goscik
d3c7f11f2d AndroidDevice: Changed kick_off signature to match BaseLinuxExamples 2016-05-09 17:06:08 +01:00
Sergei Trofimov
fe7f98a98b report_power_stats: number of entries returned always matches number of reporters
Previously, only reports that were generated were returned. With this
commit, there will be an entry for each active reporter in the returned
list. If a reporter did not produce a valid report, the entry will be
None.

This ensures consistent output, even if a run time issue causes a
reporter not to produce a report  (e.g. if cpufreq events were not
enabled).
2016-05-09 10:20:25 +01:00
Sebastian Goscik
66c18fcd31 cpustates: Fix for error when trying to use cpustates with hotplugged cores
It is not possible to read frequencies from a core that has been hotplugged.
The code will now set the current and max frequencies of hotplugged cores
to None.

This still doesn't work for devices that have dynamic hotplug enabled
2016-05-06 15:00:32 +01:00
Sergei Trofimov
d581f1f329 sysfile_getter/cpufreq: fix taball name
Commit 724f6e590e changed sysfile_getter
behavior to first tar up copied files and then gzip them. Tarball name
needs to be updated to not include '.gz' extension.
2016-05-06 13:51:09 +01:00
Sebastian Goscik
8dc24bd327 uboot: Now detects the U-Boot version to use correct line endings
Previously Linaro U-Boot releases had a bug where they used \n\r
as the line ending. This has now been fixed which caused
issues with WA. WA now detects the U-Boot version and uses the
coresponding line ending.
2016-05-04 11:54:29 +01:00
Sebastian Goscik
59066cb46d juno: Removed default bootargs
The default boot args have been removed since these cause issues with
the latest Linaro builds, which boot correctly without any bootargs.

Also made a regex string a raw-string.
2016-05-03 15:24:35 +01:00
Sergei Trofimov
a40542d57b create command: fix example parameter name in templates
Parameter name in workload templates updated to be a valid identifier.
2016-04-20 14:43:07 +01:00
Sergei Trofimov
697aefc7bb ApkWorkload: clear app data on failed uninstall.
If uninstall fails, "pm clear" should be called to make sure that the
next time the app is launched it starts from a known state (which would
normally be ensured by the uninstall).
2016-04-19 16:43:42 +01:00
Sergei Trofimov
8bc71bb810 ApkWorkload: report correct apk verison on failed install
It's possible that there is already a version of an app on target that
differs form the version of the apk on the host. In such cases, WA will
usually try to uninstall the target version and install the host
version.

It's possible that the uninstall may fail. If that happens, it will be
reported as a warning but workload exectuion will proceed with the
target version. In this case, apk_version would have already been set to
that of the host apk. This change ensures that the APK version is
correctly set to the target version (the one that actually ran).
2016-04-19 16:33:37 +01:00
Sebastian Goscik
91210f26e9 RunCommand: WA no longer runs with no workloads specs
Previously if no worklaod specs were loaded, WA would still start instruments
and then go immediately to the teardown stage. This no longer happens.
2016-04-19 16:32:53 +01:00
Sergei Trofimov
44a49db04d glbcorp: pep8 fix
Added a missing blank line between method declaration and class
attribute definitions.
2016-04-15 16:39:24 +01:00
Sebastian Goscik
73aa590056 glbench: renamed start_activity to launch_package
To match changes made in: ff5f48b7e7
2016-04-14 16:36:37 +01:00
Sebastian Goscik
985b249a24 glbench: Fixed ending regex
Updated the regex that detected the end of the benchmark to match the new
logcat format.
2016-04-14 16:36:37 +01:00
Sebastian Goscik
f5e138bed0 Merge pull request #141 from setrofim/master
boostrap: nicer error messages on config parasing.
2016-04-14 16:22:10 +01:00
Sergei Trofimov
b6c0e2e4fd boostrap: nicer error messages on config parasing.
- handle ValueError as well as SyntaxError from config parser
- Report source file in the error message
2016-04-14 16:18:31 +01:00
Sebastian Goscik
df8ef6be6b Merge pull request #133 from mcgeagh/uxperf
CpuUtilisationTimeline added. This now will generate cpu utilisation …
2016-04-14 14:05:58 +01:00
Michael McGeagh
8a3186e1c8 CpuUtilisationTimeline added. This now will generate cpu utilisation based on frequencies and a number of samples
Fixed error in percentage when frequency is 'None'. Now default to 0 in these cases

cpu_utilisation is now a separate parameter in cpustate. Now generates a floating point number representing the utilisation based on the maximum frequency of the capture. No longer performs averaging of values, this can be done as a post-processing step

cpu utilisation now based on the max cpu freq per core, not max captured freq overall
2016-04-14 14:03:28 +01:00
Sebastian Goscik
68043f2a52 Merge pull request #138 from mcgeagh/fps-allviews
fps: Can now process multiple 'view' attributes
2016-04-14 13:57:28 +01:00
Michael McGeagh
95bbce77a2 fps: Can now process multiple 'view' attributes 2016-04-14 13:12:39 +01:00
Sergei Trofimov
82e4998092 Deprecating apk_version instrument. 2016-04-14 11:33:54 +01:00
Sergei Trofimov
48259d872b ApkWorkload: add package verison to the result as a classifer. 2016-04-14 11:23:39 +01:00
setrofim
8d13e1f341 Merge pull request #128 from ep1cman/glbench_logcat_fix
glbench: Fixed updated logcat format
2016-04-13 16:46:09 +01:00
Michael McGeagh
68714e0e55 fps: Only check for crashed content if crash_check is true. 2016-04-11 12:01:12 +01:00
setrofim
9ee1666a76 Merge pull request #131 from ep1cman/master
SysfsExtractor & Busybox fixes
2016-04-07 10:31:31 +01:00
Sebastian Goscik
8dcdc9afe1 busybox: Rebuilt busybox binaries to prefer applets over system binaries
Busybox will now prefer to use its own built in applets before it tries
using the system binaries so that we are always running commands as expected.
2016-04-07 10:29:13 +01:00
Sebastian Goscik
724f6e590e SysfsExtractor: Now performs tar and gzip separately
On some devices there were permissions issues when trying to tar and gzip
the temp-fs in one command. These two steps are now done separately.
2016-04-07 10:29:13 +01:00
Sebastian Goscik
507090515b Merge pull request #134 from jimboatarm/master
Fix to install APKs with whitespace in their path name
2016-04-06 10:56:58 +01:00
James Hartley
1dfbe9e44c Fix to install APKs with whitespace in their path name 2016-04-06 10:53:08 +01:00
Sebastian Goscik
b17ae78d6b adb_shell: Now handles return codes from ADB
As of ADB 1.0.35/Android N, it will return the exit code of the command that it runs
This code handles this scenario as before WA treated a return code from ADB as an
error with ADB.
2016-04-05 15:53:41 +01:00
Sergei Trofimov
391b0b01fc pylint/pep8 fixes
- android/workload: emoved an extra bank line between methods
- trace_cmd: define member attribute inside __init__
- adb_shell: ignore pylint warning about too many branches in this case
2016-04-05 11:36:39 +01:00
James Hartley
ff5f48b7e7 Fix for packages without launch activities
If the package has no defined launch activity you must call the
activity manager in a different way.
2016-04-05 10:24:42 +01:00
Sebastian Goscik
9a301175b0 glbench: Fixed updated logcat format
The old results looked like:
I/TfwActivity(30824):    "description": "",
I/TfwActivity(30824):    "elapsed_time": 62070,
I/TfwActivity(30824):    "error": "NOERROR",

The new format is:
04-04 11:38:04.144  1410  1410 I TfwActivity:    "description": "",
04-04 11:38:04.144  1410  1410 I TfwActivity:    "elapsed_time": 62009,
04-04 11:38:04.144  1410  1410 I TfwActivity:    "error": "NOERROR",
2016-04-04 17:33:48 +01:00
setrofim
712c79020d Merge pull request #126 from ep1cman/master
ResourceResolver: Show version number when resource wasn't found.
2016-03-30 11:05:21 +01:00
Sebastian Goscik
12dfbef76b ResourceResolver: Show version number when resource wasn't found.
If the ResourceResolver was looking for a specific version of a
resource and could not find it, this version number is now shown
in the error message.
2016-03-30 11:01:35 +01:00
Sergei Trofimov
107e8414bb trace-cmd: set a minimum bound on trace pull timeout
The timeout for the pulling the trace file after the run is being set
based on the time for which the trace was collected. For workloads with
short execution time, but large number of events, the resulting timeout
might be too short. To deal with this, do not let the timout be shorter
than 1 minute.
2016-03-24 16:49:42 +00:00
Sergei Trofimov
4f8b7e9f59 trace-cmd: updating sched_switch parser to handle both formats.
Depending on the kernel, sched_switch events may be formatted one of two
different ways in the text output. Previously, we've only handled the
"old" format. This commit updates the parser to handle the new format as
well.
2016-03-24 16:33:29 +00:00
setrofim
a077e7df3c Merge pull request #124 from ep1cman/master
BaseLinuxDevice: gzipped property files are now zcat'ed
2016-03-24 16:30:32 +00:00
Sebastian Goscik
a2257fe1e2 BaseLinuxDevice: gzipped property files are now zcat'ed
Before they were cat'ed this gave garbage output for compressed files.
Cat-ing is necessary since not all properties are normal files (sysfs).
2016-03-24 16:28:19 +00:00
Sebastian Goscik
50353d0b8f Merge pull request #122 from Sticklyman1936/lmbench_update
lmbench: Tidied up the code and improved stability
2016-03-24 16:26:52 +00:00
Sascha Bischoff
2eca77fb02 sysbench: use device busybox binary
Use the full path to busybox on the target device as opposed to
assuming it is found on the path.
2016-03-24 16:21:01 +00:00
Sascha Bischoff
3de5b5fe0b lmbench: Tidied up the code and improved stability
This patch tidies up the benchmark code to bring it in line with the
style used in Workload Automation in general. Additionally, the
results from sub-benchmarks are now directly written to a file on the
device as opposed to processing the standard output/error from the
benchmark, which was error prone.
2016-03-24 10:20:32 +00:00
Sergei Trofimov
3043506d86 applaunch: pass the location of busybox into the script
applaunch creates and deploys an auxilary script in order to collect
precise timings. This script invoked busybox with the assumption that it
is in PATH.

Since recent changes mean that it is no longer deployed to /system/bin,
the busybox in not found. With this commit, the full path to busybox
will be passed into the script's template.
2016-03-23 16:28:18 +00:00
Sebastian Goscik
5abeb7aac2 adb_shell: Fixed checking exit codes on Android N
As of android N '\n' is used as the new line separator not '\r\n'.
This fix makes the function detect which is being used by the device.
2016-03-23 13:43:07 +00:00
Sebastian Goscik
15ced50640 daq: Fixed channel merging
Fixed channel merging when setting merge to True.
Channel merges done by setting a mapping manually were not affected by this bug.
2016-03-21 11:15:30 +00:00
Sebastian Goscik
3531dd6d07 dhyrstone: Fixed arm64 binary
It was dynamically linked, its is now statically linked
2016-03-15 14:38:18 +00:00
Sebastian Goscik
79554a2dbc freq_sweep: Improved documentation
- Added explanation that this instrument does not taskset workloads
 - Fixed formatting issue with the agenda example
2016-03-09 16:37:15 +00:00
Sebastian Goscik
11184750ec dhrystone: Updated executable resolution
Previously it was just using the binary in the dhrystone folder.
Now it uses WA's resource resolution to use the correct ABI.
2016-03-09 14:54:39 +00:00
Sebastian Goscik
20cd6a9c18 daq: Added check for duplicate channel labels
The daq instrument will no longer accept duplicate channel names.
This caused issues where files sent from the daq sever were being
overwritten.
2016-03-07 13:21:40 +00:00
Sergei Trofimov
0c1e01cad4 run command: more usefull error message when specifying non-existing agenda path
If the specified agenda argument is not found in the file system, WA
assumes it is the name of a workload and would then raise an "extension
not found error", which may be confusing if the user's intension was to
specify a path.

Now, WA will first check that neither path separator, nor a '.' are
present in the agenda argument before assuming it is a workload name, and
will provide a less confusing error in that case.
2016-02-29 17:26:29 +00:00
Sergei Trofimov
203a3f7d07 LinuxDevice: fixed reboot.
- Deal with the dropped connection on issuing "reboot"
- Introduced a fixed initial delay before polling for connection to
  avoid re-connecting to adevice that is still in the process of
  shutting down.
2016-02-22 09:45:42 +00:00
Brendan Jackman
a5c9b94257 Add entry_point check for config file existence
This is just to provide a friendlier error message.
Before this commit you get an IOError from imp.load_source.
2016-02-17 17:24:14 +00:00
Sebastian Goscik
de021da300 ApkWorkload: Fixed runtime permission granting
"Normal" android permissions are automatically granted and cannot
be changed. Trying to "pm grant" these caused an error, this should
no longer occur.
2016-02-15 11:38:28 +00:00
Sebastian Goscik
5203188d9e LinuxDevice: Added as_root to kick_off 2016-02-12 09:54:14 +00:00
Steve Bannister
08663209d6 Fix up lmbench commandline 2016-02-11 17:40:31 +00:00