1
0
mirror of https://github.com/ARM-software/workload-automation.git synced 2024-10-06 10:51:13 +01:00
Commit Graph

141 Commits

Author SHA1 Message Date
Sebastian Goscik
b17ae78d6b adb_shell: Now handles return codes from ADB
As of ADB 1.0.35/Android N, it will return the exit code of the command that it runs
This code handles this scenario as before WA treated a return code from ADB as an
error with ADB.
2016-04-05 15:53:41 +01:00
Sergei Trofimov
391b0b01fc pylint/pep8 fixes
- android/workload: emoved an extra bank line between methods
- trace_cmd: define member attribute inside __init__
- adb_shell: ignore pylint warning about too many branches in this case
2016-04-05 11:36:39 +01:00
Sergei Trofimov
107e8414bb trace-cmd: set a minimum bound on trace pull timeout
The timeout for the pulling the trace file after the run is being set
based on the time for which the trace was collected. For workloads with
short execution time, but large number of events, the resulting timeout
might be too short. To deal with this, do not let the timout be shorter
than 1 minute.
2016-03-24 16:49:42 +00:00
Sergei Trofimov
4f8b7e9f59 trace-cmd: updating sched_switch parser to handle both formats.
Depending on the kernel, sched_switch events may be formatted one of two
different ways in the text output. Previously, we've only handled the
"old" format. This commit updates the parser to handle the new format as
well.
2016-03-24 16:33:29 +00:00
Sebastian Goscik
5abeb7aac2 adb_shell: Fixed checking exit codes on Android N
As of android N '\n' is used as the new line separator not '\r\n'.
This fix makes the function detect which is being used by the device.
2016-03-23 13:43:07 +00:00
Sergei Trofimov
203a3f7d07 LinuxDevice: fixed reboot.
- Deal with the dropped connection on issuing "reboot"
- Introduced a fixed initial delay before polling for connection to
  avoid re-connecting to adevice that is still in the process of
  shutting down.
2016-02-22 09:45:42 +00:00
Sebastian Goscik
de021da300 ApkWorkload: Fixed runtime permission granting
"Normal" android permissions are automatically granted and cannot
be changed. Trying to "pm grant" these caused an error, this should
no longer occur.
2016-02-15 11:38:28 +00:00
Sergei Trofimov
4f30e37f22 ipython: switched to using LooseVersion for version checks.
This is a fix for

https://github.com/ARM-software/workload-automation/issues/97

IPython can use rc tags in its version strings, which StrictVersion
can't handle.
2016-02-10 09:01:40 +00:00
Sergei Trofimov
3a99a284c4 cpustate: ensure known initial state
cpustate result processor generates a view of the cpu subsystem power
state during execution of a workload from cpu_idle and cpu_frequency
ftraces. There exists a period before the first events in those
categories are seen where the state of the cpu subsystem is (partially)
unknown and it is reported as such by the result processor.

Unknown state usually exists for a relatively short period of time and
is generally not a big deal. For certain kinds of workloads, however, it
may constitude a significant protion of the trace.

Changes in this commit attempt to deal with this by a) reading starting
cpu frequencies and writing them into the trace, and b) nudging each
core to bring it out of idle; this happens before the start marker, so
that the system state between the markers should be completely known.
2016-02-04 16:08:22 +00:00
Sergei Trofimov
7edb2c8919 Adding aarch64 to architecture list for arm64 ABI
This was observer being reported on a device.
2016-02-02 09:29:22 +00:00
Brendan Jackman
2ae8c6073f doc: Apply it's/its grammar pedantry 2016-01-25 13:34:24 +00:00
Sebastian Goscik
41890589e1 ipython: Updated to work with the latest ipython version 2016-01-12 10:59:52 +00:00
Sergei Trofimov
b84f97a902 trace_cmd: updated to handle empty CPUs.
Updated trace-cmd parser to handle messages that the trace for a CPU is
empty.
2016-01-12 10:12:00 +00:00
Sergei Trofimov
e83d021a5c utils/android: fixed use of variables in as_root=True commands.
In order to execute as root, the command string gets echo'd into so;
previusly, double quotes were used in echo, which caused any veriables
in the command string to be expanded _before_ it was echoed.
2015-12-15 08:34:18 +00:00
Sergei Trofimov
0f60e9600f trace_cmd: parser for sched_switch events and fixes
- Compiled regular expressions in EVENT_PARSER_MAP now get handled
  correctly.
- regex_body_parser now attemts to convert field values to ints,
  bringing it inline with the default parser behavior.
- There is now a regex for sched_switch events.
2015-12-10 13:41:24 +00:00
Sebastian Goscik
bef8fb40ef Updated pylint for v1.5.1
Fixed WA for the latest version of pylint (1.5.1)
2015-12-09 16:52:39 +00:00
Sergei Trofimov
a06016a442 adb_shell: fixing handling of line breaks at the end of the output
- adb protcol uses "\r\n" for line breaks. This is not handled by
  Python's line break translation, as not a file. So spliting on '\n'
  when extracting the exit code resulted in stray '\r' in the output.
- adb_shell expects exit code to be echoed on the same line. This may
  not have been the case if latest output from executed command was not
  a complete line. An extra echo will now ensure that the exit code will
  be on its own line even in that case.
2015-11-24 15:50:38 +00:00
Sascha Bischoff
a6382b730b TelnetConnection:
- Allowed telnet connections without a password.

This is required as part of the upcoming Gem5Device, which uses a
password-less telnet connection to communicate with the device.
2015-11-02 10:09:59 +00:00
Sebastian Goscik
b9b4a7c65c Fixed telnet support
- Added port kwarg to telnet login function (default port=23)
2015-10-21 11:53:25 +01:00
Sergei Trofimov
691c380779 serial_port: updating to handle newer versions of pexpect
from version 4.0.0 onwards, fdpexpect is now namespaced under the main
pexpect libarary, rather than being stand-alone.
2015-10-16 17:38:19 +01:00
Sergei Trofimov
7546232c10 trace_cmd: fix parser to oportunistically convert parsed values to ints
This was done previosly but was broken by commit

1c146e3ce7
2015-10-16 17:08:09 +01:00
Sergei Trofimov
4fa3d9de6e Import pxssh via pexpect
Importing it directly causes issues in some environments.
2015-10-16 11:05:53 +01:00
Sergei Trofimov
1c146e3ce7 trace_cmd: more robust event fields parsing
The default paraser can now handle spaces in field values (but not field
names).
2015-10-08 17:13:48 +01:00
Sergei Trofimov
3247b63cb9 shh: handle backspaces in serial output 2015-09-14 10:58:14 +01:00
Sergei Trofimov
84f7adbfb2 uefi menu: updated prompt_regex to handle debug UEFI builds. 2015-09-09 17:40:02 +01:00
Sergei Trofimov
b6c497d32b Updated boolean to interpret 'off' as False 2015-09-09 14:32:27 +01:00
Sergei Trofimov
bfed59a7cf Adding an HTTP-based resource getter. 2015-09-03 13:55:44 +01:00
Sergei Trofimov
047308a904 ipython: pylint fixes 2015-09-03 12:07:41 +01:00
Sergei Trofimov
c04b98c75a ipython: updated version check.
Version 4.0.0 changes API and breaks WA's usage of it. Updated version
check to only initialize ipython utils if version is < 4.0.0.
2015-09-02 17:39:11 +01:00
Sergei Trofimov
95c3f049fb trace_cmd: handle trace headers and thread names with spaces 2015-08-17 12:37:58 +01:00
Sergei Trofimov
0e1990a2bb ssh: fixed error reporting on failed exit code extraction. 2015-08-12 11:28:38 +01:00
Sergei Trofimov
ab45c4499f utils.misc.normalize: only normalize string keys. 2015-08-10 10:42:20 +01:00
Sergei Trofimov
89da25c25e cpustates: added "Running (unknown Hz)" state to the timeline. 2015-07-20 17:04:09 +01:00
Sergei Trofimov
4debbb2a66 Updating ANROID_VERSION_MAP with recent versions. 2015-07-17 17:24:14 +01:00
Sergei Trofimov
b8e25efdd4 ssh: attempt to deal with dropped connections 2015-07-10 11:44:01 +01:00
Javi Merino
30d7ee52f4 ipynb_exporter: learn to convert the notebook to HTML 2015-07-09 12:50:02 +01:00
Javi Merino
539c3de7b8 ipython: rename IPYTHON_NBCONVERT to IPYTHON_NBCONVERT_PDF
This command is tailored for converting notebooks to pdf, but we also
want it to be able to generate html.
2015-07-09 12:49:31 +01:00
Sergei Trofimov
13e5e4d943 adb_connect: do not assume port 5555 2015-07-07 11:19:58 +01:00
Sergei Trofimov
3b4dc137d2 cpustate: check if trace marker is present and disable marker filtering if it is not. 2015-06-29 17:33:28 +01:00
Sergei Trofimov
78314c1ef2 cpustates added an option to ignore trace markers. 2015-06-29 17:28:00 +01:00
Sergei Trofimov
b018adac11 pylint fixes 2015-06-29 11:34:49 +01:00
Sergei Trofimov
26dee81164 Adding arm64-v8a to ABI map 2015-06-29 09:21:06 +01:00
Bobby Batacharia
22d72de969 Fix terminal size discovery in DescriptionListFormatter 2015-06-28 10:35:15 +01:00
Sergei Trofimov
5401a59da0 Adding support for U-Boot booting in Juno. 2015-06-25 11:32:01 +01:00
Sergei Trofimov
00561e0973 Adding support for U-Boot booting in Juno. 2015-06-25 10:59:19 +01:00
Sergei Trofimov
a85e45c6b0 cpustate: now generates a timeline csv as well as stats 2015-06-16 11:04:25 +01:00
Sergei Trofimov
ab76aa73f2 cpustate: fixing division by zero
total running time (in parallel stats) is zero when all cores on a
cluster are hotplugged. this caused a division by zero when calculating
percentage.
2015-06-12 13:02:05 +01:00
Sergei Trofimov
557b792c77 Made abi property common between Android and Linux devices
In both cases, the ABI is now obtained by executing "uname -m" on the
device and perfroming a mapping from the returned machine architecture
a known ABI. If no known ABI is found the architecture string itself is
returned.
2015-06-11 17:45:09 +01:00
Sergei Trofimov
c40a7fd644 more robust exit_code handling for ssh interface
Background processes may produce output on STDOUT. This could get
captured when obtaining the result of "echo $?" to get previos command's
exit code. So it's not safe to assume that output will always be an int.
Attempt to strip out superflous output before doing the int conversion
and, on failure, log a warning but don't error out.
2015-06-05 12:47:01 +01:00
Sergei Trofimov
02af02f0cb Adding cpustates result processor (and script) 2015-06-03 16:20:48 +01:00
Sergei Trofimov
e9b21e2ef3 Adding a generic trace-cmd paraser. 2015-06-03 16:20:48 +01:00
Sergei Trofimov
ead0be2763 Fixing merge_lists to work for list_or_* types 2015-06-01 16:18:13 +01:00
Sergei Trofimov
f59da723fb list_or: chaniging how list_or_* functions work and adding a generic list_or
list_or_* functions (e.g. list_or_string) will now always return a list,
however will accept lists or indivitual values. Also added a list_or()
generator function, similar to what already exists for list_of().
2015-06-01 15:31:25 +01:00
Sergei Trofimov
f49287cf09 Fixes for Emacs
- Do not try to use a pager if it explicitly disabled with PAGER='' in
  the environment.
- If terminal size is identified as (0, 0), fall back to default (80,
  25).
2015-06-01 10:05:23 +01:00
Sergei Trofimov
506ed57ca6 fix: telemetry: ignore all return codes
Telemetry seems to return random values as return code, so completely
ignore them and don't treat any values as errors.
2015-05-27 17:25:17 +01:00
Sergei Trofimov
c31d4ec8a3 ssh: fixing timeout behavior
Since a command would still be running on time out, it would prevent
issuing subsequent commands in the same SSH shell, make it look like
the device has become unresponsive.

If a timeout condition is his, send ^C to kill the current foreground
process and make the shell available for subsequent commands.
2015-05-27 09:13:03 +01:00
Sergei Trofimov
26dfe97ffd ssh: fix: do not attempt to check mode of keyfile if it has not been specified. 2015-05-14 11:27:50 +01:00
Sergei Trofimov
5daa9014a8 Adding "agruments" type
This type represents arguments that are passed on a command line to an
application. It has both string and list representations.
2015-05-12 11:34:30 +01:00
Sergei Trofimov
b2981a57bc ssh: ensure keyfile has the right permissions
The key file must only be readable by the owner. If the specified key
file has different access permissions, create a temporary copy with the
right permissions and use that.
2015-05-12 11:34:30 +01:00
Sergei Trofimov
b30d702f22 minor fixes to cpufreq APIs
- adding missing cpu id conversion to set_cpu_min_freq
- adding "exact" parameter to set_cpu_frequency; only produce an error
  when the specified frequency is not supported by the cpu if "axact"
  is set; (otherwise let cpufreq decide what the actual frequency will
  be).
2015-05-12 11:34:30 +01:00
Sergei Trofimov
fbde403f6f pep8 2015-05-11 12:12:41 +01:00
Sergei Trofimov
0e751bdd73 Handling duplicate prompt in pxssh in a slightly differnt way. 2015-05-11 12:12:41 +01:00
Sergei Trofimov
c5b884cf81 casless_string: added format() method 2015-05-11 12:12:41 +01:00
Sergei Trofimov
47ce9db383 Updating energy model instrument to be able to accumulate multiple power rails for each cluster. 2015-05-11 12:12:40 +01:00
Sergei Trofimov
09aa9e6792 Adding some conversion functions to misc utils.
- list_to_range and range_to_list convert between lists of integers
  and corresponding range strings, e.g. between [0,1,2,4] and '0-2,4'
- list_to_mask and mask_to_list convert between lists of integers and
  corresponding integer masks, e.g. between [0,1,2,4] and 0x17

Conflicts:
	wlauto/utils/misc.py
2015-05-11 12:12:40 +01:00
Sergei Trofimov
3112eb0a9b Adding new types and updating device parameters to use them.
- added caseless_string type. This behaves exactly like a string, except
  this ignores case in comparisons. It does, however, preserve case. E.g.

	>>> s = caseless_string('Test')
	>>> s == 'test'
	True
	>>> print s
	Test

- added list_of type generating function. This allows to dynamically
  generate type-safe list types based on an existing type. E.g.

	>>> list_of_bool = list_of(bool)
	>>> list_of_bool(['foo', 0, 1, '', True])
	[True, False, True, False, True]

- Update core_names Device Parameter to be of type caseless_string
2015-05-11 12:12:40 +01:00
Sergei Trofimov
a6ef53291b ssh: making execute() thread safe. 2015-05-05 09:09:57 +01:00
Sergei Trofimov
8b606dd5f9 config: fixing an issue introduced by previous config fix...
When the merging logic was updated to preserve duplicates within the
same list, it inadvertently broke the logic that removed items marked
for removal with a '~'. This commit rectifies that.

Note to self: merging functions are doing *way* to much; they should be
refactored into several individual function and config should be
resolved in distinct stages.
2015-04-30 13:34:21 +01:00
Sergei Trofimov
799558d201 Fix: lists with duplicate entries in parameter values
If list parameter values contained duplicates, those got removed when
merging parameter values from different sources. This commit fixes that
behavor, so that duplcates that appear within the *same* list are
preserved.
2015-04-30 08:46:24 +01:00
Sergei Trofimov
e87e4c582c Fix: properly handle carriage return stripping in ssh. 2015-04-28 12:46:21 +01:00
Sergei Trofimov
97a397d5c8 ipython utils: handle ancient versions of IPython
Very old versions of IPython do not have IPython.version_info attribute
that the ultls module relied on. This commit changes it to use the more
standard __version__ attriute that is present in all versions.
2015-04-24 10:56:23 +01:00
Sergei Trofimov
5035fe6f44 Adding ignore parameter to check_output
Adding a parater to wlauto.utils.misc.check_output to specify that it
should ignore certain error codes when they are returned by the
subprocess and not raise them as errors.
2015-04-21 15:01:15 +01:00
Javi Merino
e30386ce4a Add ipython version 3 support for the generic ipython support 2015-04-17 19:00:55 +01:00
Javi Merino
d12f5c65e1 Factor out the parsing of a valid cell run in the generic ipython implementation
run_cell() becomes more complicated when we add ipython version 3
support which upsets pylint because there are "too many
branches (15/12)".  Factor out part of the function to make pylint
happy.
2015-04-17 18:56:57 +01:00
Javi Merino
2b04cb38d9 Don't break prematurely when running a cell on an ipython kernel
The kernel may go idle before it processes the next input, which break
the while=True loop in run_cell() early.  Wait for an acknowledgement
of the input we've sent to the kernel before considering an idle
message to mean that the cell has been parsed.
2015-04-17 18:56:57 +01:00
setrofim
0faa5ae455 Merge pull request #11 from JaviMerino/ipynb_exp_improvs
Generic Ipynb_exporter improvements to ease ipython 3 support
2015-04-17 18:23:04 +01:00
Javi Merino
3bf04735c1 Factor out ipython nbconvert in ipynb_exporter results processor
ipython nbconvert CLI changes between ipython 2 and 3, so keep the
string separate so that we can update when we add ipython version 3 support.
2015-04-17 17:52:07 +01:00
Javi Merino
f54bb0981f Don't use shell_channel in ipynb_exporter result processor
It's not needed for IPython 2 and it breaks in IPython 3
2015-04-17 17:52:07 +01:00
Javi Merino
f41be396fa Use notebook format 3 for ipynb_exporter
The file assumes format 3 throughout the code, so explicitly import that
format instead of the generic "current" format.
2015-04-17 17:52:07 +01:00
Javi Merino
d31a5c3c48 Factor out the ipython implementation in ipynb_exporter
The internal ABI for ipython has changed between ipython version 2 and
3.  In its current state, the result processor only works with IPython
version 2, so fail if the user wants to use the result processor with
the wrong version.

Abstract the ipython interface to a file so that we can make it support
versions 2 and 3 at the same time.
2015-04-17 17:52:07 +01:00
Sergei Trofimov
314fecfcd4 Fixing get_meansd for numbers < 1 2015-04-17 17:27:47 +01:00
Sergei Trofimov
53f2eafd16 Fixing key-based authentication for SSH. 2015-04-17 11:59:46 +01:00
Sergei Trofimov
8e340e456d Added open_file to misc utils.
A function to open a file using an associated launcher in an OS-agnostic
way.
2015-04-14 14:18:10 +01:00
Sergei Trofimov
c5d3c2bd62 Fixing reboot on Linux devices [part 2]
- connect() to device before issuing the initial reboot, as soft reset
  requires a device connection.
- boot() has been implemented to wait properly for the device to reboot
  after reset.
- port now defaults to 22 rather than being left unset, as need
  something to connect to when polling for device after reboot.
- Only use -P option for scp when port is *not* 22; as that option
  appears to cause intermittent issues with default scp on Ubuntu 12.04
2015-04-10 17:04:22 +01:00
Sergei Trofimov
086f03730b ssh: pep8 fix 2015-04-10 09:36:55 +01:00
Sergei Trofimov
fea32de3d3 Parameterizing password prompt for sudo in Ssh interface.
On some devices, sudo presents a different prompt when asking for a
password. All the prompt to be specified in device configruation to
handle such cases.
2015-04-10 09:03:48 +01:00
Sergei Trofimov
544f5f0b92 Fixing terminalsize import error.
terminalsize was loaded from a location added to sys.path during
bootstrap. This appeared to be causing import issues. There is no longer
a good reason for terminalsize to be loaded that way, so just moved it
under wlauto.utils so that it can be loaded normally.
2015-04-09 08:32:02 +01:00
Koan-Sin Tan
cd4790dd4b s/SKD/SDK/ 2015-04-07 09:23:24 +08:00
Sergei Trofimov
adb5ea9a30 Fixing UEFI entry creation for Juno
- UEFI config can be specified as a device_config parameter
- The same config is used to create a missing UEFI entry, and
  to re-create the entry when flashing. UEFI config now wholy
  resides within the device and is not specified for vexpress
  flasher.
2015-03-26 10:42:55 +00:00
Sergei Trofimov
aa74e1e8f5 Minor fix to rst table generation.
Updated format_simple_table to take headers into account when
calculating column widths. Also updated docs where it currently used.
2015-03-18 10:47:07 +00:00
Sergei Trofimov
a747ec7e4c Initial commit of open source Workload Automation. 2015-03-10 13:09:31 +00:00