Move from pipes.quote (private) to shlex.quote (Py3.3+ standard).
Make tests of inputs against None (their default value) instead of based
on their truthiness.
Improve logging through quoted commands (runnable as-is, less confusing).
Make the command-building process straightforward for readability and
maintainability.
Command line echoing was disabled, but that disabling did not take
effect. Another part of devlib was still expecting command lines to be
echoed. That is fixed by disabling echoing when creating pxssh
connection, and removing the code that expected the line to be echoed.
As detailed in https://github.com/pexpect/pexpect/issues/552 when
sending a command with exactly the length of the prompt less than the
window width, an extra prompt is introduced into the returned data
causing the matching to fail and only return part of the commands
output. To workaround this check the length of the command to be
submitted and if of the specific length add an additional whitespace
character to the end of the command to prevent this behaviour.
Add a pseudo-attribute to ApkInfo giving all the methods available in
the APK as a list of (class, method) tuples. To keep backward
compatibility, this list is retrieved the first time the attribute is
being accessed.
Add an activities pseudo-attribute to the ApkInfo class that retrieves the
names of the activities (_i.e._ entry points) of the APK. To keep
backward compatibility, these are retrieved the first time the attribute is
being accessed.
Use try/finally clause in the contextmanager to always close the
connection if an exception is raised.
Also remove "del conn" since it only decrements the reference count,
which is done anyway when the generator function returns.
In Python 3 when re-raising an exception the original traceback will
also be included. In the `_scp` method we do not want to expose the
password so hide the original exception when re-raising.
pipes.quote (also known as shlex.quote in Python3) provides a robust
implementation of quoting and escaping strings before passing them to
POSIX-style shells. Use it instead of the escape_* functions to simplify
quoting inside devlib.
Strings are passed to subprocess.Popen is used instead of sequences, so
proper quoting of parameters in command lines is needed to avoid
failures on filenames containing e.g. parenthesis. Use pipes.quote to
quote a string to avoid any issue with special characters.
Use __get_memo_id() for keyword arguments, rather than stringifying
them. The has somewhat lower chance of producing the same identifier
for two conceptually unequal objects.
If the connection is unavailable when attempting the logout procure it
can fail leaving the connection open on the host. Now if something goes
wrong ensure that we still close the connection.
As stated in https://github.com/ARM-software/devlib/issues/308 devlib
can leave ssh connections open after they are no longer in use, so now
register the close method with atexit so the connections are no longer
left open upon exit.
Exceptions such as TargetError can sometimes be raised because of a
network issue, which is useful to distinguish from errors caused by a
missing feature for automated testing environments.
The following exceptions are introduced:
* DevlibStableError: raised when a non-transient error is encountered
* TargetStableError
* DevlibTransientError: raised when a transient error is encountered,
including timeouts.
* TargetTransientError
When there is an ambiguity on the type of exception to use, it can be
assumed that the configuration is correct, and therefore it is a
transient error, unless the function is specifically designed to probe a
property of the system. In that case, ambiguity is allowed to be lifted
by assuming a non-transient error, since we expect it to raise an
exception when that property is not met. Such ambiguous case can appear
when checking Android has booted, since we cannot know if this is a
timeout/connection issue, or an actual issue with the Android build or
configuration. Another case are the execute() methods, which can be
expected to fail on purpose. A new parameter will_succeed=False is
added, to automatically turn non transient errors into transient ones if
the caller is 100% sure that the command cannot fail unless there is an
environment issue that is outside of the scope controlled by the user.
devlib now never raises TargetError directly, but one of
TargetStableError or TargetTransientError. External code can therefore
rely on all (indirect) instances TargetError to be in either category.
Most existing uses of TargetError are replaced by TargetStableError.
If the process was killed, either the output or error can
be `None` which causes an error when attempting to join the outputs.
Also update existing error message to prevent 'None' appearing in the error
message.
This should have been handled by the @total_ordering decorator, but
isn't due to
https://bugs.python.org/issue25732
(briefly, total_ordering is back-ported from Python 3, where the base
object provides the default implementation of __ne__ based on __eq__, so
total_ordering did not override it; this, however does not happen in
Python 2).
We add the ability to explicitly take checkpoints when running with a
gem5 system. As we cannot have any state which is shared between the
host and simulated system, we first unmount the VirtIO device, take
the checkpoint, and then remount the VirtIO device into the simulated
system.
We explicitly unmount the VirtIO device (used to transfer files into
the simulated system) at the end of a run (when the connection is
closed) in order to make checkpointing the simulated system
easier. gem5 supports checkpointing the state of the simulation when
it is terminated, and future simulations are able to resume from this
snapshot. However, for the checkpoint to work correctly, we need to
make sure that there is no shared state between the simulated system
and the host. This mandates that we disconnect the VirtIO device prior
to taking a checkpoint.
Include stderr output of the executed command in the output returned by
adb_shell. This will align the AdbConnection behavior with that of
SshConnection and ensure that target.execute() behaves consistently
across Android and Linux targets.
Add types for regex and bytes_regex. In Python 3, regular expression
objects differ based on whether they were created with a str or a
bytes instance as the pattern, and can only match against instances of
the corresponding type.
To make sure we always end up using the right version (e.g. pexpect
needs bytes regexes), create functions to do the appropriate
conversions.
Convert bytes to strings (utf-8 encoding) to make compatible with
Python3 in arm.py
Use the pattern property to extract the string from the regex pattern,
to pass as a string to tty.expect.
Drop problematic characters when decoding stdout and stderr in misc.py
by setting errors='replace' in the string decode method.
We add a TraceCollector which logs the traffic on a serial port. This
can then be used to debug why a board crashes, or to extract extra
information from the device whilst it is running a workload.
Previously if a device was connected over usb then the adb_connect
method would assume the device was already connected. This can cause
issues when rebooting and the device is not ready by the time devlib
attempts to reconnect to it causing the next command to fail. Now still
only execute the 'connect' command when the device is connected over the
network, however always trying pinging the device to see if it is
connected before returning.
