Some Android devices do not have 'su'. But if they are already
rooted, there's no reason to fail. Circumvent this scenario by never
using 'su' for device already rooted.
SSH servers seem to have a maximum number of opened channels, after
which paramiko will raise an exception:
Could not open an SSH channel: ChannelException(2, 'Connect failed')
Memoizing the SFTPClient object based on the timeout setting leads to
many opened sessions, since the timeout is typically adjusted, e.g. to
match the size of the file when pulling an ftrace trace.dat file.
Solve that by memoizing the SFTPClient based only on the connection
object, with a maximum number of 1 cached object, and update its timeout
setting inplace.
Add Target{Stable,Transient}CalledProcessError exceptions, with an
.returncode and .output attributes, raised by Target.execute(),
mirroring subprocess.CalledProcessError.
This is very useful in client code that uses "exit N" to signal an
abnormal condition, and then inspects the output to find out more.
Handle in one place the decision of what is the real destination of each
file in push/pull operations.
The following can now be assumed by the connection:
* The destination does not exist.
* The folder containing the destination does exist.
This ensures consistent errors and behaviors across all connection
types, at the cost of:
* At least an extra execute() per source (up to 2 if the destination
is a file that needs to be removed to make room).
* For now, globbing will lead to a separate request for each file,
rather than a merged one. This is because the destination of each
source is prepared so that the connection will not have any
interpretation work to do.
Since we have the guarantee to have a different SshConnection per
thread, we can memoize paramiko's SFTPClient. This provides a great
performance boost.
The Popen object created for background command currently has no stdin
pipe, preventing the user from writing to it (it will be None instead of
being a file-like object).
Fix that by passing stdin=subprocess.PIPE to Popen constructor.
Move methods related to the scp command into TelnetConnection, since
SshConnection do not use it at all anymore (it has been replaced by the
"scp" python package, piggy backing on paramiko).
When Target.conn property is required while the current connection is
already in use, provide a fresh connection to avoid deadlocks. This is
enabled by the @call_conn decorator that is used on all Target methods
that use self.conn directly.
When SFTP is not available on OpenSSH, paramiko will raise a generic
exception:
paramiko.ssh_exception.SSHException: EOF during negotiation
In order to make it easier to debug, raise a TargetStableError telling
the user to enable SFTP on their server. On OpenSSH, this means
installing the sftp subsystem and enabling it in sshd_config.
On some older versions of ps the flags used to look up
the pids are not supported and results in no output.
Use the busybox implementation for consistency.
Add -k to sudo invocation to avoid using cached credentials.
If cached credentials is used, sudo will not write a prompt again,
leading to the stderr fixup code to remove a char from stderr output.
On systems that do not have git installed devlib will currently fail
to install with a FileNotFound Exception. If git is not present then
we will not have a commit hash so just ignore this error.
Using dexdump from versions 30.0.1-30.0.3 of Android build tools
does not produce valid XML caused by certain APKs
Use the lxml module for parsing xml as it is more robust and
better equipped to handle errors in input.
If there is an additional file or directory in the `build_tools` directory
then WA can fail to find a working version of aapt(2), ensure that at least
one of the binaries is a valid file path.
On a target where sudo is required, target.file_exists() erroneously
returns True despite the execute() output being:
'[sudo] password for valsch01: 0\n'
The sudo prompt is being written to stderr (as per sudo -S), but this is
still merged into the final execute() output. Get rid of the prompt to
prevent it from interfering with any command output processor.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
For connections that allow for it (ADB and SSH using SFTP
and SCP) this change enables file transfers to be polled to check
if the transfer is still in progress after some period of time or
whether the transfer should be terminated.
If a timeout is specified in the call to ``pull`` or ``push`` then the
transfer will not be polled and will terminate ordinarily when either
the transfer completes or the timeout is reached. If a timeout is
not specified, then the transfer will be polled if ``poll_transfers`` is
set, otherwise the transfer will continue with no timeout at all.
SSH transfers supply a callback to the transfer, that is called
after every block of the source is transferred. If the callback has not
been triggered within one poll period, then the transfer is cancelled.
ADB transfers have the destination size polled every poll period, and
the size compared to the previous poll to check if the transfer has
stalled. When the destination is no longer growing in size, the poller
will attempt to kill the subprocess to end the transfer.
If the transfer is still active, but the total transfer time has
exceeded the ``total_timeout`` (default: 1 hour) the transfer will then
also be killed.
Transfer polling will only begin after the ``start_transfer_poll_delay``
time has elapsed.
Polling periods that are too small may incorrectly cancel transfers.
Using scp over paramiko allows scp transfers to be treated similarly to
sftp transfers, instead of requiring subprocesses, and provides
the ability to monitor an scp transfer using a callback as can be done
using sftp.
The custom check_output function consisted of two main parts: fetching
the subprocess required for the command, and checking its output.
It is convenient to provide functions that implement these parts
distinctly, so that the output of any subprocess can be checked easily
and the creation of a typical Popen object wrapped inside
get_subprocess.
When pushing or pulling a folder, replicate the mv/cp/scp/adb behaviour,
which is:
* splitting the destination into (existing, new) components
* if {new} component is empty, set it to the basename of the source.
* mkdir {new} if necessary
* merge the hierarchies of {src} and {existing}/{new}
