For connections that allow for it (ADB and SSH using SFTP
and SCP) this change enables file transfers to be polled to check
if the transfer is still in progress after some period of time or
whether the transfer should be terminated.
If a timeout is specified in the call to ``pull`` or ``push`` then the
transfer will not be polled and will terminate ordinarily when either
the transfer completes or the timeout is reached. If a timeout is
not specified, then the transfer will be polled if ``poll_transfers`` is
set, otherwise the transfer will continue with no timeout at all.
SSH transfers supply a callback to the transfer, that is called
after every block of the source is transferred. If the callback has not
been triggered within one poll period, then the transfer is cancelled.
ADB transfers have the destination size polled every poll period, and
the size compared to the previous poll to check if the transfer has
stalled. When the destination is no longer growing in size, the poller
will attempt to kill the subprocess to end the transfer.
If the transfer is still active, but the total transfer time has
exceeded the ``total_timeout`` (default: 1 hour) the transfer will then
also be killed.
Transfer polling will only begin after the ``start_transfer_poll_delay``
time has elapsed.
Polling periods that are too small may incorrectly cancel transfers.
Using scp over paramiko allows scp transfers to be treated similarly to
sftp transfers, instead of requiring subprocesses, and provides
the ability to monitor an scp transfer using a callback as can be done
using sftp.
The custom check_output function consisted of two main parts: fetching
the subprocess required for the command, and checking its output.
It is convenient to provide functions that implement these parts
distinctly, so that the output of any subprocess can be checked easily
and the creation of a typical Popen object wrapped inside
get_subprocess.
Subprocesses that were spawned under the same pgid were not necessarily
being terminated when the parent was terminated, allowing them to
continue running. This change explicitly kills the process group
involved.
Adds the ability to set the android global setting
``stay_on_while_plugged_in``.
This setting has 4 main modes:
- 0: never stay on
- 1: stay on when plugged in to AC charger
- 2: stay on when plugged in to USB charger
- 4: stay on when wirelessly charged
These values can be OR-ed together to produce combinations.
Adds 'tid' attribute to PsEntry namedtuple. This is equal to the PID of
the process.
Adds 'threads=True' parameter to target.ps(). When true, PsEntrys will be
returned for all threads on the target, not just processes. The 'tid' will
be the distinct PID for each thread, rather than the owning process.
When pushing or pulling a folder, replicate the mv/cp/scp/adb behaviour,
which is:
* splitting the destination into (existing, new) components
* if {new} component is empty, set it to the basename of the source.
* mkdir {new} if necessary
* merge the hierarchies of {src} and {existing}/{new}
Use the context manager to simplify the file transfer cache management.
Also introduce devlib.utils.misc.nullcontext() mirroring the behavior of
contextlib.nullcontext().
When globbing=True, the source is interpreted as a globbing pattern on
the target and is expanded before pulling the files or folders.
This also aligns the behaviour of all targets:
* adb connection was supported a limited form of globbing by default
(only on the last component of the path)
* SCP was supporting a limited form of globbing
* GEM5 was not supporting globbing at all
* paramiko was not supporting globbing at all
Also fix a race condition on push/pull as root, where pushing/pulling
the same file from multiple threads would have ended up using the same
temporary file.
Create a new temporary directory to use when extracting apk methods,
as when running multiple processes in parallel the extracted files could
be overwritten by each other.
Some targets can thrown stable errors in addition to
transient errors when executing the `reboot` command.
We expect this command to not always complete cleanly
so ignore all target errors.
As the exoplayer workload did not specify a pre-existing logfile, it is
created for it by default in LogcatMonitor. This default method opens
the logfile in 'byte' mode rather than the expected 'string' mode.
Regex operations that depend on the logfile for event triggering expect it to
be in 'string' mode, which was not the case.
aapt is now depreciated in favour of aapt2 therefore prefer using the
newer binary if it is found on the system. If not present fallback to
the old implementation.
Currently all invocations of aapt within devlib are compatible with
aapt2 however expose the `aapt_version` attribute to indicate which
version has been selected to allow for allow maintaining future
compatibility.
Replace stateful loop with a nested comprehension that makes obvious:
* that self._states is a dict(cpu, [CpuidleState])
* the sysfs folder being used and the constraint applied to make use
of each level (i.e. which subfolder is used)
* that the states are sorted by index
As a side effect:
* Gracefully handle non-contiguous idle state names like "state0,
state2" without a state1 (not sure if that can happen)
* Remove some antipatterns while iterating over a dict and counting
iterations.
No longer recursively attempt to push a file to the target. If the
second attempt goes wrong we assume there is something else wrong and
therefore let the error propagate.
Also log the original error in case it is not the error we were
expecting.
logcat -v lets you specify the format for the logcat output. Add a
parameter to the LogcatCollector to allow us to pass that information
down to the logcat invocation.
Paramiko uses sftp for file transfer rather then scp as with the
previous implementation however not all targets support this.
Expose a parameter to the SSHConnection to allow falling back to
the scp implementation.
In the case of connecting to a system without a password the password
parameter needs to be set to an empty string which will currently still
cause ssh keys to be loaded. Only check for ssh keys when the password
is explicitly set to `None`.
Pipe the list of perf events via cat to ensure that a pager is not
used to display the output as this can cause some systems to hang
waiting for user input.
Allow for configuring the number of connection attempts that will be
made to the device before failing to connect. This allows for waiting longer
periods of time for the device to come online.
