Adds 'tid' attribute to PsEntry namedtuple. This is equal to the PID of
the process.
Adds 'threads=True' parameter to target.ps(). When true, PsEntrys will be
returned for all threads on the target, not just processes. The 'tid' will
be the distinct PID for each thread, rather than the owning process.
When pushing or pulling a folder, replicate the mv/cp/scp/adb behaviour,
which is:
* splitting the destination into (existing, new) components
* if {new} component is empty, set it to the basename of the source.
* mkdir {new} if necessary
* merge the hierarchies of {src} and {existing}/{new}
Use the context manager to simplify the file transfer cache management.
Also introduce devlib.utils.misc.nullcontext() mirroring the behavior of
contextlib.nullcontext().
When globbing=True, the source is interpreted as a globbing pattern on
the target and is expanded before pulling the files or folders.
This also aligns the behaviour of all targets:
* adb connection was supported a limited form of globbing by default
(only on the last component of the path)
* SCP was supporting a limited form of globbing
* GEM5 was not supporting globbing at all
* paramiko was not supporting globbing at all
Also fix a race condition on push/pull as root, where pushing/pulling
the same file from multiple threads would have ended up using the same
temporary file.
Create a new temporary directory to use when extracting apk methods,
as when running multiple processes in parallel the extracted files could
be overwritten by each other.
Some targets can thrown stable errors in addition to
transient errors when executing the `reboot` command.
We expect this command to not always complete cleanly
so ignore all target errors.
As the exoplayer workload did not specify a pre-existing logfile, it is
created for it by default in LogcatMonitor. This default method opens
the logfile in 'byte' mode rather than the expected 'string' mode.
Regex operations that depend on the logfile for event triggering expect it to
be in 'string' mode, which was not the case.
aapt is now depreciated in favour of aapt2 therefore prefer using the
newer binary if it is found on the system. If not present fallback to
the old implementation.
Currently all invocations of aapt within devlib are compatible with
aapt2 however expose the `aapt_version` attribute to indicate which
version has been selected to allow for allow maintaining future
compatibility.
Replace stateful loop with a nested comprehension that makes obvious:
* that self._states is a dict(cpu, [CpuidleState])
* the sysfs folder being used and the constraint applied to make use
of each level (i.e. which subfolder is used)
* that the states are sorted by index
As a side effect:
* Gracefully handle non-contiguous idle state names like "state0,
state2" without a state1 (not sure if that can happen)
* Remove some antipatterns while iterating over a dict and counting
iterations.
No longer recursively attempt to push a file to the target. If the
second attempt goes wrong we assume there is something else wrong and
therefore let the error propagate.
Also log the original error in case it is not the error we were
expecting.
logcat -v lets you specify the format for the logcat output. Add a
parameter to the LogcatCollector to allow us to pass that information
down to the logcat invocation.
Paramiko uses sftp for file transfer rather then scp as with the
previous implementation however not all targets support this.
Expose a parameter to the SSHConnection to allow falling back to
the scp implementation.
In the case of connecting to a system without a password the password
parameter needs to be set to an empty string which will currently still
cause ssh keys to be loaded. Only check for ssh keys when the password
is explicitly set to `None`.
Pipe the list of perf events via cat to ensure that a pager is not
used to display the output as this can cause some systems to hang
waiting for user input.
Allow for configuring the number of connection attempts that will be
made to the device before failing to connect. This allows for waiting longer
periods of time for the device to come online.
Unless we tell pexpect to expect something it will not read from the
process' buffer, or write anything to the logfile. If we follow the
collector instructions from devlib's documentation:
In [1]: from devlib import AndroidTarget, LogcatCollector
In [2]: t = AndroidTarget()
# Set up the collector on the Target.
In [3]: collector = LogcatCollector(t)
# Configure the output file path for the collector to use.
In [4]: collector.set_output('adb_log.txt')
# Reset the Collector to preform any required configuration or
# preparation.
In [5]: collector.reset()
# Start Collecting
In [6]: collector.start()
# Wait for some output to be generated
In [7]: sleep(10)
# Stop Collecting
In [8]: collector.stop()
# Retrieved the collected data
In [9]: output = collector.get_data()
adb_log.txt will be empty because between collector.start() and
collector.stop() there were no expect() calls to
LogcatMonitor._logcat. As the get_log() function already has code to
flush the log, abstract it to a function and call it in stop() before
terminating the pexpect.spawn().
By default, pexpect.spawn() is a bytes interface: its read methods
return bytes and its write/send and expect method expect bytes. Logcat
uses utf-8 strings, so put pexpect.spawn() in str mode.
This code can't have worked in python3 before, since the logcat file
is not opened in "b" mode.
If an error occurs while executing a command the `output` and `error`
variables may not get initialised, only attempted to decode their
contents if this is not the case.
check_output(combined_output=True) does not guarantee that stdout will
come before stderr, but the ordering is needed in case check_exit_code
is True, as we are expecting the exit code at the end of stdout.
Furthermore, the exceptions can't report what is stdout and what is
stderr as they are combined.
Partially revert 77a6de94537b ("utils/android: include stderr in adb_shell
output") and parse output and err independently. Return them combined
from adb_shell() to keep the functionality that 77a6de94537b was
implementing.
d4b0dedc2a63 ("utils/misc: add combined output option to
check_output") adds an option that combines stdout and stderr, but
their order is arbitrary (stdout may appear before or after
stderr). This leads to problems in adb_shell() when it tries to check
the error code. Now that adb_shell() doesn't use combined_output,
remove the option as there are no more users in devlib.
squash! utils/misc: Make the return of check_output consistent
By default parmaiko attempts to search for SSH keys even when connecting
with a password. Ensure this is disabled to prevent issues where
non-valid keys are found on the host when connecting using password
authentication.
