dex file search is modified. It collects all the available methods under the package name. Tested with other benchmarks (geekbench,pcmark,jankbench in Android 12) as well.
* Split TransferManager and TransferHandle:
* TransferManager deals with the generic monitoring. To abort a
transfer, it simply cancels the transfer and raises an exception
from manage().
* TransferHandle provides a way for the manager to query the state
of the transfer and cancel it. It is backend-specific.
* Remove most of the state in TransferManager, along with the associated
background command leak etc
* Use a daemonic monitor thread to behave as excpected on interpreter
shutdown.
* Ensure a transfer manager _always_ exists. When no management is
desired, a noop object is used. This avoids using a None sentinel,
which is invariably mishandled by some code leading to crashes.
* Try to merge more paths in the code to uncover as many issues as
possible in testing.
* Fix percentage for SSHTransferHandle (transferred / (remaining +
transferred) instead of transferred / remaining)
* Rename total_timeout TransferManager parameter and attribute to
total_transfer_timeout to match the connection name parameter.
Instead of loosely tracking the current BackgroundCommand for a
connection in _current_bg_cmds WeakSet attribute, use a normal set and
make the BackgroundCommand deregister itself upon termination.
This allows canceling any outstanding BackgroundCommand when the
connection is closed. Currently, destroying a BackgroundCommand will not
cancel the command but devlib will simply loose track of it, and some
threads will likely fail in the background if they try to use the now
broken connection.
Ensure adb_root() always raises AdbRootError so that the caller can
catch it reliably. This is especially important since adb_root() failing
is ignored and simply triggers a fallback on using `su`. Android
production builds refuse adb root nowadays, so it's important that adb
root failures are handled well.
adb_root() restarts the server, leading to aborting commands ran by
other connections and also apparently leading to command hanging in some
situations.
Therefore, only allow adb_root() when there is only one connection
active. That should not be a big issue as this is typically called by
the first connection ever made when the Target is created.
With recent versions of adb, adb root can fail if the
daemon is already running as root.
Check the raised error message for this case and avoid
raising an error in this scenario.
Some Android devices do not have 'su'. But if they are already
rooted, there's no reason to fail. Circumvent this scenario by never
using 'su' for device already rooted.
Add Target{Stable,Transient}CalledProcessError exceptions, with an
.returncode and .output attributes, raised by Target.execute(),
mirroring subprocess.CalledProcessError.
This is very useful in client code that uses "exit N" to signal an
abnormal condition, and then inspects the output to find out more.
Handle in one place the decision of what is the real destination of each
file in push/pull operations.
The following can now be assumed by the connection:
* The destination does not exist.
* The folder containing the destination does exist.
This ensures consistent errors and behaviors across all connection
types, at the cost of:
* At least an extra execute() per source (up to 2 if the destination
is a file that needs to be removed to make room).
* For now, globbing will lead to a separate request for each file,
rather than a merged one. This is because the destination of each
source is prepared so that the connection will not have any
interpretation work to do.
The Popen object created for background command currently has no stdin
pipe, preventing the user from writing to it (it will be None instead of
being a file-like object).
Fix that by passing stdin=subprocess.PIPE to Popen constructor.
On some older versions of ps the flags used to look up
the pids are not supported and results in no output.
Use the busybox implementation for consistency.
Using dexdump from versions 30.0.1-30.0.3 of Android build tools
does not produce valid XML caused by certain APKs
Use the lxml module for parsing xml as it is more robust and
better equipped to handle errors in input.
If there is an additional file or directory in the `build_tools` directory
then WA can fail to find a working version of aapt(2), ensure that at least
one of the binaries is a valid file path.
For connections that allow for it (ADB and SSH using SFTP
and SCP) this change enables file transfers to be polled to check
if the transfer is still in progress after some period of time or
whether the transfer should be terminated.
If a timeout is specified in the call to ``pull`` or ``push`` then the
transfer will not be polled and will terminate ordinarily when either
the transfer completes or the timeout is reached. If a timeout is
not specified, then the transfer will be polled if ``poll_transfers`` is
set, otherwise the transfer will continue with no timeout at all.
SSH transfers supply a callback to the transfer, that is called
after every block of the source is transferred. If the callback has not
been triggered within one poll period, then the transfer is cancelled.
ADB transfers have the destination size polled every poll period, and
the size compared to the previous poll to check if the transfer has
stalled. When the destination is no longer growing in size, the poller
will attempt to kill the subprocess to end the transfer.
If the transfer is still active, but the total transfer time has
exceeded the ``total_timeout`` (default: 1 hour) the transfer will then
also be killed.
Transfer polling will only begin after the ``start_transfer_poll_delay``
time has elapsed.
Polling periods that are too small may incorrectly cancel transfers.
When globbing=True, the source is interpreted as a globbing pattern on
the target and is expanded before pulling the files or folders.
This also aligns the behaviour of all targets:
* adb connection was supported a limited form of globbing by default
(only on the last component of the path)
* SCP was supporting a limited form of globbing
* GEM5 was not supporting globbing at all
* paramiko was not supporting globbing at all
Also fix a race condition on push/pull as root, where pushing/pulling
the same file from multiple threads would have ended up using the same
temporary file.
Create a new temporary directory to use when extracting apk methods,
as when running multiple processes in parallel the extracted files could
be overwritten by each other.
As the exoplayer workload did not specify a pre-existing logfile, it is
created for it by default in LogcatMonitor. This default method opens
the logfile in 'byte' mode rather than the expected 'string' mode.
Regex operations that depend on the logfile for event triggering expect it to
be in 'string' mode, which was not the case.
aapt is now depreciated in favour of aapt2 therefore prefer using the
newer binary if it is found on the system. If not present fallback to
the old implementation.
Currently all invocations of aapt within devlib are compatible with
aapt2 however expose the `aapt_version` attribute to indicate which
version has been selected to allow for allow maintaining future
compatibility.
logcat -v lets you specify the format for the logcat output. Add a
parameter to the LogcatCollector to allow us to pass that information
down to the logcat invocation.
Allow for configuring the number of connection attempts that will be
made to the device before failing to connect. This allows for waiting longer
periods of time for the device to come online.
Unless we tell pexpect to expect something it will not read from the
process' buffer, or write anything to the logfile. If we follow the
collector instructions from devlib's documentation:
In [1]: from devlib import AndroidTarget, LogcatCollector
In [2]: t = AndroidTarget()
# Set up the collector on the Target.
In [3]: collector = LogcatCollector(t)
# Configure the output file path for the collector to use.
In [4]: collector.set_output('adb_log.txt')
# Reset the Collector to preform any required configuration or
# preparation.
In [5]: collector.reset()
# Start Collecting
In [6]: collector.start()
# Wait for some output to be generated
In [7]: sleep(10)
# Stop Collecting
In [8]: collector.stop()
# Retrieved the collected data
In [9]: output = collector.get_data()
adb_log.txt will be empty because between collector.start() and
collector.stop() there were no expect() calls to
LogcatMonitor._logcat. As the get_log() function already has code to
flush the log, abstract it to a function and call it in stop() before
terminating the pexpect.spawn().
By default, pexpect.spawn() is a bytes interface: its read methods
return bytes and its write/send and expect method expect bytes. Logcat
uses utf-8 strings, so put pexpect.spawn() in str mode.
This code can't have worked in python3 before, since the logcat file
is not opened in "b" mode.
check_output(combined_output=True) does not guarantee that stdout will
come before stderr, but the ordering is needed in case check_exit_code
is True, as we are expecting the exit code at the end of stdout.
Furthermore, the exceptions can't report what is stdout and what is
stderr as they are combined.
Partially revert 77a6de94537b ("utils/android: include stderr in adb_shell
output") and parse output and err independently. Return them combined
from adb_shell() to keep the functionality that 77a6de94537b was
implementing.
* Unify the behavior of background commands in connections.BackgroundCommand().
This implements a subset of subprocess.Popen class, with a unified behavior
across all connection types
* Implement the SSH connection using paramiko rather than pxssh.
The `AndroidTarget` class should not depend on ADB specific commands as
is is possible to use this target with other connection types e.g. ssh.
Therefore move the adb specific commands into the `AdbConnection`.
- `wait_for_device` and `reboot_bootloader` are now exposed in AndroidTarget
as generic methods and call through to the connection method.
- `adb_kill_server` is now a standalone function of the AdbConnection.
The `AndroidTarget` would ensure that when connecting to a IP target
that it disconnected first to prevent the connection getting stuck if
the connection was not closed correctly. Move this code into the
`AdbConnection` instead as this is more relevant.
To allow for connecting to an `AndroidTarget` as root before the target
has been initialised, allow for passing `adb_as_root` as a connection
parameter to the `AdbConnection`. This will restart `adbd` as root
before attempting to connect to the target and will restart as unrooted
once all connections to that target have been closed.
Add a method to `AdbConnection` to control whether whether adb is
connected as root. This allows for the connection to track whether it is
connected as root for a particular device across all instances of a
connection.
The fix in commit 964fde2 caused issues with certain command structures,
for example running in the background. To prevent this run the original
command as a subshell.
When executing a command using `su`, the `echo` command was returning the
error code of the invocation of `su` rather than the command itself.
Usually `su` should mimic the return code of the command it is executing
however this is not always the case which can cause issues.
