* Split TransferManager and TransferHandle:
* TransferManager deals with the generic monitoring. To abort a
transfer, it simply cancels the transfer and raises an exception
from manage().
* TransferHandle provides a way for the manager to query the state
of the transfer and cancel it. It is backend-specific.
* Remove most of the state in TransferManager, along with the associated
background command leak etc
* Use a daemonic monitor thread to behave as excpected on interpreter
shutdown.
* Ensure a transfer manager _always_ exists. When no management is
desired, a noop object is used. This avoids using a None sentinel,
which is invariably mishandled by some code leading to crashes.
* Try to merge more paths in the code to uncover as many issues as
possible in testing.
* Fix percentage for SSHTransferHandle (transferred / (remaining +
transferred) instead of transferred / remaining)
* Rename total_timeout TransferManager parameter and attribute to
total_transfer_timeout to match the connection name parameter.
Make BackgroundCommand.__init__() poll all current BackgroundCommands on
the associated connection so they deregister themselves if they are
completed.
This ensures that a BackgroundCommand-heavy application that also does
not close them properly will not accumulate useless instances forever
and leak associated resources like Popen objects.
Instead of loosely tracking the current BackgroundCommand for a
connection in _current_bg_cmds WeakSet attribute, use a normal set and
make the BackgroundCommand deregister itself upon termination.
This allows canceling any outstanding BackgroundCommand when the
connection is closed. Currently, destroying a BackgroundCommand will not
cancel the command but devlib will simply loose track of it, and some
threads will likely fail in the background if they try to use the now
broken connection.
kick_off() implementation had a number of issue:
* Target specific implementation making testing more difficult.
* Not wrapping the command in sh -c lead to blocking behavior if the
command had multiple parts, e.g. "sleep 42; echo foo"
* nohup sometimes writes to stdout, breaking return code parsing in
adb_shell().
These issues are fixed by a generic implementing of kick_off() that
simply delegates to Target.background().
Fixes https://github.com/ARM-software/devlib/issues/623
trace-cmd start -B devlib -b 42 will set the buffer size for the
"devlib" ftrace instance but will not set the buffer size of the
top-level buffer.
Unfortunately, some events still end up in the top-level buffer
regardless of any configuration such as the "print" event. This can lead
to lost events because the buffer size was too small.
Avoid that by using the buffer size for both top-level and devlib's
instance.
Ensure adb_root() always raises AdbRootError so that the caller can
catch it reliably. This is especially important since adb_root() failing
is ignored and simply triggers a fallback on using `su`. Android
production builds refuse adb root nowadays, so it's important that adb
root failures are handled well.
FtraceCollector.available_events is not memoized anymore as the set of
events supported by the target can change dynamically (e.g. loading a
kernel module).
This means that calling self.available_events is somewhat expensive, so
avoid doing it in a loop. Instead, save the events in a variable and
reuse it in the function to save a substantial amount of time.
Use a buffer named "devlib" instead of using the top-level default
buffer. That will improve interop with other tools trying to collect a
trace at the same time as devlib.
Replaced all references to devlib.Target with devlib.LinuxTarget to correctly
utilise appropriate interface.
Also added relevant functionality to corectly
create the root directories of the CGroup hierarchies, handling any error that
occurs in that case.
The value for `max_async` when creating a target was being ignored
if a connection was not established as part of the __init__ method.
Save this value for use via `connect` if called directly.
Remove the "future" dependency as devlib does not support Python 2
anymore.
Also remove the "from __future__ import division" as this is the default
in Python 3.
* Remove check_output_lock as the issue has been fixed in Python 3.4
* Use Popen process as a context manager. Technically,
Popen.communicate() already achieves those but the context manager
will ensure this is done even if an exception happens at any point.
adb_root() restarts the server, leading to aborting commands ran by
other connections and also apparently leading to command hanging in some
situations.
Therefore, only allow adb_root() when there is only one connection
active. That should not be a big issue as this is typically called by
the first connection ever made when the Target is created.
Add a memoize_method decorator that works for async methods. It will not
leak memory since the memoization cache is held in the instance
__dict__, and it does not rely on hacks to hash unhashable data.
use_governor() was trying to set concurrently both per-cpu and global tunables for
each governor, which lead to a write conflict.
Split the work into the per-governor global tunables and the per-cpu
tunables, and do all that in concurrently. Each task is therefore
responsible of a distinct set of files and all is well.
Also remove @memoized on async functions. It will be reintroduced in a
later commit when there is a safe alternative for async functions.
The conncetion returned by Target.get_connection() does not have its
.busybox attribute initialized. This is expected for the first
connection, but connections created for new threads should have busybox
set.
Currently the cgroups module will pull all available controllers from
/proc/cgroups and then try to mount them, including the disabled ones.
This will result in the entire mount failing.
Lines in /proc/cgroups ending in 0 correspond to disabled controllers.
Filtering those out solves the issue.
With recent versions of adb, adb root can fail if the
daemon is already running as root.
Check the raised error message for this case and avoid
raising an error in this scenario.
When no entry has been recorded by the collector, return an empty string
rather than returning the full dmesg log.
Also fix get_data() that would fail try to add None + '\n' if dmesg_out
property returns None.
Both move_all_tasks_to() and move_tasks() take a list of grep patterns
to exclude.
It turned out that move_all_tasks_to() was calling move_tasks() with a
string instead of a list, leading to broken quoting.
Fix that by passing the pattern list to move_tasks() and let
move_tasks() add the "-e" option in front of it. Also add a
DeprecationWarning in move_tasks() if someone passes a string instead of
an iterable of strings.
