Replace as many uses of tempfile() by make_temp() as possible, as the
latter provide more reliable resource control by way of a context
manager. This also paves the way to having a single point in devlib
where temporary files are created, simplifying maintenance.
Align the parameters between the 2 methods:
* Use "None" as default value
* Do not add suffix or prefix if not asked for.
* Separate components with "-" instead of "_"
Also avoid a "None" prefix when no prefix is asked for, and set None as
the default prefix value.
Remove the "devlib-test" default value as make_temp() has nothing to do
with tests.
* Make use of Target.make_temp() in Target._xfer_cache_path() to
deduplicate temp folder creation code
* Introduce Target.tmp_directory attribute for the root of temporary
files.
* Make Target.tempfile() use Target.tmp_directory
* Rework the Target._resolve_paths() implementations:
* Target.tmp_directory is set to a default returned by "mktemp -d".
This way, if "mktemp -d" works out of of the box, all devlib
temporary files will be located in the expected location for the
that operating system.
* Target._resolve_paths() must set Target.working_directory and that
is checked with an assert. Other directories have defaults based
on this if _resolve_paths() does not set them.
Allow using trace-cmd record that continuously dump the trace to disk,
allowing to overcome the buffer size limitations when recording for
extended periods of time.
Support sending any signal to background commands, instead of only
supporting properly SIGKILL/SIGTERM/SIGQUIT.
The main issue is figuring out what PID to send the signal to, as the
devlib API allows running a whole snippet of shell script that typically
is wrapped under many layers of sh -c and sudo calls. In order to lift
the ambiguity, the user has access to a "devlib-signal-target" command
that points devlib at what process should be the target of signals:
# Run a "setup" command, then the main command that will receive the
# signals
cmd = 'echo setup; devlib-signal-target echo hello world'
with target.background(cmd) as bg:
bg.communicate()
The devlib-signal-target script can only be used once per background
command, so that it is never ambiguous what process is targeted, and so
that the Python code can cache the target PID. Subsequent invocations
of devlib-signal-target will fail.
Make cpu_frequency_devlib dependent on whether the "cpu_frequency" event
has been selected rather than dependent on the cpufreq devlib module
being loaded on the target.
The old behavior became particularly problematic with the lazy loading
of modules. However, it was never a reliable way of knowing if the user
was interested in the frequency or not.
Apply a similar mechanism for the extra idle state transitions only done
if "cpu_idle" event is selected.
Emiting the current frequency of all CPUs in the stop() hook is not
useful as the current frequency should already be known from the trace.
Either the event is emitted every time the frequency changes and the
up-to-date information is available, or the frequency never changes
(e.g. userspace governor) and the frequencies will be known from
emitting cpu_frequency_devlib in start().
_prepare_xfer() deals with all the paths resulting from glob expansion,
so it can benefit from async capabilities in order to process multiple
files concurrently.
Convert the internals to async/await to enable useful map_concurrently()
When pulling a file from the target, copy all paths as files and follow
symlinks if necessary. That fixes issues related to chmod not working on
symlinks and generally allows getting any path.
If we want to one day preserve symlinks in some capacities, we can
always add an option to Target.pull() to do so.
Paramiko seems to have had a slight change in behavior that broke
devlib: to save a remote command execution, we attempt to pull any path
as file first, and then as a folder if the former failed.
This is now broken as paramiko will create an empty destination file
when trying to pull as a file. When we attempt again to pull as folder,
the destination exists already (empty file) and we raise an exception.
To fix that, make sure we cleanup any attempt after pulling as a file
before trying again.
The logic in Target.disconnect() appears to have been duplicated by
error. While _probably_ harmless, this is at least confusing, and since
this happens outside of the lock, this may actually be a real problem.
The current behavior is to issue "adb unroot" if the device needed to be
rooted upon connection. This breaks use of nested Targets, which LISA
requires as some target interaction needs to happen in a subprocess.
Fix that by restoring the same adb root state that there was when
creating the connection, rather than blindly unrooting the device upon
disconnection.
Check that no element in the chain adds any unwanted content to stdout
or stderr when running a command. This is especially important as PAM
modules can just write arbitrary messages to stdout when using sudo,
such as password expiry notification. There unfortunately seems to be no
way of silencing it, but we can at least catch it before it triggers
errors down the line.
Since the prompt is added to stdout, remove the one-space-prompt that
currently corrupts stdout when a command is ran with sudo.
That non-empty prompt was added as Windows Subsystem for Linux version 1
(WSL1) has a broken sudo implementation that chokes on an empty prompt.
Considering this is not a platform that is normally suported by devlib,
we re-introduce that empty prompt.
Some drivers emit broken multiline dmesg output (with some Windows-style
newline ...) . In order to parse the rest of the content, allow not
raising on such input.
FtraceCollector.reset() executes 'trace-cmd reset ..' command which
clears all kprobes. This breaks tracing existing kprobe events (if any).
Thus, save kprobe events before trace-cmd reset and restore them after
the reset operation.
For the context, I want to trace an ordinary function in kernel (e.g.,
"echo 'p do_sys_open' > /sys/kernel/tracing/kprobe_events"). However,
FtraceCollector.reset() destroys kprobes, too. Preserving existing
kprobes allows me to use FtraceCollector class as is.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
inspect.iscoroutinefunction() currently detects
_AsyncPolymorphicFunction() as being a coroutine function since it
inspects x.__code__.co_flags to determine so. Since we delegate
attribute access to the async function, it makes
_AsyncPolymorphicFunction() appear as being a coroutine function even
though it is not.
Fix that by directing __code__ attribute access to __call__'s code.
Provide an implementation of re-entrant asyncio.run() that is less
brittle than what greenback provides (e.g. no use of ctypes to poke
extension types).
The general idea of the implementation consists in treating the executed
coroutine as a generator, then turning that generator into a generator
implemented using greenlet. This allows a nested function to make the
top-level parent yield values on its behalf, as if every call was
annotated with "yield from".
Once a thread exits, the connection instance it was using can be
returned to the pool so it can be reused by another thread.
Since there is no per-thread equivalent to atexit, this is achieved by
returning the connection to the pool after every top-level method call
that uses it directly, so the object the user can get by accessing
Target.conn can change after each call to Target method.
cpu-checker was planned to detect availability of KVM acceleration in
QEMU by running kvm-ok command. However, the implementation diverged
from plan and made cpu-checker redundant. Thus, remove it from apt
package list.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Some ADB servers may use non-standard port number. Hence, add 'adb_port'
property to AndroidTarget class and pass port number down to
adb_command().
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Apparently commit
492d42dddb63 ("target: tests: Address review comments on PR#667")
erroneously renamed target_configs.yaml to target_configs.yml.
Rename it to test_config.yml.
Also address 2 Docker warnings related to environment variables while we
are here.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
install_base.sh is left-over from LISA/install_base.sh. Scope of the
script in question is different (and potentially can diverge more) than
its root in LISA. Hence, give it a more descriptive (hopefully) name.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Both devlib and LISA utilizes install_base.sh script, but they install
different packages and support different input arguments. Also support
custom ANDROID_HOME environment variable in order to let LISA (or just
let users install Android SDK/tools wherever they want) choose install
location.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Apparently skins are just nice to have. Also devlib uses emulated
devices in command line (no GUI), so skins are unnecessary. Removing
skins will also reduce the disparity in install_base.sh scripts of LISA
and devlib.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Make sure devlib/install_base.sh has complete Android SDK support. This
will be the first step of removing duplicate Android SDK installation
functions from LISA/install_base.sh.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Just a house-keeping patch to do some trivial improvements:
- Move global variables to the beginning of the script
- Eliminate redundant echo commands
- Tidy up the system package list
- Drop superfluous ';'
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Raise an exception allowing diagnosis when a dmesg line does not match
the regex it is supposed to, rather than the cryptic groups()
AttributeError on None.
Use atexit handler to run Target.disconnect() when the process is about
to exit. This avoids running it with a half torn down namespace, with
ensuing exceptions and non-clean disconnect.
Allow cleanly disconnecting the Target object, so that we don't get
garbage output from __del__ later on when half of the namespace has
already disappeared.
On some systems this seems to have no effect, leaving the executed shell in the root cgroup. Before, this function would still execute and the end user would think the desired process was run in the cgroup when infact it had not.
Log any error happening in adb command ran by _ping() so it can be diagnosed.
Also fix possible deadlock by not using subprocess.PIPE along
subprocess.call(), as the documentation recommends against it.
PR#668: https://github.com/ARM-software/devlib/pull/668
- Fix mixed tab-space white-spacing issues
- s/CMDLINE_VERSION/ANDROID_CMDLINE_VERSION/ to be more precise
- s/set_host_arch/get_android_sdk_host_arch/ because the global variable
for Android host architecture is removed now
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
