1
0
mirror of https://github.com/ARM-software/devlib.git synced 2024-10-05 18:30:50 +01:00
Commit Graph

1047 Commits

Author SHA1 Message Date
Douglas Raillard
678822f9e4 utils/misc: Cleanup check_output()
* Remove check_output_lock as the issue has been fixed in Python 3.4
* Use Popen process as a context manager. Technically,
  Popen.communicate() already achieves those but the context manager
  will ensure this is done even if an exception happens at any point.
2022-08-22 09:32:11 +01:00
Douglas Raillard
be734140b3 utils/android: Make AdbConnection.active_connections thread safe
Add a lock to serialize access to the dictionary.
2022-08-22 09:32:11 +01:00
Douglas Raillard
b988e245d9 utils/android: Fix AdbConnection.adb_root()
adb_root() restarts the server, leading to aborting commands ran by
other connections and also apparently leading to command hanging in some
situations.

Therefore, only allow adb_root() when there is only one connection
active. That should not be a big issue as this is typically called by
the first connection ever made when the Target is created.
2022-08-22 09:32:11 +01:00
Douglas Raillard
b7ef2dc2e0 devlib.target: Fix AndroidTarget unpickle
Fix __setstate__ to call super().__setstate__ in order to handle the
generic part of the deserialization.
2022-08-17 10:53:43 +01:00
Douglas Raillard
492284f46d module/cpufreq: Fix typo
Fix per-cpu/global cpufreq governor tunable setting by replacing a
"pass" into a "continue".

Also name some futures to improve error reporting.
2022-08-09 14:02:17 +01:00
Douglas Raillard
fefdf29ed8 utils/asyn: Add memoize_method() decorator
Add a memoize_method decorator that works for async methods. It will not
leak memory since the memoization cache is held in the instance
__dict__, and it does not rely on hacks to hash unhashable data.
2022-07-28 14:40:15 +01:00
Douglas Raillard
0ea9c73ec0 module/cpufreq: Fix async use_governor()
use_governor() was trying to set concurrently both per-cpu and global tunables for
each governor, which lead to a write conflict.

Split the work into the per-governor global tunables and the per-cpu
tunables, and do all that in concurrently. Each task is therefore
responsible of a distinct set of files and all is well.

Also remove @memoized on async functions. It will be reintroduced in a
later commit when there is a safe alternative for async functions.
2022-07-28 14:40:15 +01:00
Douglas Raillard
2c4b16f280 devlib: Use async Target API
Make use of the new async API to speedup other parts of devlib.
2022-07-28 14:40:15 +01:00
Douglas Raillard
18ab9f80b0 target: Expose Target(max_async=50) parameter
Allow the user to set a maximum number of conrruent connections used to
dispatch non-blocking commands when using the async API.
2022-07-28 14:40:15 +01:00
Douglas Raillard
92f58e4e7a target: Enable async methods
Add async variants of Target methods.
2022-07-28 14:40:15 +01:00
Douglas Raillard
bdf8b88ac7 utils/async: Add new utils.async module
Home for async-related utilities.
2022-07-28 14:40:15 +01:00
Douglas Raillard
1da174a438 setup.py: Require Python >= 3.7
Require Python >= 3.7 in order to have access to a fully fledged asyncio
module.
2022-07-28 14:40:15 +01:00
Douglas Raillard
3c9804a45b setup.py: cleanup dependencies in setup.py
Remove dependencies that are ruled out due to the current Python minimal
version requirement.
2022-07-28 14:40:15 +01:00
Douglas Raillard
3fe105ffb7 target: Make __getstate__ more future-proof
Remove all the tls_property from the state, as they will be recreated
automatically.
2022-07-28 14:40:15 +01:00
Douglas Raillard
9bd76fd8af target: Fix Target.get_connection()'s busybox
The conncetion returned by Target.get_connection() does not have its
.busybox attribute initialized. This is expected for the first
connection, but connections created for new threads should have busybox
set.
2022-07-28 14:40:15 +01:00
Douglas Raillard
ef9384d161 utils.misc: Make nullcontext work with asyncio
Implement __aenter__ and __aexit__ on nullcontext so it can be used as
an asynchronous context manager.
2022-07-28 14:40:15 +01:00
Kajetan Puchalski
ff2268b715 module/cpuidle: Add listing & setting governors
Add support for listing the currently available idle governors and
setting the currently used one through sysfs.
2022-07-19 09:33:36 +01:00
Kajetan Puchalski
5042f474c2 module/cgroups: Skip disabled cgroup controllers
Currently the cgroups module will pull all available controllers from
/proc/cgroups and then try to mount them, including the disabled ones.
This will result in the entire mount failing.

Lines in /proc/cgroups ending in 0 correspond to disabled controllers.
Filtering those out solves the issue.
2022-07-19 09:33:23 +01:00
Marc Bonnici
a585426924 android: Don't error if ADB is already running as root
With recent versions of adb, adb root can fail if the
daemon is already running as root.

Check the raised error message for this case and avoid
raising an error in this scenario.
2022-06-22 11:23:15 +01:00
Marc Bonnici
1196e336a5 version: bump minor version number
Re-bump the minor version to prepare for dropping
Python < 3.7 support.
2022-05-24 17:50:01 +01:00
Marc Bonnici
f525374fbb version: perform additional revision release
Revert the minor version number to allow release of additional
revision release to fix some bugs that made it into the previous
release.
2022-05-24 17:50:01 +01:00
Douglas Raillard
42e62aed57 target: Fix AndroidTarget pickling
Avoid pickling the "clear_logcat_lock". Instead, discard the attribute
and re-initialize it anew.
2022-05-24 10:39:31 +01:00
Douglas Raillard
f5cfcafb08 shutils: Remove shebang
Since shutils should be run using busybox shell anyway, remove the
shebang.
2022-05-24 10:37:17 +01:00
Douglas Raillard
7853d2c85c target: Run shutils.in in busybox
Ensure shutils.in runs in a busybox shell.
2022-05-24 10:37:17 +01:00
Douglas Raillard
a9fcc75f60 collector/dmesg: Fix dmesg_out property
When no entry has been recorded by the collector, return an empty string
rather than returning the full dmesg log.

Also fix get_data() that would fail try to add None + '\n' if dmesg_out
property returns None.
2022-05-18 15:21:18 +01:00
Douglas Raillard
cd8720b901 module/cgroups: Fix move_tasks()/move_all_tasks_to()
Both move_all_tasks_to() and move_tasks() take a list of grep patterns
to exclude.

It turned out that move_all_tasks_to() was calling move_tasks() with a
string instead of a list, leading to broken quoting.

Fix that by passing the pattern list to move_tasks() and let
move_tasks() add the "-e" option in front of it. Also add a
DeprecationWarning in move_tasks() if someone passes a string instead of
an iterable of strings.
2022-05-17 19:04:29 +01:00
Marc Bonnici
03569fb01f version: Bump minor version number
This next release will drop support for Python < 3.7
therefore bump to a dev tag of the next minor version.
2022-04-29 19:38:50 +01:00
Marc Bonnici
22f53f117e version: Bump revision number 2022-04-29 19:38:50 +01:00
Douglas Raillard
e0abb9db48 collector/dmesg.py: Allow nesting DmesgCollector
Rather than systematically clearing the buffer on reset(), record the
timestamp of the last entry and use it to filter-out old entries in
DmesgCollector.entries property.

This also allows detecting if the ring buffer has ran out of memory, or
if something has cleared the buffer while collecting, leading to missing
entries.
2022-04-29 13:59:24 +01:00
Vincent Donnefort
c29d386e81 target: Allow relative path for for push/pull
Currently, it is not possible to push/pull files with a relative path when
the destination doesn't exist. This is due to the basename resolution. Fix
this behaviour.
2022-04-28 13:22:25 +01:00
Douglas Raillard
adad59fdba target: Add Target.makedirs(as_root=False) parameter
Add as_root parameter to Target.makedirs()
2022-04-28 12:55:23 +01:00
Douglas Raillard
48329b7891 target: Add exception message when module fails to load
Log the exception message when a target module fails to load.
2022-04-28 12:55:23 +01:00
Douglas Raillard
728b59ad7e utils/ssh.py: Make SshConnection._background() more robust
Raise a better exception when e.g. sudo command is not found.
2022-04-28 12:55:23 +01:00
Douglas Raillard
2b38548463 connection: Use -- in kill command
Separate options and PGIDS with -- in kill commands as otherwise, kill
seems to be confused.
2022-04-28 12:55:23 +01:00
Douglas Raillard
bdb04aa8d0 ftrace: Detect tracefs mount point
Change default FtraceCollector(tracing_path=...) to None, and
auto-detect mount point when None is given.

Also expose an FtraceCollector.find_tracing_path() method so that user
code can also access this path without having to instantiate an
FtraceCollector.
2022-04-22 18:29:53 +01:00
Douglas Raillard
8f80d8a5ee shutils.in: De-hardcode tracefs location
Detect tracefs mount point rather than hardcoding its path.
2022-04-22 18:29:53 +01:00
Douglas Raillard
ff599dfbb6 shutils.in: Simplify the dispatcher
Check that the function exists and then run it, to avoid endless
copy-pasting.

Also call it with "$@", which will achieve proper CLI params forwarding
unlike "$*" which will not.
2022-04-22 18:29:53 +01:00
Douglas Raillard
00a5bcb377 shutils.in: Add get_fs_mount_point() function
Allow getting the mount point of a given filesystem. This is useful to
detect the location of e.g. tracefs
2022-04-22 18:29:53 +01:00
Marc Bonnici
fffa040792 target/xfer: Fix detection of files with restricted permissions
The command used to detect the presence of a filepath can return
the wrong value if only accessible by the superuser.
Pass the `as_root` parameter to the detection function to ensure
that files that are to be pulled with elevated permissions are
also queried with elevated permission.
2022-04-06 18:41:09 +01:00
Vincent Donnefort
7d6ed2dd8a utils.android: force as_root=False when connected_as_root
Some Android devices do not have 'su'. But if they are already
rooted, there's no reason to fail. Circumvent this scenario by never
using 'su' for device already rooted.
2022-04-05 12:55:20 +01:00
Douglas Raillard
f6bbd2c187 modules/sched: Use correct permissions to read debugfs
Use target.list_directory(as_root=target.is_rooted) instead of doing it
as a normal user for paths in /sys/kernel/debug. Since this
list_directory() call can be used with multiple path, we do not force
as_root=True but we increase the chance of it working.
2022-04-05 12:54:59 +01:00
Douglas Raillard
a65189f028 target: Replace Target.__copy__ by __getstate__
__getstate__ is also used by the copy module, but allows pickling the
class as well. This is useful when using the multiprocessing API, which
requires pickling the Target object to send it to the new process.
2021-11-03 10:56:09 +00:00
Marc Bonnici
77f0b1f06d docs: Add readthedocs config and requirements.txt
The default versions used for sphinx and docuilts on
readthedocs are no longer compatible. Explicitly list
the package versions that should be used when building
the documentation.
2021-10-28 11:03:08 +01:00
Douglas Raillard
b4c76007c8 ftrace: Do not memoize FtraceCollector.available_events
ftrace events can be added and removed dynamically by kernel modules, so
they cannot be memoized.
2021-10-27 11:56:23 +01:00
Douglas Raillard
3f92d92a3f ssh: Reduce number of opened channels
SSH servers seem to have a maximum number of opened channels, after
which paramiko will raise an exception:

    Could not open an SSH channel: ChannelException(2, 'Connect failed')

Memoizing the SFTPClient object based on the timeout setting leads to
many opened sessions, since the timeout is typically adjusted, e.g. to
match the size of the file when pulling an ftrace trace.dat file.

Solve that by memoizing the SFTPClient based only on the connection
object, with a maximum number of 1 cached object, and update its timeout
setting inplace.
2021-10-15 16:10:02 +01:00
Douglas Raillard
0e0417c6b3 target: Fix undefined function
Remove references to non-existent dst_path_exists() function in
target.py
2021-10-11 14:42:25 +01:00
Douglas Raillard
e979bafb50 target: Speedup Target.write_value()
Avoid an execute() by doing the check in the same command. This also
allows to return early if the write is fast, and to extend for longer if
the write is slow. The speed at which you can observe a write in sysfs
depends on the backing kernel handlers, so there is a wide variety of
situations.

Also, make a more fine grained error detection by allowing the write
itself to fail, which can happen when writing invalid values to sysfs.
2021-10-08 18:22:04 +01:00
Douglas Raillard
0c1878786b Target.execute(): Add .returncode and .output exception attributes
Add Target{Stable,Transient}CalledProcessError exceptions, with an
.returncode and .output attributes, raised by Target.execute(),
mirroring subprocess.CalledProcessError.

This is very useful in client code that uses "exit N" to signal an
abnormal condition, and then inspects the output to find out more.
2021-10-08 18:22:04 +01:00
Douglas Raillard
ff57e785f8 exception: Deal with missing _message attribute
Allow subclasses of DevlibError to not have a _message attribute, in
which case it falls back on str(self), just as when _message is None.
2021-10-08 18:22:04 +01:00
Douglas Raillard
f1c8ca1a66 utils.android: Separate stdout and stderr by newline
Following what the other connections are doing, make the android
connection separate stdout and stderr by a newline.
2021-10-08 18:22:04 +01:00