1
0
mirror of https://github.com/ARM-software/devlib.git synced 2024-10-05 18:30:50 +01:00
Commit Graph

846 Commits

Author SHA1 Message Date
Marc Bonnici
7f5e0f5b4d utils/version: Bump dev version
Bump the development version due to the change in SSH interface.
2020-03-06 17:34:22 +00:00
Douglas RAILLARD
7e682ed97d target: Check that the connection works cleanly upon connection
Check that executing the most basic command works without troubles or stderr
content. If that's not the case, raise a TargetStableError.
2020-03-06 17:33:04 +00:00
Douglas RAILLARD
62e24c5764 connections: Unify BackgroundCommand API and use paramiko for SSH
* Unify the behavior of background commands in connections.BackgroundCommand().
  This implements a subset of subprocess.Popen class, with a unified behavior
  across all connection types

* Implement the SSH connection using paramiko rather than pxssh.
2020-03-06 17:33:04 +00:00
Douglas RAILLARD
eb6fa93845 utils/misc: Add redirect_streams() helper
Update a command line to redirect standard streams as specified using the
parameters. This helper allows honoring streams specified in the same way as
subprocess.Popen, by doing it as much using shell redirections as possible.
2020-03-06 17:33:04 +00:00
Douglas RAILLARD
9d5d70564f target: Use tls_property() to manage a thread-local connection
This frees the connection to have to handle threading issues, since each thread
using the Target will have its own connection. The connection will be garbage
collected when the thread using it dies, avoiding connection leaks.
2020-03-06 17:33:04 +00:00
Douglas RAILLARD
922686a348 utils/misc: Add tls_property()
Similar to a regular property(), with the following differences:
* Values are memoized and are threadlocal

* The value returned by the property needs to be called (like a weakref) to get
  the actual value. This level of indirection is needed to allow methods to be
  implemented in the proxy object without clashing with the value's methods.

* If the above is too annoying, a "sub property" can be created with the regular
  property() behavior (and therefore without the additional methods) using
  tls_property.basic_property .
2020-03-06 17:33:04 +00:00
Douglas RAILLARD
98e2e51d09 devlib.utils.misc: Use Popen.communicate(timeout=...) in check_output
Use the timeout parameter added in Python 3.3, which removes the need for the
timer thread and avoids some weird issues in preexec_fn, as it's now documented
to sometimes not work when threads are involved.
2020-03-06 17:33:04 +00:00
Javi Merino
92e16ee873 instrument/daq: Add an explicit time column to the DAQ measurements
Add the monotonic clock time to the energy measurements to help
correlate the measurement with those of other collectors, like
FtraceCollector or LogcatCollector.
2020-03-02 14:48:46 +00:00
Javi Merino
72ded188fa instrument/daq: Convert reading rows from all files to a generator
Instead of calling _read_next_rows() before the while and at the end,
it's simpler to read the rows in a for loop and have _read_rows() be a
generator.
2020-03-02 14:48:46 +00:00
Javi Merino
dcab0b3718 instrument/daq: Check that self.tempdir has been set before calling os.path.isdir()
self.tempdir is initialized to None, and os.path.isdir() throws an
exception if you don't pass it a str.  This can happen if teardown()
is called before get_data(), which WA sometimes does.  Check that
self.tempdir has been initializing before calling os.path.isdir().
2020-03-02 14:48:46 +00:00
Vincent Donnefort
37a6b4f96d target: a valid sha1 must be concatenated with the kernel version
Some SoC vendors add several sha's to the kernel version string. This is
problematic for the KernelVersion class, which might identify the wrong one.

Fixing this issue by matching the following "git describe" pattern:

  <version>.<major>.<minor>-<rc>-<commits>-g<sha1>

Where commits is the number of commits on top of the tag, which is now a
member of the class.

Prior to this patch:

>> KernelVersion("4.14.111-00001-gd913f26_audio-00003-g3ab4335").sha1
3ab4335
>>> KernelVersion("4.14.111_audio-00003-g3ab4335").sha1
3ab4335

With the modified regex:

>> KernelVersion("4.14.111-00001-gd913f26_audio-00003-g3ab4335").sha1
d913f26
>>> KernelVersion("4.14.111_audio-00003-g3ab4335").sha1
None
2020-02-28 13:10:31 +00:00
Marc Bonnici
1ddbb75e74 uilts/android: Fix parameters to adb_kill_server 2020-02-20 16:25:50 +00:00
Marc Bonnici
696dec9b91 utils/android: Ensure that adb_server is propergated to helper functions
Ensure that we use the correct `adb_server` in the adb helper functions.
2020-02-20 16:25:50 +00:00
Douglas RAILLARD
17374cf2b4 target: Update Target.modules from Target.install_modules()
Make sure the target.modules list stays up to date when a new module is
installed, since behaviors like devlib_cpu_frequency event injection rely on
content of target.modules.
2020-02-19 09:15:38 +00:00
Douglas RAILLARD
9661c6bff3 target: Handle non-existing /sys/devices/system/node
Some systems (ARM 32bits it seems) don't have this file in sysfs. Assume 1 node
in that case.
2020-01-22 09:21:56 +00:00
Douglas RAILLARD
0aeb5bc409 target: Remove use of ls
Using "ls" in scripts is highly discouraged:
http://mywiki.wooledge.org/ParsingLs
2020-01-22 09:21:56 +00:00
Javi Merino
a5640502ac devlib/AndroidTarget: Allow passing format options to dump_logcat()
logcat has a modifier for its output format.  Add a logcat_format
parameter to dump_logcat() so that we can pass it on to logcat.
2020-01-21 09:37:07 +00:00
Douglas RAILLARD
6fe78b4d47 module/cpufreq: Sort list of frequencies
Ensure the order of frequencies is deterministic to have consistent output when
printing it or when using it to carry out some actions.
2020-01-15 11:36:55 +00:00
Sergei Trofimov
5bda1c0eee host: add host attribute to LocalConnection
Add a host attribute (hard-coded to "localhost") to LocalConnection to
make it easier to seamlessly swap it out with ssh connection.
2020-01-09 16:59:13 +00:00
Ambroise Vincent
0465a75c56 devlib/trace/ftrace.py: Fix reset and stop states
A system with function_profile_enabled set to 1 prevents using
function_graph.

Using nop tracer left the tracing files in a dirty state.
2020-01-07 14:14:38 +00:00
Marc Bonnici
795c0f233f Development version bump 2019-12-20 16:25:03 +00:00
Marc Bonnici
5ff278b133 Version bump for release 2019-12-20 15:57:57 +00:00
Marc Bonnici
b72fb470e7 docs: Update to include Collector information 2019-12-20 15:16:32 +00:00
Marc Bonnici
a4fd57f023 devlib/__init__: Export LogcatCollector in devlib package 2019-12-20 15:16:32 +00:00
Marc Bonnici
cf8ebf6668 devlib/collector: Update Collectors to implement collector interface 2019-12-20 15:16:32 +00:00
Marc Bonnici
15a77a841d collector/screencapture: Refactor to use new collector interface
Update the interface to make use of the collector interface.
Notable changes are the removal of the `output_path` path provided on
initialisation which will now be provided by the dedicated `set_output`
method.
2019-12-20 15:16:32 +00:00
Marc Bonnici
9bf9f2dd1b collector: Update the Collector Interface
Update `get_trace` to `get_data` to better reflect the purpose.
The return type of said method will be a `CollectorOutput` object will
contain one or more `CollectorOutputEntry` objects which will be used to
provide the `path`and `path_kind` attributes to indicate the path to the
obtained output and it's type (currently a "file" or "directory")
respectively.
2019-12-20 15:16:32 +00:00
Marc Bonnici
19887de71e devlib/trace: Refactor trace to be collector
We now have have multiple `trace` instruments that do not match the
description and therefore are moved to have a more suitably named
hierarchy.
2019-12-20 15:16:32 +00:00
Marc Bonnici
baa7ad1650 devlib/AndroidTarget: Move adb specific commands into the ADB connection
The `AndroidTarget` class should not depend on ADB specific commands as
is is possible to use this target with other connection types e.g. ssh.
Therefore move the adb specific commands into the `AdbConnection`.

- `wait_for_device` and `reboot_bootloader` are now exposed in AndroidTarget
as generic methods and call through to the connection method.
- `adb_kill_server` is now a standalone function of the AdbConnection.
2019-12-20 15:15:45 +00:00
Marc Bonnici
75621022be devlib/AndroidTarget: Move ADB disconnect code into connection.
The `AndroidTarget` would ensure that when connecting to a IP target
that it disconnected first to prevent the connection getting stuck if
the connection was not closed correctly. Move this code into the
`AdbConnection` instead as this is more relevant.
2019-12-20 15:15:45 +00:00
Valentin Schneider
01dd80df34 module/sched: Fix get_capacities() on !SCHED_DEBUG kernels
While reading the DT-provided capacity values (exposed in sysfs) is
sufficient, get_capacities() also unconditionally fetches data from the
sched_domain procfs, which is only populated on kernels compiled with
CONFIG_SCHED_DEBUG.

Tweak the logic to only call get_sd_info() if it is both possible and
required.
2019-12-13 15:32:01 +00:00
Sergei Trofimov
eb0661a6b4 utils/android: update SDK versions map
Update the entry for API level 28 and add an entry for API level 29.
2019-12-06 16:25:11 +00:00
Marc Bonnici
f303d1326b exception/get_traceback: Fix type error
Passing a `BytesIO` object to `print_tb` returns a `TypeError` change
this to a `StringIO` object instead.
2019-12-06 08:20:12 +00:00
Marc Bonnici
abd88548d2 instrument/frames: Fix missing import 2019-12-06 08:20:12 +00:00
Marc Bonnici
2a934288eb instrument/daq: Fix error message 2019-12-06 08:20:12 +00:00
Douglas RAILLARD
2bf4d8a433 target: Return a bool in Target.check_responsive()
Since bool is a subclass of int, turning 0 into False and 1 into True should not
break any user code.
2019-12-05 18:26:09 +00:00
Valentin Schneider
cf26dee308 trace/ftrace: Support the 'function' tracer
This tracer is similar to the 'function_graph' tracer in that it helps us
trace function calls. It is however more lightweight, and only traces
functions entries (along with the caller of the function). It can also
happen that the kernel has support for the 'function' tracer but not for
'function_graph' (the opposite cannot be true, however).
2019-12-04 11:17:13 +00:00
Valentin Schneider
e7bd2a5b22 trace/ftrace: Memoize tracable functions
This is similar to what is already done for events and tracers. Also, use
this opportunity to use read_value() instead of target.execute('cat {}').
2019-12-04 11:17:13 +00:00
Valentin Schneider
72be3d01f8 trace/ftrace: Only require CONFIG_FUNCTION_PROFILER for the function profiling
We currently raise an exception when trying to use the 'function' or
'function_graph' tracer if the kernel wasn't compiled
CONFIG_FUNCTION_PROFILER, but that is a completely valid use.
2019-12-04 11:17:13 +00:00
Marc Bonnici
745dc9499a modules/flash: Add a connect parameter to the flash method
Adds a `connect` parameter to the flash method to specifiy whether
devlib should attempt to connect to the target after flashing has
completed.
2019-11-28 17:11:24 +00:00
Sergei Trofimov
6c9f80ff76 target: get model form platform
Move the resolution of the model name from targets into Platform's
_set_model_from_target() (which was already attempting to do that via
dmidecode method).
2019-11-28 11:07:58 +00:00
Javi Merino
182f4e7b3f daq: Fix teardown() removing temporary files
The teardown() method was introduced in bb1552151a ("instruments:
Add teardown method to clean up tempfiles") but it uses an undeclared
variable tempdir. Make tempdir an object variable so that it can be
used in teardown().
2019-11-26 16:29:04 +00:00
Javi Merino
4df2b9a4c4 daq: move to daqpower 2.0
daqpower 2.0 has a new interface and it is more stable.
2019-11-26 16:29:04 +00:00
Peter Puhov
aa64951398 Add NUMA nodes 2019-11-22 16:48:28 +00:00
Michalis Spyrou
0fa91d6c4c Add options to ssh connection
The user can pass a dictionary containg the key and value
pairs with the extra ssh configuration options. Multiple
options will be passed as '-o key1=value1 -o key2=value2'

Signed-off-by: Michalis Spyrou <michalis.spyrou@arm.com>
2019-11-21 14:19:34 +00:00
Douglas RAILLARD
0e6280ae31 ftrace: Ensure /proc/kallsyms contains symbol addresses
The content of /proc/kallsyms depends on the value of
/proc/sys/kernel/kptr_restrict:
* If 0, restriction is lifted and kallsyms contains real addresses
* If 1, kallsyms will contain null pointers

Since trace-cmd records the content of kallsyms into the trace.dat and uses that
to pretty-print function names (function tracer/grapher), ensure that its
content is available.

Signed-off-by: Douglas RAILLARD <douglas.raillard@arm.com>
2019-11-18 09:02:29 +00:00
Douglas RAILLARD
2650a534f3 exception: Fix DevlibError unpickling
Unpickling of BaseException is done by feeding self.args to the exception type.
This self.args attribute is initialized in two places: in
BaseException.__new__ (before __init__ is called) and in BaseException.__init__
as well.

The following code ends up with self.args == ('hello',), instead of (1, 2):

    class MyExcep(BaseException):
          def __init__(self, foo, bar):
              print('before super().__init__()', self.args)
              super().__init__('hello')
              print('after super().__init__()', self.args)

    MyExcep(1, 2)
    # Prints:
    # before super().__init__() (1, 2)
    # after super().__init__() ('hello',)

When unplickling such instance, ('hello',) will be fed to MyExcep.__init__(),
which will fail with a TypeError since it requires 2 positional arguments.

In order to fix that, super().__init__() needs to be handwritten instead of
getting the one from BaseException:

    class MyBase(BaseException):
        def __init__(self, msg):
            self.msg = msg

    class MyExcep(MyBase):
          def __init__(self, foo, bar):
              print('before super().__init__()', self.args)
              super().__init__('hello')
              print('after super().__init__()', self.args)

    MyExcep(1, 2)
    # Prints:
    # before super().__init__() (1, 2)
    # after super().__init__() (1, 2)

This will correctly initialize self.args == (1, 2), allowing unpickling to work.
2019-11-13 16:43:07 +00:00
Javi Merino
c212ef2146 module/cgroups: Really move all tasks in Controller.move_all_tasks_to()
The docstring of Controller.move_all_tasks_to() says that the function
moves all the tasks to the "dest" cgroup.  However, it iterates over
self._cgroups, which is a dictionary that is lazily populated when you
call Controller.cgroup().  For example, this doesn't work:

cpuset_cg = target.cgroups.controller("cpuset")
cpuset_cg.move_all_tasks_to("top-app")

Because you haven't populated self._cgroups yet.  You need to manually
populate the dictionary with something like:

for group in cpuset_cg.list_all():
    cpuset_cg.cgroup(group)

before you can use move_all_tasks_to().  Iterate through
self.list_all() instead of self._cgroups to really move all tasks to
to the destination directory.

Controller.move_tasks() has a try-except block to get the cgroups of
the source and destination groups.  Controller.cgroup() caches the
groups in self._cgroups and populates it if it hasn't been already.
Simplify move_tasks() and let it deal with source and dest cgroups
that exist but the controller hasn't loaded yet.
2019-11-05 10:28:43 +00:00
Javi Merino
5b5da7c392 module/cgroups: log to the class' logger
All classes in the module have a logger.  Avoid using the root logger
and use the class' logger.
2019-11-05 10:28:43 +00:00
Marc Bonnici
3801fe1d67 trace-cmd: Respect strict when setting saved_cmdlines_size
Not all devices have the `saved_cmdlines_size` node exposed and therefore
attempting to set this can fail. Raise an error for this only when
`strict` is set to `True` otherwise raise a warning instead.
2019-11-04 17:26:29 +00:00