Move from pipes.quote (private) to shlex.quote (Py3.3+ standard).
Make tests of inputs against None (their default value) instead of based
on their truthiness.
Improve logging through quoted commands (runnable as-is, less confusing).
Make the command-building process straightforward for readability and
maintainability.
To aid in checking whether any information is contained in the
`KernelConfig` ensure that that `__bool__` method value indicated the
presence of parsed input.
Right now, this module won't be loaded if the sched_domain procfs
entries are not present on the target. However, other pieces of
information may be present in which case it would make sense to load
the module.
For instance, mainline kernels compiled without SCHED_DEBUG can still
expose the cpu_capacity sysfs entry. As such, try to get a better idea
of what's available and only disable the loading of the module if it
can provide absolutely nothing.
As mentioned in the previous commit, CPU numbers would be passed to
SchedProcFSData's __init__() (instead of a proper sysfs path). When
done with CPU0, that path would be evaluated as False and the code
would carry on with the default path, which was quite confusing.
This has now been fixed (and 0 isn't such a great path to give
anyway), nevertheless this check should just catter to None.
Rather than using the conveniently provided `get_cpu_sd_info()` helper
method, `has_em()` and `get_em_capacity()` would build a
`SchedProcFSData` with `path=<CPU number>`, which is obviously broken.
Do the right thing and use `get_cpu_sd_info()` in those places.
The existing behaviour assumes that the cap_states file contains a list
of capacity|cost pairs, and attempts to return the maximum capacity by
selecting the value at the second last index of the list.
This assumption fails on some newer Qualcomm kernels where the
cap_states file contains a list of capacity|frequency|cost triplets.
Consequently, the maximum frequency would be erroneously returned
instead of the maximum capacity.
Fix the problem by dynamically calculating the index of the maximum
capacity by dividing the number of entries in cap_states by the value in
nr_cap_states.
---
For example, on a certain Snapdragon 845 device:
/proc/sys/kernel/sched_domain/cpu0/domain0/group0/energy/cap_states
54 entries:
CAP FREQ COST
--------------------
65 300000 12
87 403200 17
104 480000 21
125 576000 27
141 652800 31
162 748800 37
179 825600 42
195 902400 47
212 979200 52
228 1056000 57
245 1132800 62
266 1228800 70
286 1324800 78
307 1420800 89
328 1516800 103
348 1612800 122
365 1689600 141
381 1766400 160
/proc/sys/kernel/sched_domain/cpu0/domain0/group0/energy/nr_cap_states
18
Max capacity = 381 (third-last index)
In the previous version there appears to be a bug causing perf to
segfault as per https://github.com/ARM-software/devlib/issues/395.
Therefore update provided binaries to v3.19 which does not appear to
have this issue.
Apart from busybox, devlib itself makes use of other system provided binaries.
For example, the DmesgCollector module uses the system provided dmesg.
In cases the system provided binary does not support some of the features
required by devlib, we currently just fails with an error.
For the user it is still possible to deploy a custom/updated version of a
required binary via the Target::install API. However, that binary is not
automatically considered by devlib.
Let's ensure that all Target::execute commands use a PATH which gives priority
to devlib installed binaries.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Replace the default SIGKILL signal sent to perf to "request" its
termination by a SIGINT, allowing it to handle the signal by cleaning up
before exit. This should address issues regarding corrupted perf.data
output files.
Old util-linux binaries don't support --force-prefix. Multi-line entry
parsing will break on these, but at least we can collect the log.
Also decode the raw priority, so only the facility is not decoded in
case busybox or old util-linux is used.
Back when I first wrote this I tried to make something smart that
would automatically detect which procfs entries to pack into a
mapping, the condition to do so being "the entry ends with a
digit and there is another entry with the same name but a different
digit".
I wrongly assumed this would always work for the sched_domain entries,
but it's possible to have a domain with a single group and thus a
single "group0" entry.
Since we know which entries we want to pack, let's hard-code these and
be less smart about it.
There are two ways we can load CPU capacity. Up until 4.14, supported
kernels did not have the /sys/devices/system/cpu/cpuX/cpu_capacity file
and the only way to read cpu capacity was by grepping the EM from
procfs sched_domain output. After 4.14, that route still exists but is
complicated due to a change in the format once support for
frequency-power models was merged.
In order to avoid rewriting the procfs EM grepping code, lets switch the
order of algorithms we try to use when loading CPU capacity. All newer
kernels provide the dedicated sysfs node and all kernels which do not
have this node use the old format for the EM in sched_domain output.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
When using Python 2.7 `int`s have a maximum size which can be exceeded
when attempting to convert the hex representation back. Change `HexInt` to
be a `long` instead to avoid this issue.
Command line echoing was disabled, but that disabling did not take
effect. Another part of devlib was still expecting command lines to be
echoed. That is fixed by disabling echoing when creating pxssh
connection, and removing the code that expected the line to be echoed.
pipes.quote() doesn't like integers:
>>> from pipes import quote
>>> quote(42)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/shlex.py", line 282, in quote
if _find_unsafe(s) is None:
TypeError: expected string or bytes-like object
Convert buffer_size to str when quoting it
The hwmon module reads sysfs entries from the target during init. Since
this step is known to be fragile on some hosts, make sure to specify
tar=True when calling target.read_tree_values() to improve the chances
of reading the data properly.
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Since target.read_tree_values() has been modified to use tar as a way to
fetch the content of files from the target, it is more robust, but also
much slower. Since that level of robustness is in practice required only
for very specific use-cased, re-introduce the old way of doing read_tree
using find and grep.
read_tree_values() gains a new parameter to specify how files should be
read from the target, with or without tar. It defaults to the old way
of doing things.
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Maps Kconfig types to appropriate Python types, and act as a regular
mapping with extended API:
* tristate and bool values mapped to an Enum
* int values to int
* hex values to HexInt subclass of int that defaults to parsing and
printing in hex format.
Implement KernelConfig as a shim on top of TypedKernelConfig so they
share most of the code. Code needing a TypedKernelConfig from a
KernelConfig producer such as Target.config can trivially access the
`typed_config` attribute of KernelConfig objects.
Make a new exception type raised by KernelConfig that inherits from both
KeyError (exception raised by mappings) and IndexError (exception raised
by sequences, but also raised here for backward compatibility).
Use the canonical option name in the underlying mapping, so looking up
the canonicalized option name will work as expected.
Otherwise, looking up the key CONFIG_FONT_8x8 would not work, since the
internal representation uses 'CONFIG_FONT_8x8' and the user input is
turned into 'CONFIG_FONT_8X8' (note the capital "X").
target.read_tree_values() has several weaknesses. It doesn't support
files with ':' in their name, and it fails when reading binary files.
In essence, these limitations are cause by its fragile implementation
based on grep in shutils.
In order to robustify read_tree_values(), use tar and base64 to send the
content of a tree to the host, which can then process it from there. In
the process, read_tree_values() gains two new arguments:
- decode_unicode: must be set to work text/utf-8 content;
- strip_null_chars: must be set to remove '\00' chars from text files.
Both are set to true by default to keep backward compatibility with the
existing code.
Suggested-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
As detailed in https://github.com/pexpect/pexpect/issues/552 when
sending a command with exactly the length of the prompt less than the
window width, an extra prompt is introduced into the returned data
causing the matching to fail and only return part of the commands
output. To workaround this check the length of the command to be
submitted and if of the specific length add an additional whitespace
character to the end of the command to prevent this behaviour.
Scheduler features are a debbugging mechanism which allows to tune at
run-time some (usually experimental) features of the Linux scheduler.
Let's add a proper API abstraction to easily access the list of
supported features and tune them.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
