The Linux scheduler exposes a set of tunables via /proc/sys/kernel's
attributes staring by "sched_".
Let's add a convenient API to the sched module to read and set the
values of these attributes.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Add system_id attribute to targets. This ID is supposed unique for a
combination of hardware, kernel, and the file system, and contains
elements from each.
1. Hardware is identified by the concatenation of MAC addresses of
'link/ether' network interfaces on the system. This method is used,
as DMI tables are often unimplemented on ARM targets.
2. The kernel is identified by its version.
3. The file system is identified by the concatenation of UUID's of the
target's partitions. It would be more correct to only use UUID of
the root partition, as system_id is not intended to be affected by
removable, media, however, there is no straight-forward way of
reliably identifying that without root.
system_id is intended to be used as an key for the purposes of caching
information about a particular device (e.g. so that it does not need to
be probed on each run).
The target platform can have different hierarchies with different cgroup
controllers mounted. If we have a cgroup that uses different controllers
belonging to different hierarchies, that cgroup will be present under the
different hierarchies. Therefore, in such cases we need to take into all those
paths having that cgroup and move the task into all of them.
When running a command into a cgroup we want to make sure that the command is
only run inside the intended cgroup. If there is a hierarchy of cgroups named
with a common prefix, the script will move it to all matching cgroup and as a
result the task will end up running always at lowest level. For example, if we
have the following hierarchy of cgroups:
"/"
|__ "/tg1"
|__ "/tg1/tg1_1"
and we want to run something in cgroup "/tg1/", the lowest levels will match the
regexp and the task will ultimately be moved to "/tg1/tg1_1".
This patch fixes the issue by requiring the absolute path of the specified
cgroup to match exactly.
The previous fix for read_tree_values fixed the issue with sh not
supporting arrays, but looping over paths and counting them. Hover each
count increment requires spawning a subshell. For a large number of paths,
this can eat away any performance benefits of using read_tree_values.
Since we only care whether the count is greater than one, detect that
and break out of the loop early to re-introduce the performance
improvement.
My bash fu was off on my previous list. The output of find was being
treated as a single string, rather than an array of paths; which ment
that the test that there was more than one path returned failed,
resulting in null output not just for empty directories but for
everyting.
This fixes the issue by converting find output to an array.
grep was existing with 1 when passed an empty directory, resulting in
TargetError being raised unless explicitly suppressed with
check_exit_code=False. This case is now fixed to correctly return an
empty dict without error.
read_tree_values bash function has also been optimized to not
unnecessarily run find in a subshell if the path passed to the function
does not exist.
Add two new methods to target that allow querying values of all sysfs
nodes in a sub-directory structure all at once. The difference is that
read_tree_values_flat returns a flat dict of path->value mappings;
read_tree_values returns a nested dict structure that mimics the
scanned sub-directories tree.
The set() method of the CGroup class used to freeze tasks relies on
target's write_value(). Sometimes, the freezing procedure takes some
time and the call to write_value() in set() fails by reading "FREEZING"
while it expected "FROZEN". To avoid this issue, this commits introduces
a shutil call dedicated to changing the state of the freezer controller.
In the previous patch:
cgroups: Mount cgroups controllers in devlib working dir
we changed the default mount point for devlib managed CGroups but forgot to
update the support for execution of a workload within a specified CGroup.
The run_into support is provide by a shutil script, which still has hardcoded
the old path (i.e. /sys/fs/cgroup/devlib_*). This patch fixes this by:
- resetting the default path to the Linux standard /sys/fs/cgroup
- use-sing the existing CGMOUNT env variable to specify which CGroups mount
point to use
The cgroups::run_into is also updated to use the self.cgroup_root via
CGMOUNT when the shutils' script is called.
Moreover, an additional cgroups::run_into_cmd method is added which just
returns a properly formatted run_into shutils' call.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
If a process is added to $PIDS but then exits before the
cgroups_task_move invokation finishes, then echoing its PID to
cgroups.procs results in an I/O error. If that process is the last in
$PIDS, then that failing echo command is the last of the function, so
the script exits with an error and devlib raises an exception.
Add `|| true` to avoid this problem.
On some systems CPUs sometimes remain idle for several seconds. If a
trace capture begins during one of these long idle periods, that CPU's
idle state is unknown (in practice it is probably in its deepest
available state from cpuidle's perspective but that can't be known for
sure).
The solution to the equivalent problem for cpufreq is to read the
current frequencies from sysfs and inject artificial cpu_frequency
events using trace_marker (see cpu_freq_trace_all_frequencies in
bin/scripts/shutils.in). However we can't read the current idle state
from sysfs.
Instead, wake up each CPU by executing the "true" command on it via
taskset.
The injection of a fake cpu_frequency event in the trace may end up getting
interleaved with trace events generated by the cpufreq governor, for example:
sh ... cpu_frequency: state=120000 cpu_id=0
kschedfreq ... cpu_frequency: state=120000 cpu_id=0
kschedfreq ... cpu_frequency: state=120000 cpu_id=1
kschedfreq ... cpu_frequency: state=120000 cpu_id=2
kschedfreq ... cpu_frequency: state=120000 cpu_id=3
sh ... cpu_frequency: state=120000 cpu_id=1
sh ... cpu_frequency: state=120000 cpu_id=2
sh ... cpu_frequency: state=120000 cpu_id=3
Such a trace may confuse trace parsers during postprocessing and lead to wrong
or no results.
This patch renames the events injected by devlib so that they can be easily
recognized and then used by the trace parser in the best way.
Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com>
On some targets the run_into function is not passing all params to the
called application. This patch should fix that using shift to get a list
of all command parameters.
Moreovere, we use now exec to spawn the command execution, which avoids to
generate yet another shell to run the required command.
CGroups controller can be mounted by specifying a "noprefix" option,
in which case attribute names are named as:
<mountpoint>/<attribute_name>
instead of the (more recent) naming schema using:
<mountpoint>/<contoller_name>.<attribute_name>
For example, Android uses the old format for backward compatibility
with user-space. Thus, it's possible in general to work on a target
system where some controller are mounted "noprefix" while others not.
This patchset adds a set of updates which allows to use the proper
attributes naming schema based on how the controller has been mounted.
This patch provides a more generic implementation of the get attributes
shutils which is working also for noprefix mounted controllers.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch provides a couple of utility functions which makes it
easy to run a command under a specific CGroup or to move all tasks
from a group to another.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The current code used to read the attributes values for a controller uses
a "grep '' CONTROLLER.*" under the assumption that the output is a list of
file:value
However, if there is a single controller attribute, grep does not report
the file name in output thus returning an empty list at the python side.
This patch fix that issue by also switching to the usage of a shutil
implementation of the attributes parsing code.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Functions profiling data are reported in a set of files, one for each CPU.
This patch adds the required support to collect these data into a single
JSON formatted file. Data are collected using a shutils routing on the
target side and formatted in JSON on the host side.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch adds a couple of shutils functions to efficiently read all
the frequencies/governors and returne them into a dictionary indexed by
CPU id.
The shutils functions are returning a line per each CPU that can be easily
converted into a dictionary in the host side.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch convert some functions to the usage of shutils provided calls.
This approach is more portable and allows to use these calls also on
an Android target.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Some complex operations are faster if executed on the target side with the
help of a small script. For example, reading the current frequency and
or governor for all the online CPUs.
This patch adds a support script to be deployed on the target as well as
an internal utility function which allows to call functions provided by
that support script.
The support script has to be cross platform, thus:
1. only a limited set of shell functions can be used, which must be supported
both in BASH and the Android shell
2. some paths needs to be defined depending on the specific target
To address the last constrain, this patch provides a "template" script which
contains some tokens "__DEVLIB_<TOKEN>__" that are replaced right before to
push the script on the target.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
