Gem5's statistics log file contains plenty of interesting information
that are not exposed so far. This module enables control and parsing of
the statistics file by:
- configuring periodic dumps of statistics;
- marking Regions of Interest (ROIs);
- and extracting values of specific fields during the ROIs.
I have observerd the error message "write error: Invalid argument"
when using set_all_governors to set a non-existent governor on a
buildroot Linux system
It's common to want to do an operation on cpufreq that affects all
CPUs, but doing so explicitly for all CPUs can be unnecessarily slow
when it only needs to be done once for each cpufreq policy.
Add a method to abstract the pattern of iteration.
If we get an TargetError when trying to set a governor tunable we currently fall
back to an older sysfs layout under the assumption that the file we tried to
write doesn't exist. However it may be that the file exists, but we tried to
write an invalid value (or something else went wrong). In this case we fall back
to the old file location, fail, and produce a nonsense error about the old path
not existing.
Instead, when we get a TargetError, check if the file exists, and fall back to
the old location only if it does not.
Fixes issue #98 where the passed `cpus` parameter was ignored when setting the
governor and frequency and set for all cpus instead of those specified in the
passed parameter.
Also fixes kwargs not being correctly passed for governor tunables.
The current run_into method attempts to execute:
<...>/shutils CGMOUNT=<...> cgroups_run_into <...>
Which should instead be:
CGMOUNT=<...> <...>/shutils cgroups_run_into <...>
So just use cgroups_run_into_cmd to generate the command, then execute that.
Unit info from comments on `struct cpuidle_state` definition in
include/linux/cpuidle.h in Linux 4.9 source (see drivers/cpuidle/sysfs.c
for the code that exposes that data to userspace)
Some systems mount multiple CGroup controllers in the same hierarchy, this
happens in desktop systems using systemd but it's also the default for
systems using CGroups v2. Unfortunately the current CGroup modules can
mount only single-controller hierarchies, thus failing when a controller
is already mounted with others in a single hierarchy.
This patch fixes this issue by:
- keeping track of which controllers are mounted in each hierarchy
- muting hierarchies instead of controllers
- creating Controller objects which reference the hierarchy they belong to
Thus, the new constructor of the Controller object requires to specify
the hierarchy ID as well as the list of controller which have to be
mounted in that hierarchy. The hierarchy ID is used to create a single
mount point for all the controllers in that hierarchy. This single mount
points are now named:
<CGMOUNT>/devlib_cgh<ID>
where "cgh" stands for "CGroup Hierarchy" and <ID> is the specified
numerical identified of that hierarchy.
This patch removes also the Controller::__new__() method, which seems not to
have sense anymore, as well as the Controller::probe() method, which is
not more used. Indeed, all and only the available hierarchies are pre-mounted
at module initialization time.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This is just a simple re-factoring patch in preparation of the following
one. Since we have a list of CGroups subsystems, which includes the
hierarchy ID for each entry, let's use directly these namedtuples in the
mouting loop. The following patch will make use of the hierarchy ID to
properly mount each controller.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
In the previous patch:
cgroups: Mount cgroups controllers in devlib working dir
we changed the default mount point for devlib managed CGroups but forgot to
update the support for execution of a workload within a specified CGroup.
The run_into support is provide by a shutil script, which still has hardcoded
the old path (i.e. /sys/fs/cgroup/devlib_*). This patch fixes this by:
- resetting the default path to the Linux standard /sys/fs/cgroup
- use-sing the existing CGMOUNT env variable to specify which CGroups mount
point to use
The cgroups::run_into is also updated to use the self.cgroup_root via
CGMOUNT when the shutils' script is called.
Moreover, an additional cgroups::run_into_cmd method is added which just
returns a properly formatted run_into shutils' call.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
These quotes end up being passed literally into the shutils function
arguments (i.e $3 is '-e', $4 is '"foo"') which means that the grep
command never finds matches. `exclude` is a list of comms, which don't
have spaces in them, so we can just remove the quotes.
Android seems to have a buggy `mount` command which causes
`mount -o remount` to result in duplicated mounts. We can't unmonunt and
then re-mount /sys/fs/cgroup because there may be pre-existing mounts at
subdirectories. So we create a 'cgroups' directory in the devlib working
directory and mount cgroup controller FS's there.
Without this patch, this will result in an error due to trying to call
`.cgroup` on None. Making this a RuntimeError instaed means that a) you
get a more useful error message and b) you can catch the exception
without blanket `except Exception as e`.
On some systems CPUs sometimes remain idle for several seconds. If a
trace capture begins during one of these long idle periods, that CPU's
idle state is unknown (in practice it is probably in its deepest
available state from cpuidle's perspective but that can't be known for
sure).
The solution to the equivalent problem for cpufreq is to read the
current frequencies from sysfs and inject artificial cpu_frequency
events using trace_marker (see cpu_freq_trace_all_frequencies in
bin/scripts/shutils.in). However we can't read the current idle state
from sysfs.
Instead, wake up each CPU by executing the "true" command on it via
taskset.
Some systems mount CGroups RO, thus we need to ensure we remount that
FS as root otherwise we cannot create new groups.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
When this information is not needed, this avoids executing 4 commands on
the host for each CPU, which significantly speeds up initialising the
cpuidle module.
Currently when using set_all_governors to try to set an unsupported
governor, the error is a somewhat cryptic "sh: echo: I/O error". Add a
check that diagnoses this error.
In order to avoid slowing down the "good" path, the check for
unsupported governors is only done if the failure has already occured.
When passing an idle state ID to get_states(), nothing is returned. Also, there
is a wrong call to enable() in the method.
Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com>
The cgroups module requires busybox and shutil to properly initialise.
This patch required the module to be initialized once the setup has
completed.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
In case a target does not report a configuration file, we can still check
the user-space API to verify it CGroups are supported.
NOTE: a rooted target is still a mandatory requirement because some commands
are still dependant on the possibility to run them with root permissions.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Certain commands requires in general root permissions to be properly
executed (for example on an Android target).
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
CGroups controller can be mounted by specifying a "noprefix" option,
in which case attribute names are named as:
<mountpoint>/<attribute_name>
instead of the (more recent) naming schema using:
<mountpoint>/<contoller_name>.<attribute_name>
For example, Android uses the old format for backward compatibility
with user-space. Thus, it's possible in general to work on a target
system where some controller are mounted "noprefix" while others not.
This patchset adds a set of updates which allows to use the proper
attributes naming schema based on how the controller has been mounted.
This patch makes use of the Controller::_noprefix option to properly
build the attribute path. It adds also a check which reports a more
clear error in case an attribute is set which is not provided by the
controller.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
CGroups controller can be mounted by specifying a "noprefix" option,
in which case attribute names are named as:
<mountpoint>/<attribute_name>
instead of the (more recent) naming schema using:
<mountpoint>/<contoller_name>.<attribute_name>
For example, Android uses the old format for backward compatibility
with user-space. Thus, it's possible in general to work on a target
system where some controller are mounted "noprefix" while others not.
This patchset adds a set of updates which allows to use the proper
attributes naming schema based on how the controller has been mounted.
This first patch keeps track of whatever a controller has been mounted
using the noprefix option.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch provides a couple of utility functions which makes it
easy to run a command under a specific CGroup or to move all tasks
from a group to another.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The info statements are clobbering the "normal" output of devlib users.
This patch demote the logging to debug level. The user can still
log-report these information from the corresponding functions call site.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The current code used to read the attributes values for a controller uses
a "grep '' CONTROLLER.*" under the assumption that the output is a list of
file:value
However, if there is a single controller attribute, grep does not report
the file name in output thus returning an empty list at the python side.
This patch fix that issue by also switching to the usage of a shutil
implementation of the attributes parsing code.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch adds a couple of shutils functions to efficiently read all
the frequencies/governors and returne them into a dictionary indexed by
CPU id.
The shutils functions are returning a line per each CPU that can be easily
converted into a dictionary in the host side.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch convert some functions to the usage of shutils provided calls.
This approach is more portable and allows to use these calls also on
an Android target.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
