The docstring of Controller.move_all_tasks_to() says that the function
moves all the tasks to the "dest" cgroup. However, it iterates over
self._cgroups, which is a dictionary that is lazily populated when you
call Controller.cgroup(). For example, this doesn't work:
cpuset_cg = target.cgroups.controller("cpuset")
cpuset_cg.move_all_tasks_to("top-app")
Because you haven't populated self._cgroups yet. You need to manually
populate the dictionary with something like:
for group in cpuset_cg.list_all():
cpuset_cg.cgroup(group)
before you can use move_all_tasks_to(). Iterate through
self.list_all() instead of self._cgroups to really move all tasks to
to the destination directory.
Controller.move_tasks() has a try-except block to get the cgroups of
the source and destination groups. Controller.cgroup() caches the
groups in self._cgroups and populates it if it hasn't been already.
Simplify move_tasks() and let it deal with source and dest cgroups
that exist but the controller hasn't loaded yet.
Exceptions such as TargetError can sometimes be raised because of a
network issue, which is useful to distinguish from errors caused by a
missing feature for automated testing environments.
The following exceptions are introduced:
* DevlibStableError: raised when a non-transient error is encountered
* TargetStableError
* DevlibTransientError: raised when a transient error is encountered,
including timeouts.
* TargetTransientError
When there is an ambiguity on the type of exception to use, it can be
assumed that the configuration is correct, and therefore it is a
transient error, unless the function is specifically designed to probe a
property of the system. In that case, ambiguity is allowed to be lifted
by assuming a non-transient error, since we expect it to raise an
exception when that property is not met. Such ambiguous case can appear
when checking Android has booted, since we cannot know if this is a
timeout/connection issue, or an actual issue with the Android build or
configuration. Another case are the execute() methods, which can be
expected to fail on purpose. A new parameter will_succeed=False is
added, to automatically turn non transient errors into transient ones if
the caller is 100% sure that the command cannot fail unless there is an
environment issue that is outside of the scope controlled by the user.
devlib now never raises TargetError directly, but one of
TargetStableError or TargetTransientError. External code can therefore
rely on all (indirect) instances TargetError to be in either category.
Most existing uses of TargetError are replaced by TargetStableError.
In:
commit 454b9450 ("pylint fixes")
we added:
```diff
@@ -363,7 +368,7 @@ class CgroupsModule(Module):
# Get the list of the available controllers
subsys = self.list_subsystems()
- if len(subsys) == 0:
+ if subsys:
self.logger.warning('No CGroups controller available')
return
```
which changed the invariant condition to enabled the cgroup module:
the module is enabled we we can find a "non empty" list of subsystems.
Let's fix this to bail out on an empyt list.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The shutils run_into support assumes that we always specify a full
cgroup path starting by "/". If, by error, we specify a cgroup name
without the leading "/" we get an ambiguous message about the cgroup not
being found.
Since this already happened to me many times, let's add an explicit
check about the cgroup name parameter to better info the user about the
requirement.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The tasks() function allows to get the tasks that are in a cgroup.
Filters for the tasks TID, name and cmdline have been added to the
parameters of the function such that it is possible to select the tasks
that match these patterns.
Signed-off-by: Elieva Pignat <Elieva.Pignat@arm.com>
The set() method of the CGroup class used to freeze tasks relies on
target's write_value(). Sometimes, the freezing procedure takes some
time and the call to write_value() in set() fails by reading "FREEZING"
while it expected "FROZEN". To avoid this issue, this commits introduces
a shutil call dedicated to changing the state of the freezer controller.
The current run_into method attempts to execute:
<...>/shutils CGMOUNT=<...> cgroups_run_into <...>
Which should instead be:
CGMOUNT=<...> <...>/shutils cgroups_run_into <...>
So just use cgroups_run_into_cmd to generate the command, then execute that.
Some systems mount multiple CGroup controllers in the same hierarchy, this
happens in desktop systems using systemd but it's also the default for
systems using CGroups v2. Unfortunately the current CGroup modules can
mount only single-controller hierarchies, thus failing when a controller
is already mounted with others in a single hierarchy.
This patch fixes this issue by:
- keeping track of which controllers are mounted in each hierarchy
- muting hierarchies instead of controllers
- creating Controller objects which reference the hierarchy they belong to
Thus, the new constructor of the Controller object requires to specify
the hierarchy ID as well as the list of controller which have to be
mounted in that hierarchy. The hierarchy ID is used to create a single
mount point for all the controllers in that hierarchy. This single mount
points are now named:
<CGMOUNT>/devlib_cgh<ID>
where "cgh" stands for "CGroup Hierarchy" and <ID> is the specified
numerical identified of that hierarchy.
This patch removes also the Controller::__new__() method, which seems not to
have sense anymore, as well as the Controller::probe() method, which is
not more used. Indeed, all and only the available hierarchies are pre-mounted
at module initialization time.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This is just a simple re-factoring patch in preparation of the following
one. Since we have a list of CGroups subsystems, which includes the
hierarchy ID for each entry, let's use directly these namedtuples in the
mouting loop. The following patch will make use of the hierarchy ID to
properly mount each controller.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
In the previous patch:
cgroups: Mount cgroups controllers in devlib working dir
we changed the default mount point for devlib managed CGroups but forgot to
update the support for execution of a workload within a specified CGroup.
The run_into support is provide by a shutil script, which still has hardcoded
the old path (i.e. /sys/fs/cgroup/devlib_*). This patch fixes this by:
- resetting the default path to the Linux standard /sys/fs/cgroup
- use-sing the existing CGMOUNT env variable to specify which CGroups mount
point to use
The cgroups::run_into is also updated to use the self.cgroup_root via
CGMOUNT when the shutils' script is called.
Moreover, an additional cgroups::run_into_cmd method is added which just
returns a properly formatted run_into shutils' call.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
These quotes end up being passed literally into the shutils function
arguments (i.e $3 is '-e', $4 is '"foo"') which means that the grep
command never finds matches. `exclude` is a list of comms, which don't
have spaces in them, so we can just remove the quotes.
Android seems to have a buggy `mount` command which causes
`mount -o remount` to result in duplicated mounts. We can't unmonunt and
then re-mount /sys/fs/cgroup because there may be pre-existing mounts at
subdirectories. So we create a 'cgroups' directory in the devlib working
directory and mount cgroup controller FS's there.
Without this patch, this will result in an error due to trying to call
`.cgroup` on None. Making this a RuntimeError instaed means that a) you
get a more useful error message and b) you can catch the exception
without blanket `except Exception as e`.
Some systems mount CGroups RO, thus we need to ensure we remount that
FS as root otherwise we cannot create new groups.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The cgroups module requires busybox and shutil to properly initialise.
This patch required the module to be initialized once the setup has
completed.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
In case a target does not report a configuration file, we can still check
the user-space API to verify it CGroups are supported.
NOTE: a rooted target is still a mandatory requirement because some commands
are still dependant on the possibility to run them with root permissions.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Certain commands requires in general root permissions to be properly
executed (for example on an Android target).
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
CGroups controller can be mounted by specifying a "noprefix" option,
in which case attribute names are named as:
<mountpoint>/<attribute_name>
instead of the (more recent) naming schema using:
<mountpoint>/<contoller_name>.<attribute_name>
For example, Android uses the old format for backward compatibility
with user-space. Thus, it's possible in general to work on a target
system where some controller are mounted "noprefix" while others not.
This patchset adds a set of updates which allows to use the proper
attributes naming schema based on how the controller has been mounted.
This patch makes use of the Controller::_noprefix option to properly
build the attribute path. It adds also a check which reports a more
clear error in case an attribute is set which is not provided by the
controller.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
CGroups controller can be mounted by specifying a "noprefix" option,
in which case attribute names are named as:
<mountpoint>/<attribute_name>
instead of the (more recent) naming schema using:
<mountpoint>/<contoller_name>.<attribute_name>
For example, Android uses the old format for backward compatibility
with user-space. Thus, it's possible in general to work on a target
system where some controller are mounted "noprefix" while others not.
This patchset adds a set of updates which allows to use the proper
attributes naming schema based on how the controller has been mounted.
This first patch keeps track of whatever a controller has been mounted
using the noprefix option.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch provides a couple of utility functions which makes it
easy to run a command under a specific CGroup or to move all tasks
from a group to another.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The info statements are clobbering the "normal" output of devlib users.
This patch demote the logging to debug level. The user can still
log-report these information from the corresponding functions call site.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
The current code used to read the attributes values for a controller uses
a "grep '' CONTROLLER.*" under the assumption that the output is a list of
file:value
However, if there is a single controller attribute, grep does not report
the file name in output thus returning an empty list at the python side.
This patch fix that issue by also switching to the usage of a shutil
implementation of the attributes parsing code.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This patch refactors the CGroup module to support multiple controllers.
The new interface is based on two main classes:
.:: Controller(kind)
Provides a wrapper class for a CGgroup controller of "kind".
At module initialization time, all the controller available in the
system are enumerated and a corresponding controller class created.
A valid reference to the Controller class for a specific CGroup
controller can be obtained using the controller(kind) method exposed
by the module, e.g. to get a reference to the CPUSET controller:
cpuset = target.cgroups.controller('cpuset')
.:: CGroup(name)
Provides a wrapper class for a CGroup of "name", i.e. a folder which
path is "name" relative to the controller mount point.
At module initialization time, the root control group for each
controller is created by the controller class.
A valid reference to a CGroup for a specific controller can be
obtained using the cgroup(name) method exposed by the controller
object, e.g. to get a reference to the "/LITTLE" cgroup of the
cpuset controller:
cpuset_littles = cpuset.cgroup('/LITTLE')
The CGroup object exposes simple get()/set() methods which allows to
read and configure all the supported attributes of a controller.
For example, to set the "cpus" of a cpuset control group:
cpuset_littles.set(cpus=[0,1])
NOTE: the set() method accepts a variable list of parameters which name
must match a valid (writable) attribute for that controller.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
