1
0
mirror of https://github.com/ARM-software/devlib.git synced 2024-10-06 02:40:50 +01:00
Commit Graph

685 Commits

Author SHA1 Message Date
Douglas RAILLARD
fbf0875357 serial_port: Handle exception in open_serial_connection
Use try/finally clause in the contextmanager to always close the
connection if an exception is raised.

Also remove "del conn" since it only decrements the reference count,
which is done anyway when the generator function returns.
2018-11-28 10:56:52 +00:00
Quentin Perret
b7ab340d33 instrument: Fix active_channels for Python 3
Since we can iterate over the active_channel attribute of Instrument
more than once, make sure to cast the return value of filter() to a list
to avoid issues with python 3.

Signed-off-by: Quentin Perret <quentin.perret@arm.com>
2018-11-27 16:43:28 +00:00
Juri Lelli
beb824256d bin: Add ppc64le binaries
Add busybox and trace-cmd binaries for ppc64le so that devlib can be
used on POWER machines.

Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
2018-11-23 15:08:51 +00:00
Marc Bonnici
efbf630422 utils/ssh: Remove original traceback from exception
In Python 3 when re-raising an exception the original traceback will
also be included. In the `_scp` method we do not want to expose the
password so hide the original exception when re-raising.
2018-11-21 15:11:37 +00:00
Douglas RAILLARD
389ec76c1e escaping: Use pipes.quote instead of escape_*
pipes.quote (also known as shlex.quote in Python3) provides a robust
implementation of quoting and escaping strings before passing them to
POSIX-style shells. Use it instead of the escape_* functions to simplify
quoting inside devlib.
2018-11-21 15:07:14 +00:00
Douglas RAILLARD
1f50b0ffc2 quoting: Use shlex.split instead of str.split
Split command lines using str.split ignores the quoting. Fixes that by
using shlex.split.
2018-11-21 15:07:14 +00:00
Douglas RAILLARD
ed7f0e56a2 subprocess: Fix quoting issues
Strings are passed to subprocess.Popen is used instead of sequences, so
proper quoting of parameters in command lines is needed to avoid
failures on filenames containing e.g. parenthesis. Use pipes.quote to
quote a string to avoid any issue with special characters.
2018-11-21 15:07:14 +00:00
Valentin Schneider
d376bc10ee module/sched: SchedDomain: Turn flags definition into an enum
This lets us easily iterate over the known flags, and
makes the code a little neater.
2018-11-21 10:54:46 +00:00
Valentin Schneider
60c2e7721e setup.py: Add enum34 dependency for Python < 3.4 2018-11-21 10:54:46 +00:00
Volker Eckert
5e13a045a3 utils/ssh.py: try to make failure to parse response more obvious 2018-11-19 13:13:35 +00:00
syltaylor
c4c76ebcf8 devlib/target: change get_rotation (Android)
Command 'settings get system user_rotation' does not report current screen orientation value when auto-rotate is enabled or device does not support it (returning null).

Replaced command used by get_rotation to 'dumpsys input'. It should now return current screen orientation value (0-3).
2018-11-15 16:09:01 +00:00
Sergei Trofimov
bdaea26f6f utils/misc: memoized: fix kwarg IDs
Use __get_memo_id() for keyword arguments, rather than stringifying
them.  The has somewhat lower chance of producing the same identifier
for two conceptually unequal objects.
2018-11-14 13:52:12 +00:00
Sergei Trofimov
a3c04fc140 utils/misc: document memoized limitation
Document the issue with using memoized with mutable types.
2018-11-14 13:49:34 +00:00
Valentin Schneider
94c1339efd module/cpufreq: Add userspace special case for use_governor()
Frequencies are not considered as governor tunables by
list_governor_tunables(), so add a special case to restore userspace
frequency when using use_governor().

While at it, reword the internal "governor" variable to not clash
with the parameter of the same name.
2018-11-13 13:44:00 +00:00
Sergei Trofimov
85e0fb08fe target: add page_size_kb property
Add a property to get the target's virtual memory page size in kB.
2018-11-02 11:46:07 +00:00
Valentin Schneider
74444210e7 module/sched: Harden probe()
Running some tests on a VM I hit that unexpected scenario where the
required sched_domain file is there but then read_tree_values() fails
because there are no files to read, only directories.

This does a quick check that there's actual data to be read.
2018-11-02 10:40:25 +00:00
Douglas RAILLARD
da3afeba2e target: Remove failed modules from target.modules
When a module is not supported by the target, remove it from the
"modules" list attribute, so code can check if a module was successfully
loaded by just looking at that list after Target.setup() is done.
2018-10-31 17:43:41 +00:00
Marc Bonnici
4a4739cefb doc: Fix formatting 2018-10-31 10:17:21 +00:00
Marc Bonnici
01c39cfe4c doc/conns: Update documentation to include strip_colors parameter 2018-10-31 10:17:21 +00:00
Quentin Perret
b9b38a20f6 target: Ensure consistency between target.execute() and conn.execute()
Commit 511d478164 ("exceptions: Classify transient exceptions")
introduced a "will_succeed" argument in target.execute(). This argument
is then passed down to conn.execute(). However, since it is not
labelled, this argument happens to overwrite the "strip_colors" argument
of conn.execute because of its position in the list of parameters.

Fix this by labelling the parameters given to conn.execute(). While at
it, introduce a "strip_colors" parameters for target.execute() hence
keeping the APIs consistent.

Signed-off-by: Quentin Perret <quentin.perret@arm.com>
2018-10-30 15:36:31 +00:00
Valentin Schneider
809d987f84 AndroidTarget: Change screen resolution acquisition method
The current regexp doesn't seem to work anymore on a more recent
Android version (current master which reports 'Q').

Looking at other means of getting the screen resolution, this one: [1]
seems to have been there for some time already (2014), and works with
the current version.

[1]: https://stackoverflow.com/a/26185910/5096023
2018-10-22 10:23:39 +01:00
Valentin Schneider
bf1310c278 module/systrace: Handle empty lines for category listing 2018-10-22 10:23:39 +01:00
Valentin Schneider
78de479a43 module/systrace: Fix subprocess interactions for Python 3
Using 'universal_newlines' gives use pure strings instead of byte
strings
2018-10-22 10:23:39 +01:00
Valentin Schneider
75332cf14a module/systrace: Fix platform_tools use
platform_tools is only updated after the very first connection to the
target, and it will be None until then.

As I understand it, using `from devlib.utils.android import
platform_tools` will create a symbol local to the imported systrace
module for `platform_tools` that just takes its current value,
and it won't be updated when it should be.

Changing the `from x import y` statement to a simple `import x` seems
to counteract this.
2018-10-22 10:23:39 +01:00
Valentin Schneider
6089eaf40a module/sched: Fix sched procfs parsing for >= 10 CPU systems
The regexp was a bit too greedy and would match 'cpu1' as the
name of a 'cpu10' node.
2018-10-17 15:40:56 +01:00
Marc Bonnici
fa41bb01d2 modules: Update docs with 'setup' stage for module initialization
Update documentation to include the 'setup' stage for module
initialization.
2018-10-10 10:31:45 +01:00
Marc Bonnici
8654a6dc2b devlib: Add development tag to version number
To allows better versioning control between releases add `dev1` to the
current version. This allows other under development tools such as
Workload Automation to ensure that a sufficiently up to date version of
devlib is installed on the system.
2018-09-21 13:04:27 +01:00
Marc Bonnici
150fe2b32b instrument/daq: Provide available devices in error message
Display available daq devices in error message if requested device is
unavailable.
2018-09-21 13:04:27 +01:00
Marc Bonnici
f2a88fd1dc host: Allow pull method to deal with directories
Check to see if the the source path is a directory before attempting to
pull from the host. Use the copy_tree implementation from `distutils`
instead of `shutil` to allow copying into an existing directory.
2018-09-21 13:04:27 +01:00
Marc Bonnici
b7a04c9ebc devlib: Fix incorrect imports 2018-09-21 13:04:27 +01:00
Valentin Schneider
5d97c3186b Add a fallback for sys.stdout.encoding uses
In some environments (e.g. nosetest), sys.stdout.encoding can be None.
Add a fallback to handle these cases.
2018-09-21 09:19:52 +01:00
Volker Eckert
d86d67f49c target.py: cope with non-root users and with non-standard home directories 2018-09-20 10:29:11 +01:00
Marc Bonnici
996ee82f09 utils/rendering: Fix Python 3 compatibility
Add missing encoding of string when writing out fps data.
2018-09-04 13:18:45 +01:00
Sergei Trofimov
61208ce2e0 target: read_tree_values: handle multiline values
Extend Target.read_tree_values() to handle notes that contain line
breaks in their values. Underlying this method, is a call to

	grep -r '' /tree/root/

When files under that location contain multiple lines, grep will output
each line prefixed with the path; e.g. a file "test" with the contents
"one\n\ntwo\n" will be output by grep as:

	/tree/root/test: one
	/tree/root/test:
	/tree/root/test: two

Previous implementation of read_tree_values() was assuming one value per
line, and that the paths were unique. Since it wasn't checking for
duplicate paths, it would simply override the earlier entries resulting
with the value of "two" for test.

This change ensure that such multiline values are now handled correctly,
and the entire value is preserved.

To keep compatibility with existing uses of read_tree_values(), the
trailing new lines are stripped.
2018-08-29 16:17:19 +01:00
Valentin Schneider
8cd1470bb8 module/cpufreq: Add a contextmanager for temporary governor changes
We may sometime want to temporarily use another governor, and then
restore whatever governor was used previously - for instance, RT-app
calibration ([1]) needs to use the performance governor, but we don't
want this to interfere with e.g. our current experiment.

[1]: https://github.com/ARM-software/lisa/blob/master/libs/wlgen/wlgen/rta.py#L118
2018-08-24 15:13:35 +01:00
Valentin Schneider
66be73be3e module/hotplug: Add list_hotpluggable_cpus helper
This is similar to the list_{online/offline}_cpus helpers from Target
2018-08-23 14:30:43 +01:00
Pierre-Clement Tosi
63d2fb53fc Instrument/BaylibreAcme: Add IIO-based ACME instr.
Add BaylibreAcmeInstrument, a new instrument providing support for the
Baylibre ACME board as a wrapper for the IIO interface. This class
provides better access to the ACME hardware (e.g. the ability to control
the sampling frequency) and to the retrieved samples than what the other
instrument, AcmeCapeInstrument, provides. Furthermore, it removes an
unnecessary and limiting dependency by interfacing directly with the IIO
drivers instead of relying on an intermediate script ("iio-capture") potentially
introducing unexpected bugs. Finally, it allows handling multiple probes
(the ACME can have up to 8) through an easy-to-use single instance of this
class instead of having to have an instance of AcmeCapeInstrument per channel
potentially (untested) leading to race conditions between the underlying
scripts for accessing the hardware.

This commit does not overwrite AcmeCapeInstrument as
BaylibreAcmeInstrument does not provide interface compatibility with
that class. Anyhow, we believe that anything that can be achieved with
AcmeCapeInstrument can be done with BaylibreAcmeInstrument (the
reciprocal is not true) so that BaylibreAcmeInstrument might eventually
replace AcmeCapeInstrument.

Add BaylibreAcmeInstrument documentation detailing the class interface
and the ACME instrument itself and discussing the way it works and its
potential limitations.
2018-08-22 16:00:25 +01:00
Marc Bonnici
30dc161f12 trace/perf: Add support for collecting metrics with perf 2018-08-22 14:43:47 +01:00
Marc Bonnici
d6df5c81fd utils/ssh: Force connection to be closed if logout is unsuccessful
If the connection is unavailable when attempting the logout procure it
can fail leaving the connection open on the host. Now if something goes
wrong ensure that we still close the connection.
2018-08-21 09:46:43 +01:00
Marc Bonnici
b0463e58d8 ssh: Use atexit to automatically close ssh connections
As stated in https://github.com/ARM-software/devlib/issues/308 devlib
can leave ssh connections open after they are no longer in use, so now
register the close method with atexit so the connections are no longer
left open upon exit.
2018-08-21 09:46:43 +01:00
Valentin Schneider
512c5f3737 Remove duplicate copyright headers
I used the "Arm Limited" spelling in some headers, but it seems that
didn't get caught by the copyright update script that was used for
9fd690efb3 ("Update copyrights").

This resulted in a duplicated header being inserted, although with the
"ARM Limited" spelling. Remove the previous header and use this one
instead.
2018-08-17 16:06:41 +01:00
Douglas RAILLARD
cc0582ef59 exceptions: Update doc for transient exceptions 2018-08-15 14:32:53 +01:00
Douglas RAILLARD
ec717e3399 netstats: fix typo exception in message 2018-08-15 14:32:53 +01:00
Douglas RAILLARD
511d478164 exceptions: Classify transient exceptions
Exceptions such as TargetError can sometimes be raised because of a
network issue, which is useful to distinguish from errors caused by a
missing feature for automated testing environments.

The following exceptions are introduced:
* DevlibStableError: raised when a non-transient error is encountered
    * TargetStableError

* DevlibTransientError: raised when a transient error is encountered,
including timeouts.
    * TargetTransientError

When there is an ambiguity on the type of exception to use, it can be
assumed that the configuration is correct, and therefore it is a
transient error, unless the function is specifically designed to probe a
property of the system. In that case, ambiguity is allowed to be lifted
by assuming a non-transient error, since we expect it to raise an
exception when that property is not met. Such ambiguous case can appear
when checking Android has booted, since we cannot know if this is a
timeout/connection issue, or an actual issue with the Android build or
configuration. Another case are the execute() methods, which can be
expected to fail on purpose. A new parameter will_succeed=False is
added, to automatically turn non transient errors into transient ones if
the caller is 100% sure that the command cannot fail unless there is an
environment issue that is outside of the scope controlled by the user.

devlib now never raises TargetError directly, but one of
TargetStableError or TargetTransientError. External code can therefore
rely on all (indirect) instances TargetError to be in either category.
Most existing uses of TargetError are replaced by TargetStableError.
2018-08-15 14:32:53 +01:00
Marc Bonnici
d6d322c8ac devlib/__init__: Update installed version to conform with PEP440
In commit fec0868 setup.py was updated to ensure that commit id is
included within the package version however this was not updated to
reflect the change.
2018-07-26 11:50:46 +01:00
Marc Bonnici
ae99db3e24 utils/version: Fix check to only decode bytes
When using Python3 the returned value of the commit is a byte string and
therefore needs to be decoded.
2018-07-26 11:50:46 +01:00
Patrick Bellasi
241c7e01bd cgroups: fix pylin bug
In:

   commit 454b9450 ("pylint fixes")

we added:

```diff
@@ -363,7 +368,7 @@ class CgroupsModule(Module):

         # Get the list of the available controllers
         subsys = self.list_subsystems()
-        if len(subsys) == 0:
+        if subsys:
             self.logger.warning('No CGroups controller available')
             return

```

which changed the invariant condition to enabled the cgroup module:
the module is enabled we we can find a "non empty" list of subsystems.

Let's fix this to bail out on an empyt list.

Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
2018-07-25 15:16:17 +01:00
Patrick Bellasi
68b418dac2 cgroups: explicitly check for proper CGroup naming
The shutils run_into support assumes that we always specify a full
cgroup path starting by "/". If, by error, we specify a cgroup name
without the leading "/" we get an ambiguous message about the cgroup not
being found.

Since this already happened to me many times, let's add an explicit
check about the cgroup name parameter to better info the user about the
requirement.

Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
2018-07-25 15:16:17 +01:00
Sergei Trofimov
df61b2a269 utils/misc: check_output: handle unset sys encoding
Default to assuming 'utf-8' encoding for environments where
sys.stdout.encoding is not set.
2018-07-18 16:54:40 +01:00
Sergei Trofimov
e8a03e00f3 doc: mention ChromeOsTarget in overview
Mention ChromeOsTarget when listing available target types in the
overview.
2018-07-18 09:59:11 +01:00