This is copied from WA (workload-automation/wa/utils/misc.py).
Hence, published another PR [1] removes the implementation from WA.
OTOH, this patch uses ``ruamel`` instead of ``yaml`` because of the
latter's design issues.
And also this patch fixes pylint issues in ``load_struct_from_yaml()``.
[1] https://github.com/ARM-software/workload-automation/pull/1248
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
'if list.get(elem, None)' like probing ignores list elements whose
values are falsy.
Here is a sample test code:
```Python
connection_settings={'host': '127.0.0.1',
'port': 8022,
'username': 'root',
'password': 'root',
'strict_host_check': False}
ssh_conn_params = ['host', 'username', 'password', 'port', 'strict_host_check']
print(f'connection_settings={connection_settings}')
ssh_connection_settings = {}
for setting in ssh_conn_params:
if connection_settings.get(setting, None):
print(f'1. setting "{setting}" to "{connection_settings[setting]}"...')
ssh_connection_settings[setting] = connection_settings[setting]
else:
print(f'1. "{setting}" is None!')
ssh_connection_settings = {}
for setting in ssh_conn_params:
if setting in connection_settings:
print(f'2. setting "{setting}" to "{connection_settings[setting]}"...')
ssh_connection_settings[setting] = connection_settings[setting]
else:
print(f'2. "{setting}" is None!')
```
And its output:
```
connection_settings={'host': '127.0.0.1', 'port': 8022, 'username': 'root', 'password': 'root', 'strict_host_check': False}
1. setting "host" to "127.0.0.1"...
1. setting "username" to "root"...
1. setting "password" to "root"...
1. setting "port" to "8022"...
1. "strict_host_check" is None!
2. setting "host" to "127.0.0.1"...
2. setting "username" to "root"...
2. setting "password" to "root"...
2. setting "port" to "8022"...
2. setting "strict_host_check" to "False"...
```
Also fix a typo in a log message.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Loading host keys breaks setting up SSH connection (paramiko throws
BadHostKeyException exception) if issuer does not want/need strict key
matching.
One use case for ignoring strict_host_check is automating virtual guests
(i.e., over QEMU). Issuer may want to skip loading host keys and start
with a blank list of known host keys for sure.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
Not all command executions (or write operations in this specific case)
requires being root. So, allow write_value() and dependent
revertable_write_value() to support non-root executions by introducing
'as_root' optional parameter whose default is True to preserve current
behavior of the aforementioned methods.
Meanwhile, update the copyright year of the touched file, too.
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
I noticed the following errors during invocation of uibench/uibenchjanktests:
job: Initializing job wk1 (uibench) [1]
signal: Sending before-workload-initialized from wk1 (uibench) [1]
apk: Resolving package on host system
resolver: Resolving <<Workload uibench>'s apk 14>
resolver: Trying user.get
signal: Sending error-logged from <ErrorSignalHandler (DEBUG)>
signal: Disconnecting <bound method Executor._error_signalled_callback of executor> from error-logged(<class 'louie.sender.Any'>)
signal: File "/repos/lisa/external/workload-automation/wa/framework/signal.py", line 324, in wrap
signal: yield
signal: File "/repos/lisa/external/workload-automation/wa/framework/job.py", line 97, in initialize
signal: self.workload.initialize(context)
signal: File "/repos/lisa/external/workload-automation/wa/utils/exec_control.py", line 83, in wrapper
signal: return method(*args, **kwargs)
signal: File "/repos/lisa/external/workload-automation/wa/framework/workload.py", line 305, in initialize
signal: self.apk.initialize(context)
signal: File "/repos/lisa/external/workload-automation/wa/framework/workload.py", line 717, in initialize
signal: self.resolve_package(context)
signal: File "/repos/lisa/external/workload-automation/wa/framework/workload.py", line 734, in resolve_package
signal: self.resolve_package_from_host(context)
signal: File "/repos/lisa/external/workload-automation/wa/framework/workload.py", line 774, in resolve_package_from_host
signal: apk_file = context.get_resource(ApkFile(self.owner,
signal: File "/repos/lisa/external/workload-automation/wa/framework/execution.py", line 197, in get_resource
signal: result = self.resolver.get(resource, strict)
signal: File "/repos/lisa/external/workload-automation/wa/framework/resource.py", line 268, in get
signal: result = source(resource)
signal: File "/repos/lisa/external/workload-automation/wa/framework/getters.py", line 139, in get
signal: return get_from_location(directory, resource)
signal: File "/repos/lisa/external/workload-automation/wa/framework/getters.py", line 106, in get_from_location
signal: return get_generic_resource(resource, files)
signal: File "/repos/lisa/external/workload-automation/wa/framework/getters.py", line 63, in get_generic_resource
signal: if resource.match(f):
signal: File "/repos/lisa/external/workload-automation/wa/framework/resource.py", line 165, in match
signal: uiauto_matches = uiauto_test_matches(path, self.uiauto)
signal: File "/repos/lisa/external/workload-automation/wa/framework/resource.py", line 335, in uiauto_test_matches
signal: info = get_cacheable_apk_info(path)
signal: File "/repos/lisa/external/workload-automation/wa/utils/android.py", line 192, in get_cacheable_apk_info
signal: info = ApkInfo(path)
signal: File "/repos/lisa/external/workload-automation/wa/utils/android.py", line 116, in __init__
signal: super().__init__(path)
signal: File "/repos/lisa/external/devlib/devlib/utils/android.py", line 152, in __init__
signal: self.parse(path)
signal: File "/repos/lisa/external/devlib/devlib/utils/android.py", line 159, in parse
signal: output = self._run([self._aapt, 'dump', 'badging', apk_path])
signal:
signal: Sending error-logged from <ErrorSignalHandler (DEBUG)>
signal: AttributeError('ApkInfo' object has no attribute '_aapt')
signal: Sending after-workload-initialized from wk1 (uibench) [1]
signal: Sending error-logged from <ErrorSignalHandler (DEBUG)>
runner: Skipping remaining jobs due to "'ApkInfo' object has no attribute '_aapt'".
This is due to the fact we might call self.parse in ApkInfo::__init__, if the
path variable is set to a non-empty value, but the initialization of both
self._aapt and self._aapt_version is after this call.
Fix this by moving the initialization of both variables before the call to
self.parse.
Fix inadequate use of module-level __getattr__ (it is not used by the
global variable lookup path). Instead, detect all tools lazily in the
same fashion as with _AndroidEnv()
Module-level __getattr__ is not called on the global variable lookup
path, rendering it useless for what we want to do here.
Instead, use the _AndroidEnvironment class and make it lazy so that we
will not raise an exception by just importing the module.
Android tools detection was manipulating os.environ['PATH'] which has
an impact beyond devlib (and even beyond the current process as it will
be inherited by any child).
Remove that hack and instead use global variables to get adb and
fastboot paths. These tools are now detected by _AndroidEnvironment()
like the others.
* Use lazy global var init using module-level __getattr__() and remove
all the _check_env() calls.
* Cleanup the code by removing unnecessary statefullness. While doing so,
prune paths that can never happen.
* Encapsulate all the logic in _AndroidEnvironment() instead of mutating
it using standalone functions.
* Set "adb" and "fastboot" global variables to None as fastboot was
always set to None, and adb was set to None on the path with
ANDROID_HOME env var set.
PID detection can sometimes fail for unknown reason. Maybe there is a
latency between the process being started and "ps" being able to see it
that can sometimes be high enough that we look for the process before
it's exposed.
In order to remedy that, add a retry logic to avoid plain failures.
Introduce is_screen_locked() which returns true if device screen is
locked and false otherwise.
This will be useful to automate unlocking the screen [1].
Also fix a typo in is_screen_on()'s documentation.
[1] https://github.com/ARM-software/workload-automation/pull/1246
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
is_screen_on() should also check if the screen is in 'Dozing' state. If
the device is dozing, then is_screen_on() should return false.
Without this patch, is_screen_on() throws 'Could not establish screen
state' exception if the device is idling (screen is completely off).
Signed-off-by: Metin Kaya <metin.kaya@arm.com>
dex file search is modified. It collects all the available methods under the package name. Tested with other benchmarks (geekbench,pcmark,jankbench in Android 12) as well.
Android GKI kernels have versions such as:
5.15.110-android14-11-ga6d7915820a0-ab10726252
Update the parsing regex to include:
* gki_abi: 10726252 in this example
* android_version: 14 in this example
This also allows parsing the git sha1 correctly, which otherwise is
broken on a version like that.
Fixes https://github.com/ARM-software/devlib/issues/654
Adding thermal zones and trip points are only done at thermal module
initialization. There is no need for these functions to be public.
Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
The thermal module currently only reads thermal zone ids and allow
temperature reading. The mandatory thermal zone 'type' describes
what the zone is and is therefore quite useful information. This
commit also adds support for reading the current thermal zone
policy and available policies along with a few other properties.
This commit also adds async support to the thermal module.
Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
The sysfs documentation mentions that the value written
to buffer_size_kb ftrace field may be rounded up.
So skip the verify loop on this field.
The case we are worried about, a requested buffer
size that the target cannot fulfill is caught anyway,
as the sysfs write returns with an error that is caught.
Signed-off-by: Christian Loehle <christian.loehle@arm.com>
Using SSHConnection(use_scp=True) lead to an exception:
UnboundLocalError: local variable 'handle' referenced before assignment
This is cause by some (false) cyclic dependency between initialization
of SSHTransferHandle and creation of the SCPClient. We can fix that by
adding a level of indirection to tie together both objects.
Add a Collector for accessing Google's Perfetto tracing infrastructure.
The Collector takes a path to an on-device config file, starts tracing
in the background using the perfetto binary and then stops by killing
the tracing process.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Add the "is_running" function that can be used to check if a given
process is running on the target device. It will return True if a
process matching the name is found and Falsa otherwise.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Update busybox to version 1.36.1 with defconfig and uniformly built on
Alpine v3.18, statically linked to musl libc.
Binaries were built using lisa-build-asset from LISA project:
lisa-build-asset busybox --native-build
Since we now set the top buffer size to be the same as the devlib buffer
size, this effectively halves the maximum available buffer size that can
be set while using devlib. Whatever size is passed as `buffer_size` will
be allocated twice, even if the top buffer is hardly used at all.
This commit separates them into `buffer_size` and `top_buffer_size`. If
the latter is not passed, the behaviour will not change compared to now.
Fixes: e0c53d09990b5501e493d048a5dce067d8990281
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Installing busybox sometimes fails with:
cp: /data/local/tmp/bin/busybox: Text file busy
This happens when trying to modify a binary file while a process is
still running (e.g. unclean previous disconnection).
Fix that by using the -f option, which will remove the destination file
first and retry the copy in case of failure.
Only run the minimal amount of commands, as executing a command can be
costly.
In the sequence reset() -> start(), we only need to get the output of
dmesg upon start() to know what part of the log will be ignored
(everything before the call to start()). There is no need to perform
that upon reset() since the sequence:
reset() -> start() -> stop() -> start() -> stop()
\______1________/ \______2________/
is anyway equivalent to:
reset() -> start() -> stop()
\______2________/
So reset() can essentially be a no-op and the actual reset logic lives
in start().
Change the SurfaceFlingerFrameCollector to match activities by prefix
instead of looking for an exact match. This will allow to account for
activities with variable suffixes.
Raise an error if more than one activity matches the provided view.
Show a warning if no activities match the provided view in order to
avoid silently failing.
Suggested-by: Andriani Mappoura <andriani.mappoura@arm.com>
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
