Change the recording of the run stage during workload recording mode to be
optional as the user may want to use the command to re-record a specific
stage of a workload without affecting the workload recording.
Hitting CTRL-C will abort execution of the current job, but will still
trigger run finalization, and possibly, post-processing and teardown of
the current job. If an exception is raised during this
post-process/teardown, the previous exception state (for the
KeybardInterrupt) will be clobbered. That means that, after the new
exception has been handled, WA would attempt to execute the next job,
rather than go to finalization of the run.
To avoid this, set a flag in the context upon catching KeybardInterrupt,
and check this flag before attempting to execute the next job in the
queue.
Rather than relying on a custom Logger with a context to add events when
they are logged, have the runner register hooks for corresponding
signals that do that.
The issue with the previous approach is that the event ended up
being added too late -- after the current job was already cleared, so
all events were added at run level.
With the new approach, the event will be added when it is first logged,
while there is still a current job being set. There will be no
duplication for Exceptions being re-raised and handled at different
levels because log_error() will ensure that each Exception is logged
only once.
- Add SimpleTargetDescriptor that simply returns a global list of target
descriptions.
- Add create_target_description function to populate that list with a
description crated from the specified components.
- Add add_description_for_target helper function to create a description
for the specified target class, and specified components. For
components that are not explicitly specified, attempt to guess
sensible defaults based on the target class' bases.
- fix TargetDescription._set to set lists rather than dicts for params
to be consistent with how params are set after creation (should
probably get rid of this entirely...)
- fix the comment describing the structure of TARGETS
- Keep track of logged exceptions inside log_error itself.
- signal: log the exception, if there is one in the finally clause of
the signal wrapper; this will ensure that the error will be logged
closer to the command that originated.
- entrypoint: use log.log_error for top-level error logging, rather than
the entrypoint logger directly; this will ensure that errors are not
repeated unnecessarily.
- Log CTRL-C message at zeroth indent level to make it easier to see in
the non-verbose output where it occurred.
If a target error occurs, check whether the target is unresponsive. If
it is, attempt to hard reset it if possible, or gracefully terminate
execution if not.
Add a method to check that the target is still responsive, and cache
the value so that the target does not need to be repeatedly probed
during various stages of wrapping up execution.
We remove the requirement for rt-app to run on a rooted device as this
does not appear to be required. We make sure that we run the workload
from a directory where we have write permissions by first changing the
directory before executing the workload (in_directory argument of
invoke).
In the case that the device is rooted, we run rt-app as
root. Otherwise, we simply run the workload.
By default rt-app writes to stderr, not to stdout. Therefore, if we
only sample the stdout, we get no output from the workload itself.
With this change, we redirect stderr to stdout to allow us to get the
workload output.
ExecutionContext.end_run() does final updates to the run info in the run
output (final status, run duration, etc). This was previously accessed
via self.output in the context. Typically, this would correctly resolve
to the run output, as there would be no current job. However, in the
event of a crash, current_job would be set, and this would resolve to
the job output itself, resulting in run info not being updated. Use
run_output to avoid this.
- RebootPolicy: correctly handle being passed another instance of
RebootPolicy on creation; this a requirement for it to be usable as a
ConfigurationPoint kind.
- RebootPolicy: move static method to above __init__ (consistency).
- RunConfiguration: handle deserialization of 'device_config' entry.
- RunConfiguration: move static method to above __init__ (consistency).
Save classifiers at Result as well as Metric level. Reason: when
processing output, one might want to filter complete results, as well as
individual metrics. While it is in theory possible to get the
classifiers for a job by simply extracting the common classifiers
between all metrics, this fails when there are no metrics generated for
a job (note that one might still want to process the output in this
case, e.g. for the artifacts).
Set target_binary as a class, rather than instance, attribute. This
happens only only once per run, and setting it as instance attribute the
first time, makes it unavailable for subsequent instances of the same
workload.
Do not explode if a result file for a job is missing when loading a
RunOutput. Specify job status as "UNKNOWN" and add the exception from
attempting to load the file to the events.
Fix an issue introduced by commit 42fb3eb
framework/workload: make "no matching package" message more useful
PackageHandler was incorrectly passing self.name into the error message
instead of self.owner.name.
Make the error message reported when no matching Android app package
could be found more useful by including the workload name, and, if
applicable, sought version.
Always prepend the format for the file into the downloaded filepath to
ensure matching is correctly performed. Previously the ogg file did not
specify this causing it to be re-downloaded for each run.
Rename the `start` method to `run` as this is what is what is called
by the threading module's `start` method, otherwise this causes the
polling to be done in the main thread blocking execution.
Add support for loading plugins from packages that register themselves
with WA by writing their names into $WA_USER_DIRECTORY/packages.
This is functionality that existed in WA2 and was omitted in WA3 until
now.
Output processor names are converted to identifiers when they are
loaded. Ensure the same thing happens when looking up installed output
processors with get_output_processor().
In devlib the default working directory has changed causing the workload
to look in the wrong directory, now the workload should use the passed
working directory and extract the relevant name from there.
Both the instrument and utility functions were called `trace-cmd.py`.
`"-"`'s are not valid python identifiers therefore when trying to import
the `trace-cmd` module this causes an error.
- Add an option to the poller binary insert a marker into ftrace that
aligns with the first output entry. The option is exposed as an
instrument parameter.
- If this parameter is set, the resulting .csv will be post-processed to
update the timestamps to align with trace-cmd output.
- Change poller artifact names to use - instead of _ to be consistent
with trace-cmd artifact naming.
Allows setting of select android runtime configuration for devices running
chromeos and which support android. Currently only 'brightness' is functioning
correctly therefore the other parameters are only enable for standard android devices.
Previously when validating cpufreq runtime parameters it would check too
early whether the cpu was online, not allowing for the fact that the cpu
might be being onlined before the parameter was committed to the device.
To improve reproducibility of tests, it could help to ensure that the
same screen orientation is used on media playback.
This adds a new workload parameter which allows to ensure a LANDSCAPE
mode, which can be useful to better represent video playback scenarios.
When not enforced by the agenda, we enforce a PORTRAIT orientation.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Fix the setting of runtime parameters when the casing of the parameter in
the agenda does not match the "canonical" casing.
To make the writing of agendas easier, the casing of the parameters is
supposed to be ignored. To achieve this, parameter names are converted to
caseless_string type before they are looked up.
caseless_string's do not work with dicts. Both __contains__ (used for
the "in" operator) and __getitem__ (used for the [] operator) implement
hash-based look up, and it is not possible to have a caseless_string
match against multiple hashes to cover the different casing
possibilities.
So instead, iterate over the items in parameters dict, comparing the
caseless_string name to the key, and returning the value if it matches.
On some devices an installed apk filepath can contain an '=' character
which was previously used to end the regex match. Now match with the
package name as well to ensure the file path is extracted correctly.
Uses os.eviron as the default environment for creation of a
MetaConfiguration object, this allows an instance to be created without
providing any parameters and fixes re-creating an instance from pod.
Changed location of revent binary on target to be a class attribute so that
it is shared across all instances. Also changed discovery method to a
static method which can be memoized to prevent the target being queried
unnecessarily.
use the new get_target_description() when initializing the target inside
TargetManager, instead of manually searching through results of
list_target_descriptions().
The show command will try to use pandoc and man to display the relevant
information and will fall back to printing the markdown if pandoc is not
available.
Instead of taking an extension as a parameter and assuming there it has
an attribute 'parameters' instead take a list of parameters directly
which allows the method to be used with parameter lists with different
names.
Make TargetInfo an attribute of run output, replacing the read/write
methods for the targetfile. Instead, always load it on creation, if
targetfile exists (useful for external scripts), and have a method to
set it after creation (uselful during WA run, where the output is
created before connecting to the target).
Fix TargetInfo.from_pod(). As part of creating the POD, some of devlib's
classes such as Cpuinfo, get serialized. Since they are from devlib,
they do not implement WA's serialization interface, and are instead
serialized by extracting their internal structures. They are then not
deserialized properly, since their __init__'s expect text rather than
parsed structures.
This implements deserialization functions for these classes which
initialize them properly from TargetInfo POD.
Fix output when running for a specified number of loops (rather than
duration) and adding delay to threads. Previously, the output for the
last thread was not printed due to a missing check for a negative
duration.
