added more paths to pull by default when device.get_properties is
invoked during run initialization. Also moved the LinuxDevice
implementation into BaseLinuxDevice, so that AndroidDevice tires to pull
the same files (on top of the Android-specific stuff).
list_or_* functions (e.g. list_or_string) will now always return a list,
however will accept lists or indivitual values. Also added a list_or()
generator function, similar to what already exists for list_of().
- Do not try to use a pager if it explicitly disabled with PAGER='' in
the environment.
- If terminal size is identified as (0, 0), fall back to default (80,
25).
As instrumentation can be enabled/disabled for a specfic workload
execution, it is sometimes not enough to verify that an instrument has
been installed for the run; one might need to check whether it is
currently enabled.
Since a command would still be running on time out, it would prevent
issuing subsequent commands in the same SSH shell, make it look like
the device has become unresponsive.
If a timeout condition is his, send ^C to kill the current foreground
process and make the shell available for subsequent commands.
- Adding "classifiers" field to Metric objects. This is a dict mapping
classifier names (arbitrary strings) to corresponding values for that
specific metrics. This is to allow extensions to add
extension-specific annotations to metric that could be handled in a
generic way (e.g. by result processors).
- Updating telemetry workload to add classifiers for the url and internal
iteration (or "time") for a particular result.
- Updating csv result processor with the option to use classifiers to
add columns to results.csv (either using all classifiers found, or
only for the specific ones listed).
The key file must only be readable by the owner. If the specified key
file has different access permissions, create a temporary copy with the
right permissions and use that.
np.vectorize was being unconditionally invoked at top level. On an
ImportError, np as set to None, so this was resuling in an
AttributeError when loading the module if one of the dependent libraries
was not present on the host system. This moves the invocation into the
try block with the imports to avoid an error when energy_model module is
loaded by the extension is not used.
- adding missing cpu id conversion to set_cpu_min_freq
- adding "exact" parameter to set_cpu_frequency; only produce an error
when the specified frequency is not supported by the cpu if "axact"
is set; (otherwise let cpufreq decide what the actual frequency will
be).
- adding missing cpu id conversion to set_cpu_min_freq
- adding "exact" parameter to set_cpu_frequency; only produce an error
when the specified frequency is not supported by the cpu if "axact"
is set; (otherwise let cpufreq decide what the actual frequency will
be).
Make sure when migrating sshd to root cgroup also migrate their
children, including the bash for the wa session. So the subsequent
processes kicked off from that shell can be taskset to any cluster.
Matplotlib defautls to the GTK backend. This can cause problems when
running in a headless session (e.g. over SSH). Since energy_model
istrument generates PNG plots, rather than rendering directly to UI, it
doesn't actually need GTK; set backend to AGG so that energy_model works
in headless environments.
This instrument can be used to generate an energy model for a device
based on collected power and performance measurments. The instrument
produces a C file with an energy model and an accompanying HTML report.
This instrument is very different from other instrumentation, as it
actually generates the run queue on the fly based on the operating
frequencies and idle states it discovers on the device. The agenda needs
only to contain the single "base" spec that defines the workload to be
used for performance measurement.
cpuidle module implements cpuidle state discovery, query and
manipulation for a Linux device. This replaces the more primitive
get_cpuidle_states method of LinuxDevice.
Renamed APIs (and added a couple of new ones) to be more consistent:
"core" APIs take a core name as the parameter (e.g. "a15") or whatever
is listed in core_names for that device.
"cluster" APIs take a numeric cluster ID (eg. 0) as the parameter. These
get mapped using core_clusters for that device.
"cpu" APIs take a cpufreq cpu ID as a parameter. These could be
integers, e.g. 0, or full string id, e.g. "cpu0".
integer and boolen are defined in wlauto.utils.types; they perform more
intuitive conversions from other types, particularly strings, so are
more suitable than int and bool for parameters. If, for whatever reason,
native types are in fact desired for a Parameter, this behavior can be
supressed by specifying convert_types=False when defining the parameter.
- list_to_range and range_to_list convert between lists of integers
and corresponding range strings, e.g. between [0,1,2,4] and '0-2,4'
- list_to_mask and mask_to_list convert between lists of integers and
corresponding integer masks, e.g. between [0,1,2,4] and 0x17
Conflicts:
wlauto/utils/misc.py
- added caseless_string type. This behaves exactly like a string, except
this ignores case in comparisons. It does, however, preserve case. E.g.
>>> s = caseless_string('Test')
>>> s == 'test'
True
>>> print s
Test
- added list_of type generating function. This allows to dynamically
generate type-safe list types based on an existing type. E.g.
>>> list_of_bool = list_of(bool)
>>> list_of_bool(['foo', 0, 1, '', True])
[True, False, True, False, True]
- Update core_names Device Parameter to be of type caseless_string
- added the ability to run based on time, rather than number of
requests.
- added a parameter to taskset to specific core(s).
Conflicts:
wlauto/workloads/sysbench/__init__.py
A "please rate me" dialog occasionally pops but when returning to the
initial screen (when re-running the test). The check to dismiss it
wasn't being done at the right time, so it was still preventing
mutli-times execution. This commit resolves that issue.
"times" parameter didn't work properly for version 5 because an extra
back button press was required due to UI changes from previous versions.
This commit adds the button press.
On some systems the temporary directory may still be "busy" after WA is
don't with it. Since it's just an empty dir and it will be reused on the
subsequent runs; do check if rm -rf succeeded.
When the merging logic was updated to preserve duplicates within the
same list, it inadvertently broke the logic that removed items marked
for removal with a '~'. This commit rectifies that.
Note to self: merging functions are doing *way* to much; they should be
refactored into several individual function and config should be
resolved in distinct stages.
If list parameter values contained duplicates, those got removed when
merging parameter values from different sources. This commit fixes that
behavor, so that duplcates that appear within the *same* list are
preserved.
Keep duplicates in lists when merging device_config dict from agenda
with the rest of the config; This will ensure that core_name sand
core_clusters aren't reduced to just unique elements.
- raise an error if an agenda contains duplicate keys (by default PyYAML
will silently ignore this)
- raise an error if config section in an agenda is not dict-like
(before, this was allowed to propagate and relsulted an a traceback
further down the line).
Very old versions of IPython do not have IPython.version_info attribute
that the ultls module relied on. This commit changes it to use the more
standard __version__ attriute that is present in all versions.
check_output will ignore error code 1 returned by telemetry execution,
as this happens when individiual sub-tests and partial results may, and
should, still be extracted.
Adding a parater to wlauto.utils.misc.check_output to specify that it
should ignore certain error codes when they are returned by the
subprocess and not raise them as errors.
Commit 2276ae0c5b ("Fixing config processing for extensions with
non-identifier names.") broke customizing the trace-cmd instrumentation
from the agenda. With an agenda like:
config:
instrumentation: [trace-cmd, delay]
trace_events: ['thermal*']
trace_buffer_size: 28000
trace_events and trace_buffer_size get added to the RunConfiguration's
_raw_config under the trace-cmd name, but then when it's looked up in
_finalize_config_list(), the dictionary is actually looked up using
identifier(extname), i.e. 'trace_cmd'. Fix this by adding the user's
configuration using identifier(name) as well.
run_cell() becomes more complicated when we add ipython version 3
support which upsets pylint because there are "too many
branches (15/12)". Factor out part of the function to make pylint
happy.
The kernel may go idle before it processes the next input, which break
the while=True loop in run_cell() early. Wait for an acknowledgement
of the input we've sent to the kernel before considering an idle
message to mean that the cell has been parsed.
The internal ABI for ipython has changed between ipython version 2 and
3. In its current state, the result processor only works with IPython
version 2, so fail if the user wants to use the result processor with
the wrong version.
Abstract the ipython interface to a file so that we can make it support
versions 2 and 3 at the same time.
The default value for buffer_size_file contained a path under "/d/",
which is an Android-specific alias for "/sys/kernel/debug". This commit
updates the default value to use the system-agnostic path.
Internally, WA expects extension names to be valid Python identifiers.
When this is not the case, conversion takes place when loading
configuration for the extension (e.g. "trace-cmd" gets converted to
"trace_cmd").
The conversion is intended to be transparent to the user, so
configuration stores values as they are provided by the user, however it
needs to perform the conversion internally, e.g. when querying
ExtensionLoader. This conversion was missing when performing a lookup of
one of the internal structures, which was causing earlier-collected
settings to not be propagated into the final config.
Updating SysfsExtractor with a parameter to explicitly enable/disable
tempfs caching. Previously, this was determined entirely by whether the
device is rooted.
This result processor displays a desktop notification when the run
finishes. It's useful when you are running a long agenda in WA and want
to be notified when the results are available.
The param documentation states that for a boolean, "kind" should be
"as_bool" from wlauto.utils.misc, but there is no "as_bool". Currently,
workload automation automatically converts native python types like bool
and int to workload automation specific ones. Remove this bit from the
documentation as it's not true any more.
Change-Id: I0100708530bcf67556eda386c39bc444a3e0f2b2
The error message will now contain the offending extension (either
package name or full path to extension file) if a Python error occurs
when ExtensionLoader attempts to load it.
- connect() to device before issuing the initial reboot, as soft reset
requires a device connection.
- boot() has been implemented to wait properly for the device to reboot
after reset.
- port now defaults to 22 rather than being left unset, as need
something to connect to when polling for device after reboot.
- Only use -P option for scp when port is *not* 22; as that option
appears to cause intermittent issues with default scp on Ubuntu 12.04
On some devices, sudo presents a different prompt when asking for a
password. All the prompt to be specified in device configruation to
handle such cases.
Make the WorkloadError() message clearer if the sysbench
binary cannot be found on the device and host.
Also make trivial fixes to improve descriptions and fix
spelling errors.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
terminalsize was loaded from a location added to sys.path during
bootstrap. This appeared to be causing import issues. There is no longer
a good reason for terminalsize to be loaded that way, so just moved it
under wlauto.utils so that it can be loaded normally.
Some Linux devices may run on minimal root file systems
(e.g. buildroot) where /usr/local/bin path doesn't exist. Create
the binaries_directory if it doesn't exist instead of letting WA
quit and return errors such as:
INFO Runner: Skipping the rest of the iterations for this spec.
ERROR Runner: Error while running memcpy
ERROR Runner: CalledProcessError("Command 'sshpass
-p '<password>' /usr/bin/scp -r
/usr/local/lib/python2.7/dist-packages/
wlauto-2.3.0-py2.7.egg/wlauto/workloads/memcpy
/memcpy root@192.168.x.x:/usr/local/bin/memcpy'
returned non-zero exit status 1")
ERROR Runner: Got:
ERROR Runner:
ERROR Runner: scp: /usr/local/bin/memcpy:
No such file or directory
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
- UEFI config can be specified as a device_config parameter
- The same config is used to create a missing UEFI entry, and
to re-create the entry when flashing. UEFI config now wholy
resides within the device and is not specified for vexpress
flasher.
- Updated sysbench binary to a statically linked verison
- Added missing LICENSE file for the sysbench binary
- Removed Android browser launch and shutdown from workload (now runs on both
Linux and Android)
- Updated update_result to work with the new binary
- Added missing descriptions for parameters
- Added file_test_mode parameter -- this is a mandatory argumet if test is
fileio
- Added cmd_params parameter to pass options directily to sysbench invocation
The struct used to read events is being padded when built for 64
bit platforms. The padding has been made explicit in the struct, and
matching padding was added when writing the events during recording.
On generic_linux devices, one might ssh as the root user, in which
case there is no need to use sudo.
In addition, some root filesystems may not have sudo (e.g. minimal
buildroot/busybox).
This patch attempts to detect the root user using 'id -u' and, if it
detects the root user, avoids the use of sudo for running commands as
well.
Cc: Lisa Nguyen <lisa.nguyen@linaro.org>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
(setrofim: modified to only test once)
Then number of input event files was being written as a size_t but read
as an int by revent. These types have different sizes with 64bit GCC,
causing revent not being able to replay recorded files. This comint
updates revent to use size_t when both reading and writing.
The boot monitor seems to have some buffer overrun issue while loading the latest Linaro android images.
The behavior is after loading the initrd, the kernel loading will fail, very likely to due command line buffer overrun.
the new initrd in the Linaro image is 3 times larger than the previous version, which could cause the issue.
putting a dummy enter between loading initrd and the kernel could resolve the issue.
