- Replace ugly while True loop with a simple regex substitution
achieving the same thing. This is required to match the command in
the shell output when the command wraps around due to the length of
the command.
- Implementation of a gem5 device which allows simulated systems to be
used in the place of a real device. Currently, only Android is
supported.
- The gem5 simulation is started automatically based on a command line
passed in via the agenda. The correct telnet port to connect on is
extracted from the standard error from the gem5 process.
- Resuming from gem5 checkpoints is supported, and can be specified as
part of the gem5 system description. Additionally, the agenda option
checkpoint_post_boot can be used to create a checkpoint
automatically once the system has booted. This can then by used for
subsequent runs to avoid booting the system a second time.
- The Gem5Device waits for Android to finish booting, before sending
commands to the simulated device. Additionally, the device supports
a sleep option, which will sleep in the simulated system for a
number of seconds, prior to running the workload. This ensures that
the system can quieten down, prior to running the workload.
- The Gem5Device relies of VirtIO to pull files into the simulated
environment, and therefire diod support is required on the host
system. Additionally, VirtIO 9P support is required in the guest
system kernel.
- The m5 writefile binary and gem5 pseudo instruction are used to
extract files from the simulated environment.
- Allowed telnet connections without a password.
This is required as part of the upcoming Gem5Device, which uses a
password-less telnet connection to communicate with the device.
Add video-long.json and video-short.json files and
update spreading-tasks.json. The originals can be
found in git.linaro.org/power/rt-app.git repo.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
Test 0 (write) creates a file that is used by subsequent tests.
Therefore if this test is not specified withen selecting which tests to
run, izone will fail with an error.
To avoid this, check the tests list specified by the user andd add test
0 if necessary.
In order to show more detailed results for the iozone
workload, inform users to enable classifiers in their
agenda or config file.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
When users specify tests, the parse_metrics()
function doesn't capture the last report name and its
results during the parsing process. Fix the
parse_metrics() function to make sure the data for
all reports are captured.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
Pulling entries from procfs does not work on some platforms. This commit
updates get_properites() to cat the contents of a property_file and
write it to output file on the host, rather than pulling it (directories
are still pulled).
When enabled, this wil cause the instrument to insert a marker into
ftrace, while at the same time setting a GPIO pin high.
For this to work, GPIO sysfs interface must be be enabed in the kernel
and the specified pin must be exported.
Bbench only works on Android. It should advertise that fact by setting
supported_platforms to ['android'].
Telemetry is a Chrome browser workload that is only supported on
ChromeOS and Android.
With this option, only extensions packaged with WA itself will be
listed. Extensions discovered from other packages or from local paths
will not appear in the list.
A newer version of rt-app has been released. Built
binaries from git://git.linaro.org/power/rt-app.git.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
Added functions to parse thread-mode results and
non-thread mode results accordingly, in addition
to rewriting the update_result() function.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
When no files are listed for one the specified directoris, the
instrument checks whether there are any files in that directory on the
device. If that directory itself does not exist; that will result in an
error. This is now handled correctly.
The warning refers to trace.bin, which is not the extension that we use
for it. Instead, use the variable with the default trace
name (trace.dat) for the warning.
- fixed_before_start is a fixed time period equivalent to
temperature_before_start
- Changed existing *_between_specs and *_between_iterations callbacks to
be very_slow, as they should always run after everything else.
A quadratic is now fitted to single and two-core power measured across
frequencies. This quadratic is then used in projection of cluster power.
This mitigates issues with cluster powers going negative or "crossing
over".
There's a known issue that HDMI will lose sync with monitor, adding video kernel parameter will make the HDMI more stable for juno
> HDMI can lose sync with the monitor intermittently, particularly at higher resolutions.
> If you are affected by this then try adding a kernel command line argument that forces
> a video mode with reduced blanking, such as the following:
> video=DVI-D-1:1920x1080R@60
Failed jobs will now be automatically retired. This is controlled by two
new settings:
retry_on_status - a list of statuses which will be consided failures and
result in a retry
max_retries - number of retries before giving up
It is now possible to generate agendas for a set of extensions with
wa create agenda ext1 ext2 ext3 -o output.yaml
Confiuration will include all parameters for those extensions with
default values.
Dynamic modules may be loaded automatically on device initialization if
the device supports them. Dynamic modules implent probe() method to
determine whether they are supported by a particular deviced.
devcpufreq and cpuidle have been converted into dynamic modules
- standardisded on a single context argument
- removed Device.init() no longer necessary as initilize now
automatically gets propagated up the hierarchy. Renamed the existing
use of it to "initilize".
- related pylint cleanup.
uname is not available on all Android devices, and we cannot rely on
busybox for establishing the ABI, as we need to get the ABI before we
can depoly the right version of busybox.
Added -p option to the list command. This alows filtering results by
supported platforms, e.g.
wa list workloads -p linux
Also adding missing supported_platforms attribute to various extensions.
If an extension does not have this attribute, the assumption is that it
is supported by all available platforms.
In previous versions of Android, "mount" returned output in the format
similar to fstab entries, which is what list_file_systems expected. This
fixes it to be able to handle the more traditional "mount" output in the
format
<device> on <mount point> type <fs type> <options>
as well as continue to parse the Android output correctly.
Adding global aliases to parameters to fix old configs that still used
the deprecated "<ext name>_<param name>" format for specifying parameter
values.
- added initialze and finalize methods to workloads, which were the only
major extension types that did not have them
- Semanatics for initialize/finalize for *all* Extensions are changed so
that now they will always run at most once per run. They will not be
executed twice even if invoke via istances of different subclasses (if
those subclasses defined their own verions, then their versions will
be invoked once each, but the base version will only get invoked
once).
In both cases, the ABI is now obtained by executing "uname -m" on the
device and perfroming a mapping from the returned machine architecture
a known ABI. If no known ABI is found the architecture string itself is
returned.
- add IterationResult-level classifiers that get merged into every
added metric (saves having to pass the same classifiers to each
metric added).
- Added a global alias to csv result processor's use_all_classifiers
attribute.
- updated energy_model to accept dhrystone as well as sysbench as
the workload
- added "threads" parameter to sysbench (basically, an alias for
"num_threads") to be consistent with dhrystone
- added "taskset_mask" parameter to dhrystone to allow pinning
it to specific cores.
- Now works with earlier versions of the DAQmx driver. This is needed to
be able to run the server on Linux systems, which support older
verisions of the driver only.
- DAQ error messages are now properly propaged to the client (PyDAQmx
uses "mess" rather than "message" attribute to store the message in
the Exception obejects).
- pylint and pep8 fixes
Background processes may produce output on STDOUT. This could get
captured when obtaining the result of "echo $?" to get previos command's
exit code. So it's not safe to assume that output will always be an int.
Attempt to strip out superflous output before doing the int conversion
and, on failure, log a warning but don't error out.
Some options in config_example.py are not reachable. It makes users
confused. So make the options in the file as gobal_alias.
Signed-off-by: zhizhou.zhang <zhizhou.zhang@spreadtrum.com>
added more paths to pull by default when device.get_properties is
invoked during run initialization. Also moved the LinuxDevice
implementation into BaseLinuxDevice, so that AndroidDevice tires to pull
the same files (on top of the Android-specific stuff).
list_or_* functions (e.g. list_or_string) will now always return a list,
however will accept lists or indivitual values. Also added a list_or()
generator function, similar to what already exists for list_of().
- Do not try to use a pager if it explicitly disabled with PAGER='' in
the environment.
- If terminal size is identified as (0, 0), fall back to default (80,
25).
As instrumentation can be enabled/disabled for a specfic workload
execution, it is sometimes not enough to verify that an instrument has
been installed for the run; one might need to check whether it is
currently enabled.
Since a command would still be running on time out, it would prevent
issuing subsequent commands in the same SSH shell, make it look like
the device has become unresponsive.
If a timeout condition is his, send ^C to kill the current foreground
process and make the shell available for subsequent commands.
- Adding "classifiers" field to Metric objects. This is a dict mapping
classifier names (arbitrary strings) to corresponding values for that
specific metrics. This is to allow extensions to add
extension-specific annotations to metric that could be handled in a
generic way (e.g. by result processors).
- Updating telemetry workload to add classifiers for the url and internal
iteration (or "time") for a particular result.
- Updating csv result processor with the option to use classifiers to
add columns to results.csv (either using all classifiers found, or
only for the specific ones listed).
The key file must only be readable by the owner. If the specified key
file has different access permissions, create a temporary copy with the
right permissions and use that.
np.vectorize was being unconditionally invoked at top level. On an
ImportError, np as set to None, so this was resuling in an
AttributeError when loading the module if one of the dependent libraries
was not present on the host system. This moves the invocation into the
try block with the imports to avoid an error when energy_model module is
loaded by the extension is not used.
- adding missing cpu id conversion to set_cpu_min_freq
- adding "exact" parameter to set_cpu_frequency; only produce an error
when the specified frequency is not supported by the cpu if "axact"
is set; (otherwise let cpufreq decide what the actual frequency will
be).
- adding missing cpu id conversion to set_cpu_min_freq
- adding "exact" parameter to set_cpu_frequency; only produce an error
when the specified frequency is not supported by the cpu if "axact"
is set; (otherwise let cpufreq decide what the actual frequency will
be).
Make sure when migrating sshd to root cgroup also migrate their
children, including the bash for the wa session. So the subsequent
processes kicked off from that shell can be taskset to any cluster.
Matplotlib defautls to the GTK backend. This can cause problems when
running in a headless session (e.g. over SSH). Since energy_model
istrument generates PNG plots, rather than rendering directly to UI, it
doesn't actually need GTK; set backend to AGG so that energy_model works
in headless environments.
This instrument can be used to generate an energy model for a device
based on collected power and performance measurments. The instrument
produces a C file with an energy model and an accompanying HTML report.
This instrument is very different from other instrumentation, as it
actually generates the run queue on the fly based on the operating
frequencies and idle states it discovers on the device. The agenda needs
only to contain the single "base" spec that defines the workload to be
used for performance measurement.
cpuidle module implements cpuidle state discovery, query and
manipulation for a Linux device. This replaces the more primitive
get_cpuidle_states method of LinuxDevice.
Renamed APIs (and added a couple of new ones) to be more consistent:
"core" APIs take a core name as the parameter (e.g. "a15") or whatever
is listed in core_names for that device.
"cluster" APIs take a numeric cluster ID (eg. 0) as the parameter. These
get mapped using core_clusters for that device.
"cpu" APIs take a cpufreq cpu ID as a parameter. These could be
integers, e.g. 0, or full string id, e.g. "cpu0".
integer and boolen are defined in wlauto.utils.types; they perform more
intuitive conversions from other types, particularly strings, so are
more suitable than int and bool for parameters. If, for whatever reason,
native types are in fact desired for a Parameter, this behavior can be
supressed by specifying convert_types=False when defining the parameter.
- list_to_range and range_to_list convert between lists of integers
and corresponding range strings, e.g. between [0,1,2,4] and '0-2,4'
- list_to_mask and mask_to_list convert between lists of integers and
corresponding integer masks, e.g. between [0,1,2,4] and 0x17
Conflicts:
wlauto/utils/misc.py
- added caseless_string type. This behaves exactly like a string, except
this ignores case in comparisons. It does, however, preserve case. E.g.
>>> s = caseless_string('Test')
>>> s == 'test'
True
>>> print s
Test
- added list_of type generating function. This allows to dynamically
generate type-safe list types based on an existing type. E.g.
>>> list_of_bool = list_of(bool)
>>> list_of_bool(['foo', 0, 1, '', True])
[True, False, True, False, True]
- Update core_names Device Parameter to be of type caseless_string
- added the ability to run based on time, rather than number of
requests.
- added a parameter to taskset to specific core(s).
Conflicts:
wlauto/workloads/sysbench/__init__.py
A "please rate me" dialog occasionally pops but when returning to the
initial screen (when re-running the test). The check to dismiss it
wasn't being done at the right time, so it was still preventing
mutli-times execution. This commit resolves that issue.
"times" parameter didn't work properly for version 5 because an extra
back button press was required due to UI changes from previous versions.
This commit adds the button press.
On some systems the temporary directory may still be "busy" after WA is
don't with it. Since it's just an empty dir and it will be reused on the
subsequent runs; do check if rm -rf succeeded.
When the merging logic was updated to preserve duplicates within the
same list, it inadvertently broke the logic that removed items marked
for removal with a '~'. This commit rectifies that.
Note to self: merging functions are doing *way* to much; they should be
refactored into several individual function and config should be
resolved in distinct stages.
If list parameter values contained duplicates, those got removed when
merging parameter values from different sources. This commit fixes that
behavor, so that duplcates that appear within the *same* list are
preserved.
Keep duplicates in lists when merging device_config dict from agenda
with the rest of the config; This will ensure that core_name sand
core_clusters aren't reduced to just unique elements.
- raise an error if an agenda contains duplicate keys (by default PyYAML
will silently ignore this)
- raise an error if config section in an agenda is not dict-like
(before, this was allowed to propagate and relsulted an a traceback
further down the line).
Very old versions of IPython do not have IPython.version_info attribute
that the ultls module relied on. This commit changes it to use the more
standard __version__ attriute that is present in all versions.
check_output will ignore error code 1 returned by telemetry execution,
as this happens when individiual sub-tests and partial results may, and
should, still be extracted.
Adding a parater to wlauto.utils.misc.check_output to specify that it
should ignore certain error codes when they are returned by the
subprocess and not raise them as errors.
Commit 2276ae0c5b ("Fixing config processing for extensions with
non-identifier names.") broke customizing the trace-cmd instrumentation
from the agenda. With an agenda like:
config:
instrumentation: [trace-cmd, delay]
trace_events: ['thermal*']
trace_buffer_size: 28000
trace_events and trace_buffer_size get added to the RunConfiguration's
_raw_config under the trace-cmd name, but then when it's looked up in
_finalize_config_list(), the dictionary is actually looked up using
identifier(extname), i.e. 'trace_cmd'. Fix this by adding the user's
configuration using identifier(name) as well.
run_cell() becomes more complicated when we add ipython version 3
support which upsets pylint because there are "too many
branches (15/12)". Factor out part of the function to make pylint
happy.
The kernel may go idle before it processes the next input, which break
the while=True loop in run_cell() early. Wait for an acknowledgement
of the input we've sent to the kernel before considering an idle
message to mean that the cell has been parsed.
The internal ABI for ipython has changed between ipython version 2 and
3. In its current state, the result processor only works with IPython
version 2, so fail if the user wants to use the result processor with
the wrong version.
Abstract the ipython interface to a file so that we can make it support
versions 2 and 3 at the same time.
The default value for buffer_size_file contained a path under "/d/",
which is an Android-specific alias for "/sys/kernel/debug". This commit
updates the default value to use the system-agnostic path.
Internally, WA expects extension names to be valid Python identifiers.
When this is not the case, conversion takes place when loading
configuration for the extension (e.g. "trace-cmd" gets converted to
"trace_cmd").
The conversion is intended to be transparent to the user, so
configuration stores values as they are provided by the user, however it
needs to perform the conversion internally, e.g. when querying
ExtensionLoader. This conversion was missing when performing a lookup of
one of the internal structures, which was causing earlier-collected
settings to not be propagated into the final config.
Updating SysfsExtractor with a parameter to explicitly enable/disable
tempfs caching. Previously, this was determined entirely by whether the
device is rooted.
This result processor displays a desktop notification when the run
finishes. It's useful when you are running a long agenda in WA and want
to be notified when the results are available.
The param documentation states that for a boolean, "kind" should be
"as_bool" from wlauto.utils.misc, but there is no "as_bool". Currently,
workload automation automatically converts native python types like bool
and int to workload automation specific ones. Remove this bit from the
documentation as it's not true any more.
Change-Id: I0100708530bcf67556eda386c39bc444a3e0f2b2
The error message will now contain the offending extension (either
package name or full path to extension file) if a Python error occurs
when ExtensionLoader attempts to load it.
- connect() to device before issuing the initial reboot, as soft reset
requires a device connection.
- boot() has been implemented to wait properly for the device to reboot
after reset.
- port now defaults to 22 rather than being left unset, as need
something to connect to when polling for device after reboot.
- Only use -P option for scp when port is *not* 22; as that option
appears to cause intermittent issues with default scp on Ubuntu 12.04
On some devices, sudo presents a different prompt when asking for a
password. All the prompt to be specified in device configruation to
handle such cases.
Make the WorkloadError() message clearer if the sysbench
binary cannot be found on the device and host.
Also make trivial fixes to improve descriptions and fix
spelling errors.
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
terminalsize was loaded from a location added to sys.path during
bootstrap. This appeared to be causing import issues. There is no longer
a good reason for terminalsize to be loaded that way, so just moved it
under wlauto.utils so that it can be loaded normally.
Some Linux devices may run on minimal root file systems
(e.g. buildroot) where /usr/local/bin path doesn't exist. Create
the binaries_directory if it doesn't exist instead of letting WA
quit and return errors such as:
INFO Runner: Skipping the rest of the iterations for this spec.
ERROR Runner: Error while running memcpy
ERROR Runner: CalledProcessError("Command 'sshpass
-p '<password>' /usr/bin/scp -r
/usr/local/lib/python2.7/dist-packages/
wlauto-2.3.0-py2.7.egg/wlauto/workloads/memcpy
/memcpy root@192.168.x.x:/usr/local/bin/memcpy'
returned non-zero exit status 1")
ERROR Runner: Got:
ERROR Runner:
ERROR Runner: scp: /usr/local/bin/memcpy:
No such file or directory
Signed-off-by: Lisa Nguyen <lisa.nguyen@linaro.org>
- UEFI config can be specified as a device_config parameter
- The same config is used to create a missing UEFI entry, and
to re-create the entry when flashing. UEFI config now wholy
resides within the device and is not specified for vexpress
flasher.
- Updated sysbench binary to a statically linked verison
- Added missing LICENSE file for the sysbench binary
- Removed Android browser launch and shutdown from workload (now runs on both
Linux and Android)
- Updated update_result to work with the new binary
- Added missing descriptions for parameters
- Added file_test_mode parameter -- this is a mandatory argumet if test is
fileio
- Added cmd_params parameter to pass options directily to sysbench invocation
The struct used to read events is being padded when built for 64
bit platforms. The padding has been made explicit in the struct, and
matching padding was added when writing the events during recording.
On generic_linux devices, one might ssh as the root user, in which
case there is no need to use sudo.
In addition, some root filesystems may not have sudo (e.g. minimal
buildroot/busybox).
This patch attempts to detect the root user using 'id -u' and, if it
detects the root user, avoids the use of sudo for running commands as
well.
Cc: Lisa Nguyen <lisa.nguyen@linaro.org>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
(setrofim: modified to only test once)
Then number of input event files was being written as a size_t but read
as an int by revent. These types have different sizes with 64bit GCC,
causing revent not being able to replay recorded files. This comint
updates revent to use size_t when both reading and writing.
The boot monitor seems to have some buffer overrun issue while loading the latest Linaro android images.
The behavior is after loading the initrd, the kernel loading will fail, very likely to due command line buffer overrun.
the new initrd in the Linaro image is 3 times larger than the previous version, which could cause the issue.
putting a dummy enter between loading initrd and the kernel could resolve the issue.