Improve checking of whether cpu idle state information is available for
processing.
Add debug message to inform user if the cpuidle module is not detected
on the target.
- Add modules entry to TargetInfo
- When retrieving TargetInfo from cache, make sure info modules match
those for the current target, otherwise mark info as stale and
re-generate.
Before exporting output to ensure that we are still connected to the
database. The connection may be dropped so reconnect if necessary, this
is a more of an issue with longer running jobs.
Reflecting the addition to being able to store directories as Artifacts
enable uploading of a directory as a compressed tar file rather than
storing the file directly.
Update the postgres database schema:
- Rename "resourcegetters" schema to "resource_getters" for
consistency
- Rename "retreies" colum to "retry" to better relflect it purpose
- Store additional information including:
- POD serialization data
- Missing target information
- JSON formatted runstate
Previously run level artifacts would be added with a particular job_id,
and updated artifacts would be stored as new objects each time. Refactor
to remove unnecessary instance variables, only provide a job_id when
required and add an update capability for largeobjects to ensure this
does not happen.
In the case of a failure in the initialization of one output_processor the
remaining `initialize` methods may not get called causing variables to
not be initialized correctly.
The Output processor which is used to upload the results found in the
wa_output folder to a Postgres database, whose schema is defined by the
WA Create Database command.
Expose the context to the initialize and finalize functions for Output
Processors. This was found to be necessary for the upcoming PostgreSQL
Output Processor.
- Add copyright headers to files that did not already have them
- Update the year to the last year the file was modified in files with
existing headers.
Add cpustates output processor. This is more-or-less a port of the
cpustates processor from WA, however there are some differences:
- Idle states are now tracked individually per-CPU. This will fix
processing traces from targets that have different number of idle
states on different clusters.
- Simplify the parameter list for report_power_stats:
- Replace paths to individual report files with a path to a single
directory. A subdirectory will be created under it which will
contain all the reports.
- Replace the individual bits bits of information about CPUs (core
names, idle states, etc) with a list of CpuInfo objects.
- Clean up and simplify the code a bit:
- Make all reports mandatory -- the marginal cost of generating
an additional report is minimal compared to tracking power states
in the first place.
- Standardize the interface for Reporters and Reports.
- Rename some of the reports to something a bit more meaningful.
- The stand-alone command line interface is not ported for now, as it
is now possible to run this offline on existing results using
"wa process".
Make sure runtime_params and boot_params are to_pod()'d when they get
written, as they are obj_dict's that json serializer does not know how
to handle otherwise.
This is WA2 sqlite result processor ported to WA3 output processor API.
It is schema-compatible with the WA2 version (in fact, there are *no*
schema changes).
Note: There is a medium-term goal of adding a more generic SQL database
exporter with a more complete schema sitting on top of something like
sqlalchemy to make it agnostic of DBMS implementations. At point, this
will likely be deprecated.
