Utility: rptimes

The rptimes command is used to ingest runtime information from a series of run.xml files and build an SQLite database of results:

rptimes run*.xml

This will scan through all matching run.xml files. Each file represents one run of a particular tool. Each version of a tool has a corresponding SQLite database; for example, any runs associated with the tool qdot revision 69 will be stored in a file qdot_r69.sql3. The database file is created, if necessary, and information about the job is added to that file.

You can peek into each database with the following command:

rptimes -v qdot_r69.sql3

This will print out some of the more useful fields for all records in the file. You can see, for example, how many jobs have been stored in the file and the CPU time and Wall time for each job.

Each database file contains two tables: "parameters" and "jobs"

The parameters table contains a list of all known input parameters for a particular version of a particular tool. Each parameter is given a short unique name of the form x001, x002, etc., associated with its full Rappture path (e.g., input.number(temp)) and its value type (INTEGER, REAL, or TEXT).

The jobs table contains a record for each job with a unique token string. Each job has a timestamp indicating when it finished, along with its CPU time, Wall time, number of CPUs and execution venue. Each job also has a list of normalized input values for all parameters (x001, x002, etc). This data accumulated from a large number of runs can be used to build a model for execution time, so that we can predict the run time for each new set of input parameters.