ResultsDb

class rubin_sim.maf.db.ResultsDb(out_dir=None, database=None, verbose=False)

Bases: object

The ResultsDb is a sqlite database containing information on the metrics run via MAF, the plots created, the display information (such as captions), and any summary statistics output.

Methods Summary

build_summary_name(metric_name, ...[, ...])	Standardize a complete summary metric name, combining the metric + slicer + summary + info_label
close()	Close connection to database.
downgrade_database()	Downgrade resultsDb to work with v0.10<MAF< v1.0 There is also a 'upgradeDatabase' to update to the newer style with 'metric_info_label.
get_all_metric_ids()	Return a list of all metric_ids.
get_metric_data_files([metric_id])	Get the metric data filenames for all or a single metric.
get_metric_display_info([metric_id])	Get the contents of the metrics and displays table, together with the 'basemetric_name' (optionally, for metric_id list).
get_metric_id(metric_name[, slicer_name, ...])	Find metric bundle Ids from the metric table.
get_metric_id_like([metric_name_like, ...])	Find metric bundle Ids from the metric table, but search for names 'like' the values.
get_metric_info([metric_id, with_sim_name])	Get the simple metric info, without display information.
get_plot_files([metric_id, with_sim_name])	Return the metric_id, name, info_label, and all plot info (optionally for metric_id list).
get_run_name()	Return a list of the run_names for the metric bundles in the database.
get_summary_stats([metric_id, summary_name, ...])	Get the summary stats (optionally for metric_id list).
open()	Open connection to database
update_database()	Update the results_db from 'metricMetaData' to 'metric_info_label' and now also changing the camel case to snake case (metricId to metric_id, etc.).
update_display(metric_id, display_dict[, ...])	Add a row to or update a row in the displays table.
update_metric(metric_name, slicer_name, ...)	Add a row to or update a row in the metrics table.
update_plot(metric_id, plot_type, plot_file)	Add a row to or update a row in the plot table.
update_summary_stat(metric_id, summary_name, ...)	Add a row to or update a row in the summary statistic table.

Methods Documentation

static build_summary_name(metric_name, metric_info_label, slicer_name, summary_stat_name=None)

Standardize a complete summary metric name, combining the metric + slicer + summary + info_label

close()

Close connection to database.

downgrade_database()

Downgrade resultsDb to work with v0.10<MAF< v1.0 There is also a ‘upgradeDatabase’ to update to the newer style with ‘metric_info_label.

get_all_metric_ids()

Return a list of all metric_ids.

get_metric_data_files(metric_id=None)

Get the metric data filenames for all or a single metric. Returns a list.

get_metric_display_info(metric_id=None)

Get the contents of the metrics and displays table, together with the ‘basemetric_name’ (optionally, for metric_id list). Returns a numpy array of the metric information + display information.

One underlying assumption here is that all metrics have some display info. In newer batches, this may not be the case, as the display info gets auto-generated when the metric is plotted.

get_metric_id(metric_name, slicer_name=None, metric_info_label=None, run_name=None)

Find metric bundle Ids from the metric table.

Parameters:

metric_namestr

Name of the Metric

slicer_namestr, opt

Name of the Slicer to match

metric_info_labelstr, opt

Metadata value to match

run_namestr, opt

Name of the simulation (run_name) to match

Returns:

metric_idlist of int

List of matching metric_ids

get_metric_id_like(metric_name_like=None, slicer_name_like=None, metric_info_label_like=None, run_name=None)

Find metric bundle Ids from the metric table, but search for names ‘like’ the values. (instead of a strict match from get_metric_id).

Parameters:

metric_namestr

Partial name of the Metric

slicer_namestr, opt

Partial name of the Slicer to match

metric_info_labelstr, opt

Partial info_label value to match

run_namestr, opt

Name of the simulation (run_name) to match (exact)

Returns:

metric_idlist of int

List of matching metric_ids

get_metric_info(metric_id=None, with_sim_name=False)

Get the simple metric info, without display information.

Parameters:

metric_idint list, or None

If None, data for all metrics are returned. Otherwise, only data corresponding to the supplied metric ID or IDs are returned

with_sim_namebool

If True, include the run name in the fields returned

Returns:

plotFilesnumpy.recarray

metric_id

The metric ID

metric_name

The metric name

basemetric_names

The base metric names

slicer_name

The name of the slicer used in the bundleGroup

sql_constraint

The full sql constraint used in the bundleGroup

metric_info_label

Metadata extracted from the sql_constraint (usually the filter)

metric_datafile

The file name of the file with the metric data itself.

run_name

The name of the run plotted (if with_sim_name was True)

get_plot_files(metric_id=None, with_sim_name=False)

Return the metric_id, name, info_label, and all plot info (optionally for metric_id list). Returns a numpy array of the metric information + plot file names.

Parameters:

metric_idint list, or None

If None, plots for all metrics are returned. Otherwise, only plots corresponding to the supplied metric ID or IDs are returned

with_sim_namebool

If True, include the run name in the fields returned

Returns:

plotFilesnumpy.recarray

metric_id

The metric ID

metric_name

The metric name

metric_info_label

info_label extracted from the sql constraint (usually the filter)

plot_type

The plot type

plot_file

The full plot file (pdf by default)

thumb_file

A plot thumbnail file name (png)

run_name

The name of the run plotted (if with_sim_name was True)

get_run_name()

Return a list of the run_names for the metric bundles in the database.

get_summary_stats(metric_id=None, summary_name=None, summary_name_like=None, summary_name_notlike=None, with_sim_name=False)

Get the summary stats (optionally for metric_id list). Optionally, also specify the summary metric name. Returns a numpy array of the metric information + summary statistic information.

Parameters:

metric_idint or list of int

Metric bundle Ids to match from the metric table

summary_namestr, opt

Match this summary statistic name exactly.

summary_name_likestr, opt

Partial match to this summary statistic name.

summary_name_notlikestr, opt

Exclude summary statistics with summary names like this.

with_sim_namebool, opt

If True, add the run_name to the returned numpy recarray.

Returns:

summarystatsnp.recarray

Numpy recarray containing the selected summary statistic information.

open()

Open connection to database

update_database()

Update the results_db from ‘metricMetaData’ to ‘metric_info_label’ and now also changing the camel case to snake case (metricId to metric_id, etc.).

This updates results_db to work with the current version of MAF, including RunComparison and showMaf. There is also a ‘downgrade_database’ to revert to the older style with ‘metricMetadata.

update_display(metric_id, display_dict, overwrite=True)

Add a row to or update a row in the displays table.

Parameters:

metric_idint

The metricID for this metric bundle in the metrics table

display_dictdict

Dictionary containing the display info (group/subgroup/order/caption)

overwritebool, opt

Replaces existing row with same metric_id if overwrite is True (default=True).

update_metric(metric_name, slicer_name, run_name, sql_constraint, metric_info_label, metric_datafile)

Add a row to or update a row in the metrics table.

Parameters:

metric_namestr

Name of the Metric

slicer_namestr

Name of the Slicer

run_namestr

Name of the simulation (run_name, simName, run_name..)

sql_constraintstr

sql_constraint relevant for the metric bundle

metric_info_labelstr

Information associated with the metric. Could be derived from the sql_constraint or could be a more descriptive version, specified by the user.

metric_datafilestr

The data file the metric bundle output is stored in.

Returns:

metric_idint

The Id number of this metric in the metrics table.

If same metric (same metric_name, slicer_name, run_name, sql_constraint, infoLabel)

already exists, it does nothing.

update_plot(metric_id, plot_type, plot_file, overwrite=False)

Add a row to or update a row in the plot table.

Parameters:

metric_idint

The metric Id of this metric bundle in the metrics table

plot_typestr

The type of this plot (oneDbinned data, healpix map, etc.)

plot_filestr

The filename for this plot

overwritebool

Replaces existing row with the same metric_id and plot_type, if True. Default False, in which case additional plot is added to output (e.g. with different range)

update_summary_stat(metric_id, summary_name, summary_value, ntry=3, pause_time=100)

Add a row to or update a row in the summary statistic table.

Most summary statistics will be a simple name (string) + value (float) pair. For special summary statistics which must return multiple values, the base name can be provided as ‘name’, together with a np recarray as ‘value’, where the recarray also has ‘name’ and ‘value’ columns (and each name/value pair is then saved as a summary statistic associated with this same metric_id).

Parameters:

metric_idint

The metric Id of this metric bundle

summary_namestr

The name of this summary statistic

summary_value:float or numpy.ndarray

The value for this summary statistic. If this is a numpy recarray, then it should also have ‘name’ and ‘value’ columns to save each value to rows in the summary statistic table.

ntryint, opt

The number of times to retry if database is locked. Default 3 times.

pause_timeint, opt

Time to wait until trying again. Default 100s.