Run Comparison¶

rubin_sim.maf.run_comparison.bundle_dict_from_disk(result_db, run_name, metric_data_path)[source]¶

Load metric data from disk and import them into metricBundles. via PCW Hackathan 2020 Resources

Parameters:

results_db (dict) – A ResultsDb object
run_name (str) – The name of the opsim database for the metrics in results_db
metric_data_path (str) – The path to the directory where the metric datafiles are stored.

Returns:

bundle_dict – A dictionary of metricBundles reconstructed from the data stored on disk.

Return type:

dict

rubin_sim.maf.run_comparison.combine_result_dbs(run_dirs, dbfilename='resultsDb_sqlite.db')[source]¶

Helper function for gather_summaries

Parameters:

run_dirs (list of str) – A list of directories to search for MAF result databases.
dbfilename (str (resultsDb_sqlite.db)) – The database filename to look for (default: resultsDb_sqlite.db).

rubin_sim.maf.run_comparison.confusion_matrix_plot(comparison_matrix, xlabel, ylabel, run_name, t_e_range, save_folder)[source]¶

Plots a confusion matrix type plot comparing two metric types.

Parameters:

comparison_matrix (np.ndarray, (N,)`) – Array comparing two metric types (A and B) with the following shape: [[(Yes A and Yes B), (Yes A and No B)], [(No A and Yes B), (No A and No B)]] where Yes A and Yes B are the number of events that pass both the A and B criteria.
xlabel (str) – Sring of xlabel (also used in file name of figure)
ylabel (str) – Sring of ylabel (also used in file name of figure)
run_name (str) – Name of the OpSim run that was used in the metric (used in labels and file name)
t_e_range (str) – String of the range of the tE (used in labels and file name)
save_folder (str) – String of folder name to save figures

rubin_sim.maf.run_comparison.create_metric_subset(metric_subset_name, metrics, short_name=None, style='-', invert=False, mag=False)[source]¶

Create a DataFrame that defines a metric subset.

Parameters:

metric_subset_name (str) – The name of the new metric subset.
metrics (list [str]) – A list of metric names in the subset.
short_name (list [str], optional) – A list of shorter metric names, by default None
style (list [str], optional) – The matplotlib line style symbol for lines representing the metric, by default “-”
invert (list [bool], optional) –

Are smaller values of the metric better, such as for errors?,
by default False
mag (list [bool], optional) – Is the metric an astronomical magnitude?, by default False

Returns:

metric_subset – A table of metrics and normalization and plotting flags defining the content of a metric subset.

Return type:

pandas.DataFrame

rubin_sim.maf.run_comparison.describe_families(families, summary=None, table_metric_subset=None, plot_metric_subset=None, baseline_run=None, round_table=2)[source]¶

Display (in a jupyter on IPython notebook) family descirptions

Parameters:

families (pandas.DataFrame) – Data family descriptions as returned by get_family_descriptions.
summary (pandas.DataFrame) – Summary metrics for each run, as returned by get_metric_summaries.
table_metric_subset (pandas.DataFrame) – Metadata on metrics to be included in the table, with columns and index as returned by get_metric_subsets. None if no metrics should be included in the table.
plot_metric_subset (pandas.DataFrame) – Metadata on metrics to be included in the plot, with columns and index as returned by get_metric_subsets. None if no plot should be made.
baseline_run (str) – The name of the run to use to normalize metrics in the plot. None if normalization should be skipped.
round_table (int, opt) – Decimal places to which to round the table_metrics. Default 2.

Returns:

fig (matplotlib.figure.Figure) – The plot figure.
ax (matplotilb.axes.Axes) – The plot axes.

rubin_sim.maf.run_comparison.detected_fisher_comparison(fisher_results, detect_results, fisher_sigmat_e_t_e_cutoff=0.1)[source]¶

Returns an array of the following form where A = fisher criteria and B = detection criteria: [[(Yes A and Yes B), (Yes A and No B)], [(No A and Yes B), (No A and No B)]] where Yes A and Yes B are the number of events that pass both the A and B criteria.

Parameters:

fisher_results (np.ndarray, (N,)) – Array of results from running the Fisher metric of the microlensing metric
detect_results (np.ndarray, (N,)) – Array of results from running the detect metric of the microlensing metric
fisher_sigmat_e_t_e_cutoff (float) – Maximum normalized uncertainty in tE (sigmatE/tE) as determined by 3sigma values of pubished planet microlensing candidates

rubin_sim.maf.run_comparison.detected_npts_comparison(detect_results, npts_results, npts_required=10)[source]¶

Returns an array of the following form where A = detect criteria and B = npts criteria: [[(Yes A and Yes B), (Yes A and No B)], [(No A and Yes B), (No A and No B)]] where Yes A and Yes B are the number of events that pass both the A and B criteria.

Parameters:

detect_results (np.ndarray, (N,)) – Array of results from running the detect metric of the microlensing metric
npts_results (np.ndarray, (N,)) – Array of results from running the Npts metric of the microlensing metric
npts_required (int) – Number of poitns within 2tE required for the number of points fraction.

rubin_sim.maf.run_comparison.download_runs(runs, dest_dir=None, runs_source=None, clobber=False)[source]¶

Download opsim visit databases for specified runs to a local directory.

Parameters:

runs (pandas.DataFrame or iterable [str]) – If a pandas.DataFrame is provided, the OpsimRun column will be used to get run names, and data will be read from the url specified in the url column. If a collection of str is provided, these will be interpreted as run names supplied by data originating in the run metadata provided by the runs_source parameter.
dest_dir (str) – The local directory into which to write downloaded visit databases.
runs_source (str) – File name or URL for the json file from which to load the metadata. If it is set to None, the data is loaded from the URL specified by the archive.RUNS_SOURCE constant. This parameter is ignored if the runs parameter is set to a pandas.DataFrame.
clobber (bool) – If False, runs that would clobber an existing file will be skipped. If True, existing files will be overwritten.

Returns:

runs – Metadata on runs downloaded (in the same structure as the return of archive.get_runs).

Return type:

pandas.DataFrame

rubin_sim.maf.run_comparison.fisher_npts_comparison(fisher_results, npts_results, npts_required=10, fisher_sigmat_e_t_e_cutoff=0.1)[source]¶

Returns an array of the following form where A = fisher criteria and B = npts criteria: [[(Yes A and Yes B), (Yes A and No B)], [(No A and Yes B), (No A and No B)]] where Yes A and Yes B are the number of events that pass both the A and B criteria.

Parameters:

fisher_results (np.ndarray, (N,)) – Array of results from running the Fisher metric of the microlensing metric
npts_results (np.ndarray, (N,)) – Array of results from running the Npts metric of the microlensing metric
npts_required (int) – Number of poitns within 2tE required for the number of points fraction.
fisher_sigmat_e_t_e_cutoff (float) – Maximum normalized uncertainty in tE (sigmatE/tE) as determined by 3sigma values of pubished planet microlensing candidates

rubin_sim.maf.run_comparison.gather_summaries()[source]¶: Find resultsDbs in a series of directories and gather up their summary stats into a single CSV or hdf5 file. Outputs one row per unique run name.

rubin_sim.maf.run_comparison.get_family_descriptions(family_source=None)[source]¶

Get description of families or funs.

Parameters:: family_source (`str) – File name or URL for the json file from which to load the family descriptinos. If it is set to None, the data is loaded from the URL specified by the archive.FAMILY_SOURCE constant.
Returns:: families – Family descriptions, with comments.
Return type:: pandas.DataFrame

rubin_sim.maf.run_comparison.get_family_runs(run_source=None)[source]¶

Load a data frame that supplies run names for each run family

Parameters:

run_source (None or str) – File name or URL for the json file from which to load the metadata. If it is set to None, the data is loaded from the URL specified by the archive.RUNS_SOURCE constant.

Returns:

families –

families: The index is the run family. (str)
run: the project-standard name for the run (str)
OpsimGroup: The name for the group to which the runs belong (str)
OpsimComment: Short description of the run (str)
OpsimVersion: Opsim version name (str)
OpsimDate: Date for the version of opsim (TODO: ?)
brief: A list of descriptions for the run. Runs may have a different description for each family it belongs to, so it a list of the same length as the families column (list [str])
url: The URL from which the opsim output database for this run can be downloaded.

Return type:

pandas.DataFrame

Notes

Because runs can be members of multiple families, more than one row may provide metadata on the same run.

The same content (in a different form) can be obtained using get_runs. get_runs is more convenient when indexing by run; get_family_runs when indexing by family.

rubin_sim.maf.run_comparison.get_metric_subsets(metric_subset_source=None)[source]¶

Get metadata on named subsets of related metrics.

Parameters:

metric_subset_source (str or None) – File name or URL for the json file from which to load the data. If it is set to None, the data is loaded from metric_subsets.json in the $rubin_sim_data.maf directory.

Returns:

metric_subsets –

metric_subset: The 1st level of the index is the name of a subset of metrics (str).
metric: The 2nd level of the index is the full name of the metric (str).
metric: The full name of the metric (str).
short_name: An abbreviated name for the metric (str)..
style: The matplotlib linestyle suggested for plots of the metric (str).
invert: When normalizing, invert the metric value first? (bool)
mag: Is the value an (astronomical) magnitude? (bool)

Return type:

pandas.DataFrame

rubin_sim.maf.run_comparison.get_metric_summaries(run_families=(), metric_subsets=(), runs=(), metrics=(), summary_source=None, runs_source=None, metric_subset_source=None, run_order='family', metric_order='summary')[source]¶

Get summary metric values for a set of runs and metrics.

Parameters:

run_families (iterable [str]) – Families of runs to include in the summary.
metric_subsets (iterable [str]) – subsets of metrics to include in the summary.
runs (iterable [str]) – Runs to include in the summary (in addition to any that are part of families included in run_families).
metrics (iterable [str]) – Metrics to include in the summary (in addition to any that are part of subsets included in metric_subsets).
summary_source (str or pandas.DataFrame) – File name or URL for the file from which to load the data. If the supplied value is a pandas.DataFrame, it the table returned will be a subset of this supplied table.
run_source (pandas.DataFrame or str) – Either a pandas.DataFrame of runs metadata (as returned by archive.get_runs), or a file name or URL for the json file from which to load the run metadata. If it is set to None, the data is loaded from the URL specified by the archive.RUNS_SOURCE constant.
metric_subset_source (pandas.DataFrame or str) – Either a pandas.DataFrame of metric subset specifications (as returned by archive.get_metric_subsets) or a file name for the json file from which to load the data.
run_order (str) – Sort runs according to family definition (“family”) or summary file (“summary”) order.
metric_order (str) – Sort metrics according to subset definition (“subset”) or summary file (“summary”) order.

Returns:

summaries – Metric summary values are returned in a pandas.DataFrame, with each column providing the metrics for one run, and each row the values for one metric. The metric names constitute the index, and the column names are the canonical run names.

Return type:

pandas.DataFrame

Note

The entire summary statistic values for all of the runs and metrics can be downloaded from the default sources first, by simply calling

summary = get_metric_summaries()

Then, you can use get_metric_summaries to get a subset without redownloading the whole set by passing summary_source=summary. If you are selecting multiple subsets of the summary, this avoids needing to download the summary data multiple times.

rubin_sim.maf.run_comparison.get_results(df, run_type, fisher_sigmat_e_t_e_cutoff=0.1)[source]¶

Plots the results from the discovery/detect metric, Npts metric, and Fisher metric in three sub plots

Parameters:

df (pandas.Dataframe) – Pandas dataframe of the results npz file
run_types (np.ndarray, (N,)) – Array of strings describing microlensing metric type: either ‘detect’, ‘Npts’, or ‘Fisher’ as parsed by the file name
fisher_sigmat_e_t_e_cutoff (float) – Maximum normalized uncertainty in tE (sigmatE/tE) as determined by 3sigma values of pubished planet microlensing candidates

rubin_sim.maf.run_comparison.get_results_dbs(result_db_path)[source]¶

Create a dictionary of result_db from result_db files via PCW Hackathan 2020 Resources

Parameters:: result_db_path (str) – Path to the directory storing the result databases generated by MAF.
Returns:: result_dbs – A dictionary containing the ResultDb objects reconstructed from result databases in the provided directory.
Return type:: dict

rubin_sim.maf.run_comparison.get_runs(run_source=None)[source]¶

Load metadata on opsim runs into a pandas.DataFrame.

Parameters:

run_source (None or str) – File name or URL for the json file from which to load the metadata. If it is set to None, the data is loaded from the URL specified by the archive.RUNS_SOURCE constant.

Returns:

runs –

run: The index of the DataFrame is the project-standard name for the run (str)
family: A list of run families to which this run belongs (list [str])
version: The simulation version
brief: A list of descriptions for the run. Runs may have a different description for each family it belongs to, so it a list of the same length as the families column (list [str])
filepath: The file path, relative to a base opsim output directory.
url: The URL from which the opsim output database for this run can be downloaded.

Return type:

pandas.DataFrame

Notes

The same content (in a different form) can be obtained using get_family_runs. get_runs is more convenient when indexing by run; get_family_runs when indexing by family.

rubin_sim.maf.run_comparison.microlensing_fom(save_folder, result_db_path, metric_data_path, figsize=None, figure_name='microlensing_fom')[source]¶

Processes a folder, puts together results for discovery/detect metric, Npts metric, and Fisher metric, and plots them in the four tE bins of 1 - 10 days, 10 - 30 days, 30 - 100 days, and 100 - 1000 days.

Parameters:

result_db_path (str) – Path to the directory storing the result databases generated by MAF.
metric_data_path (str) – Path to the directory storing the npz files generated by MAF.

rubin_sim.maf.run_comparison.normalize_for_radar(summary, norm_run='baseline', invert_cols=None, reverse_cols=None, mag_cols=[])[source]¶

Normalize values in a dataframe to a given run, return output in a dataframe.

This provides a similar functionality as the normalize_metric_summaries method, and returns a similar dataframe. The options for specifying which columns to invert, reverse, or identify as ‘magnitudes’ are slightly different, instead of using a ‘metric_set’.

Parameters:

summary (pandas.DataFrame) – The data frame containing the metric summary stats to normalize (such as from get_metric_summaries). Note that this should contain only the runs and metrics to be normalized – e.g. summary.loc[[list of runs], [list of metrics]] summary should be indexed by the run name.
norm_run (str) – The name of the run to use to define the normalization.
invert_cols (list [str]) – A list of column names that should be inverted (e.g., columns that are uncertainties and are better with a smaller value)
reverse_cols (list [`str]) – Columns to reverse (e.g., magnitudes)
mag_cols (list [str]) – Columns that are in magnitudes

rubin_sim.maf.run_comparison.normalize_metric_summaries(baseline_run, summary, metric_subsets=None)[source]¶

Create a normalized pandas.DataFrame of metric summary values.

Parameters:

baseline_run (str or list of `str) – The name of the run that defines a normalized value of 1. If a list is provided, the median value of each metric across that list is used as the reference.
summary (pandas.DataFrame) – The summary metrics to normalize (as returned by get_metric_summaries)
metric_subsets (pandas.DataFrame) – Metric metadata as returned by archive.get_metric_subsets

Returns:

norm_summary – Metric summary values are returned in a pandas.DataFrame, with each column providing the metrics for one run, and each row the values for one metric. The metric names constitute the columns, and the index names are the canonical run names. Values of 1 indicate metric values that match that of the baseline, differences with with indicate fractional improvement (if > 1) or degradation (if < 1).

Return type:

pandas.DataFrame

rubin_sim.maf.run_comparison.parse_t_e_run_types(name)[source]¶

Parses names of MicrolensingMetric file names

Parameters:: name (str) – A MicrolensingMetric file name

rubin_sim.maf.run_comparison.plot_compare(results, run_names, run_types, min_t_e, max_t_e, save_folder, npts_required=10)[source]¶

Plots confusion matrix type plots comparing fraction detected, characterized (via Fisher), and fraction of events with at least npts_required points within 2 tE

Parameters:

results (np.ndarray, (N,)) – Results from the MicrolensingMetric from get_results() from the respective microlensing metric type
run_names (np.ndarray, (N,)) – Array of names of the OpSim run that was used in the metric
run_types (np.ndarray, (N,)) – Array of strings describing microlensing metric type: either ‘detect’, ‘Npts’, or ‘Fisher’ as parsed by the file name
min_t_e (np.ndarray, (N,)) – Array of values describing the minium einstein crossing time (tE) as parsed by the file name
max_t_e (np.ndarray, (N,)) – Array of values describing the maximum einstein crossing time (tE) as parsed by the file name
save_folder (str) – String of folder name to save figures
npts_required (int) – Number of poitns within 2tE required for the number of points fraction.

rubin_sim.maf.run_comparison.plot_fom(results, run_names, run_types, min_t_e, max_t_e, save_folder, figure_name, figsize)[source]¶

Plots the results from the discovery/detect metric, Npts metric, and Fisher metric in three sub plots

Parameters:

results (np.ndarray, (N,)) – Results from the MicrolensingMetric from get_results() from the respective microlensing metric type
run_names (np.ndarray, (N,)) – Array of names of the OpSim run that was used in the metric
run_types (np.ndarray, (N,)) – Array of strings describing microlensing metric type: either ‘detect’, ‘Npts’, or ‘Fisher’ as parsed by the file name
min_t_e (np.ndarray, (N,)) – Array of values describing the minium einstein crossing time (tE) as parsed by the file name
max_t_e (np.ndarray, (N,)) – Array of values describing the maximum einstein crossing time (tE) as parsed by the file name
save_folder (str) – String of folder name to save figure
figure_name (str) – String of figure name
figsize ((int, int)) – Tuple of figure size in inches. Default is None, which sets figsize = (25, 30)

rubin_sim.maf.run_comparison.plot_run_metric(summary, baseline_run=None, vertical_quantity='run', horizontal_quantity='value', run_label_map=None, metric_label_map=None, metric_subset=None, ax=None, cmap=[[0.188235, 0.635294, 0.854902], [0.988235, 0.309804, 0.188235], [0.898039, 0.682353, 0.219608], [0.427451, 0.564706, 0.309804], [0.545098, 0.545098, 0.545098], [0.090196, 0.745098, 0.811765], [0.580392, 0.403922, 0.741176], [0.839216, 0.152941, 0.156863], [0.121569, 0.466667, 0.705882], [0.890196, 0.466667, 0.760784], [0.54902, 0.337255, 0.294118], [0.737255, 0.741176, 0.133333], [0.227451, 0.003922, 0.513725], [0.0, 0.262745, 0.0], [0.058824, 1.0, 0.662745], [0.368627, 0.0, 0.25098], [0.776471, 0.741176, 1.0], [0.258824, 0.313725, 0.321569], [0.721569, 0.0, 0.501961], [1.0, 0.717647, 0.701961], [0.490196, 0.007843, 0.0], [0.380392, 0.14902, 1.0], [1.0, 1.0, 0.603922], [0.682353, 0.788235, 0.670588], [0.0, 0.52549, 0.486275], [0.333333, 0.227451, 0.0], [0.580392, 0.988235, 1.0], [0.0, 0.74902, 0.0], [0.490196, 0.0, 0.627451], [0.670588, 0.447059, 0.0], [0.568627, 1.0, 0.0], [0.003922, 0.745098, 0.541176], [0.0, 0.270588, 0.482353], [0.784314, 0.509804, 0.435294], [1.0, 0.121569, 0.513725], [0.866667, 0.0, 1.0], [0.019608, 0.454902, 0.0], [0.392157, 0.266667, 0.380392], [0.533333, 0.560784, 1.0], [1.0, 0.713725, 0.956863], [0.32549, 0.384314, 0.215686], [0.807843, 0.521569, 1.0], [0.407843, 0.415686, 0.517647], [0.745098, 0.705882, 0.745098], [0.647059, 0.376471, 0.537255], [0.584314, 0.827451, 1.0], [0.003922, 0.0, 0.972549], [1.0, 0.501961, 0.007843], [0.545098, 0.160784, 0.270588], [0.678431, 0.627451, 0.427451], [0.32549, 0.270588, 0.545098], [0.784314, 1.0, 0.85098], [0.666667, 0.27451, 0.0], [1.0, 0.47451, 0.560784], [0.513725, 0.827451, 0.443137], [0.564706, 0.619608, 0.74902], [0.580392, 0.0, 0.960784], [0.921569, 0.815686, 0.607843], [0.678431, 0.545098, 0.694118], [0.0, 0.388235, 0.290196], [1.0, 0.862745, 0.0], [0.533333, 0.466667, 0.317647], [0.494118, 0.670588, 0.639216], [0.0, 0.0, 0.592157], [0.960784, 0.0, 0.776471], [0.396078, 0.2, 0.160784], [0.0, 0.4, 0.470588], [0.015686, 0.890196, 0.784314], [0.654902, 0.215686, 0.682353], [0.772549, 0.858824, 0.882353], [0.301961, 0.431373, 1.0], [0.607843, 0.576471, 0.003922], [0.803922, 0.345098, 0.419608], [0.937255, 0.870588, 0.996078], [0.47451, 0.352941, 0.0], [0.372549, 0.533333, 0.603922], [0.705882, 1.0, 0.572549], [0.368627, 0.447059, 0.419608], [0.321569, 0.0, 0.4], [0.019608, 0.529412, 0.317647], [0.517647, 0.12549, 0.435294], [0.235294, 0.588235, 0.019608], [0.396078, 0.45098, 0.0], [0.945098, 0.627451, 0.423529], [0.372549, 0.313725, 0.270588], [0.741176, 0.0, 0.290196], [0.815686, 0.407843, 0.152941], [0.843137, 0.588235, 0.670588], [0.537255, 0.364706, 1.0], [0.509804, 0.423529, 0.462745], [0.168627, 0.333333, 0.72549], [0.431373, 0.486275, 0.733333], [0.905882, 0.835294, 0.827451], [0.364706, 0.0, 0.094118], [0.486275, 0.231373, 0.003922], [0.501961, 0.694118, 0.490196], [0.784314, 0.85098, 0.490196], [0.0, 0.909804, 0.231373], [0.486275, 0.698039, 1.0], [1.0, 0.333333, 1.0], [0.643137, 0.152941, 0.129412], [0.113725, 0.894118, 1.0], [0.490196, 0.686275, 0.231373], [0.482353, 0.294118, 0.568627], [0.878431, 1.0, 0.282353], [0.419608, 0.0, 0.768627], [0.803922, 0.658824, 0.592157], [0.745098, 0.388235, 0.768627], [0.537255, 0.803922, 0.807843], [0.27451, 0.011765, 0.784314], [0.368627, 0.572549, 0.47451], [0.254902, 0.290196, 0.003922], [0.019608, 0.654902, 0.615686], [0.811765, 0.54902, 0.215686], [1.0, 0.972549, 0.815686], [0.262745, 0.329412, 0.443137], [0.709804, 0.266667, 1.0], [0.811765, 0.286275, 0.576471], [0.811765, 0.643137, 0.87451], [0.580392, 0.831373, 0.0], [0.654902, 0.580392, 0.854902], [0.176471, 0.647059, 0.345098], [0.552941, 0.890196, 0.713725], [0.643137, 0.662745, 0.615686], [0.423529, 0.360784, 0.717647], [1.0, 0.494118, 0.368627], [0.654902, 0.513725, 0.541176], [0.686275, 0.745098, 0.847059], [0.164706, 0.768627, 1.0], [0.65098, 0.407843, 0.239216], [0.964706, 0.568627, 0.996078], [0.529412, 0.294118, 0.392157], [1.0, 0.047059, 0.294118], [0.129412, 0.368627, 0.137255], [0.258824, 0.572549, 1.0], [0.529412, 0.513725, 0.615686], [0.403922, 0.176471, 0.270588], [0.694118, 0.309804, 0.254902], [0.0, 0.305882, 0.32549], [0.372549, 0.105882, 0.0], [0.678431, 0.254902, 0.403922], [0.313725, 0.196078, 0.403922], [0.839216, 1.0, 0.992157], [0.498039, 0.709804, 0.819608], [0.662745, 0.72549, 0.411765], [1.0, 0.588235, 0.796078], [0.784314, 0.454902, 0.584314], [0.211765, 0.313725, 0.223529], [1.0, 0.815686, 0.388235], [0.368627, 0.345098, 0.384314], [0.529412, 0.580392, 0.462745], [0.662745, 0.470588, 1.0], [0.011765, 0.784314, 0.388235], [0.905882, 0.745098, 0.831373], [0.831373, 0.890196, 0.815686], [0.529412, 0.403922, 0.564706], [0.537255, 0.486275, 0.152941], [0.803922, 0.862745, 1.0], [0.666667, 0.403922, 0.419608], [0.196078, 0.203922, 0.454902], [1.0, 0.368627, 0.662745], [0.0, 0.607843, 0.690196], [0.443137, 1.0, 0.866667], [0.470588, 0.360784, 0.219608], [0.313725, 0.396078, 0.607843], [0.8, 0.0, 0.701961], [0.341176, 0.482353, 0.333333], [0.317647, 0.431373, 0.482353], [0.003922, 0.372549, 0.572549], [0.666667, 0.741176, 0.745098], [0.003922, 0.498039, 0.6], [0.015686, 0.866667, 0.592157], [0.529412, 0.227451, 0.172549], [0.941176, 0.588235, 0.556863], [0.458824, 0.776471, 0.666667], [0.439216, 0.411765, 0.364706], [0.8, 0.862745, 0.035294], [0.686275, 0.521569, 0.341176], [0.847059, 0.0, 0.458824], [0.615686, 0.247059, 0.505882], [0.85098, 0.270588, 0.0], [0.866667, 0.403922, 0.329412], [0.372549, 1.0, 0.47451], [0.835294, 0.694118, 0.45098], [0.384314, 0.14902, 0.368627], [0.729412, 0.635294, 0.239216], [0.85098, 0.94902, 0.701961], [0.341176, 0.007843, 0.560784], [0.631373, 0.607843, 0.666667], [0.301961, 0.290196, 0.152941], [0.643137, 0.662745, 1.0], [0.67451, 0.909804, 0.858824], [0.6, 0.34902, 0.003922], [0.67451, 0.0, 0.886275], [0.278431, 0.509804, 0.184314], [0.796078, 0.764706, 0.678431], [0.0, 0.772549, 0.713725], [0.380392, 0.32549, 0.470588], [0.2, 0.427451, 0.407843], [0.647059, 0.572549, 0.501961], [0.517647, 0.6, 0.635294], [0.992157, 0.341176, 0.392157], [0.439216, 0.588235, 0.823529], [0.447059, 0.552941, 0.027451], [0.498039, 0.0, 0.298039], [0.082353, 0.188235, 0.627451], [0.819608, 0.756863, 0.886275], [0.788235, 0.521569, 0.815686], [0.423529, 0.270588, 0.294118], [0.498039, 0.0, 0.141176], [0.0, 0.635294, 0.47451], [0.698039, 0.662745, 0.811765], [0.976471, 0.0, 0.0], [0.690196, 0.913725, 1.0], [0.576471, 0.619608, 0.313725], [0.447059, 0.478431, 0.509804], [0.85098, 0.180392, 0.333333], [0.278431, 0.380392, 0.003922], [0.0, 0.34902, 1.0], [0.466667, 0.25098, 0.709804], [0.67451, 0.894118, 0.376471], [0.403922, 0.270588, 0.145098], [0.321569, 0.364706, 0.317647], [0.584314, 0.45098, 0.407843], [0.662745, 0.894118, 0.603922], [0.639216, 0.0, 0.345098], [0.85098, 0.384314, 0.964706], [0.556863, 0.490196, 0.811765], [1.0, 0.741176, 0.576471], [0.639216, 0.0, 0.572549], [0.603922, 1.0, 0.72549], [0.654902, 0.760784, 1.0], [0.956863, 0.384314, 0.0], [0.898039, 0.941176, 1.0], [0.721569, 0.611765, 0.643137], [0.376471, 0.588235, 0.580392], [1.0, 0.623529, 0.207843], [0.54902, 0.160784, 0.0], [0.447059, 0.419608, 0.196078], [0.87451, 0.509804, 0.305882], [0.686275, 0.482353, 0.835294], [0.737255, 0.176471, 0.0], [0.482353, 0.435294, 0.639216], [0.282353, 0.262745, 0.384314], [0.780392, 0.639216, 1.0], [0.0, 0.301961, 0.156863], [0.768627, 0.776471, 0.556863], [0.878431, 0.282353, 0.843137], [0.905882, 0.913725, 0.396078], [0.898039, 0.756863, 0.043137], [0.0, 0.956863, 0.945098], [0.623529, 0.356863, 0.635294], [0.298039, 0.254902, 0.717647], [0.396078, 0.2, 0.556863], [0.462745, 0.494118, 0.423529], [0.662745, 0.541176, 0.211765]], linestyles=None, markers=['o'], shade_fraction=0.05)[source]¶

Plot normalized metric values as colored points on a cartesian plane.

Parameters:

summary (pandas.DataFrame) – Values to be plotted. Should only include runs and metrics that should actually appear on the plot.
baseline_run (str) – Name of the run to use as the baseline for normalization (see (archive.normalize_metric_summaries).
vertical_quantity ({'run', 'metric', 'value'}) – Should the run, metric name, or metric value be mapped onto the y axis?
horizontal_quantity ({'run', 'metric', 'value'}) – Should the run, metric name, or metric value be mapped onto the x axis?
vwidth (float) – The width of the plot, in normalized metrics summary units. (The limits of the x axis will be 1-vwidth/2 and 1+width/2).
run_label_map (mapping) – A python mapping between canonical run names and run labels as they should appear on plot labels. Use of this option is discouraged, because it makes it harder to match plots to data. run_label_map could be created by archive.get_runs().loc[these_runs][‘brief’]
metric_label_map (mapping) – A python mapping between canonical metric names and metric labels as they should appear on plot labels. Use this option carefully, because it makes it harder to match plots to metric calculation code.. metric_label_map could be equivalent to metric_subset[‘short_name’]
metric_subset (pandas.DataFrame) – Metric metadata as returned by archive.get_metric_subsets
ax (matplotlib.axes.Axes) – The axes on which to plot the data.
cmap (matplotlib.colors.ColorMap) – The color map to use for point colors.
linestyles (list) – A list of matplotlib linestyles to use to connect the lines
markers (list, opt) – A list of matplotlib markers to use to represent the points
shade_fraction (float, opt) – Add a red/green shading to the plot, starting at 1 +/- shade_fraction. Set to 0 or None for no shading.

Returns:

fig (matplotlib.figure.Figure) – The plot figure.
ax (matplotilb.axes.Axes) – The plot axes.

The run order and metric order (imposed into the summary dataframe passed here as summary) are important and preserved in the plot. These should be set in the (subset) summary dataframe passed here; the metric_subset is available, but used for normalization and plot styling.

rubin_sim.maf.run_comparison.plot_run_metric_mesh(summary, metric_subset=None, baseline_run=None, color_range=1, run_label_map=None, metric_label_map=None, ax=None, cmap=<matplotlib.colors.LinearSegmentedColormap object>)[source]¶

Plot normalized metric values as colored points on a cartesian plane.

Parameters:

summary (pandas.DataFrame) – Values to be plotted. Should only include runs and metrics that should actually appear on the plot.
baseline_run (str) – Name of the run to use as the baseline for normalization (see (archive.normalize_metric_summaries).
color_range (float) – The color range of the plot, in normalized metrics summary units. (The color range will be 1-color_range/2 and 1+color_range/2).
run_label_map (mapping) – A python mapping between canonical run names and run labels as they should appear on plot labels.
metric_label_map (mapping) – A python mapping between canonical metric names and metric labels as they should appear on plot labels.
ax (matplotlib.axes.Axes) – The axes on which to plot the data.
cmap (matplotlib.colors.ColorMap) – The color map to use for point colors. A red/blue diverging color map - CET_D1A_r or CET_D1_r A rainbow diverging color map - CET_R3_r

Returns:

fig (matplotlib.figure.Figure) – The plot figure..
ax (matplotilb.axes.Axes) – The plot axes.

rubin_sim.maf.run_comparison.plot_run_metric_uncert(summary, uncertainty, run_label_map=None, metric_label_map=None, metric_subset=None, cmap=None, linestyles=['-'], markers=['.'], sep_plots=True, ax=None)[source]¶

Plot normalized metric values as colored points on a cartesian plane.

Parameters:

summary (pandas.DataFrame) – Values to be plotted. Should only include runs and metrics that should actually appear on the plot.
uncertainty (pandas.DataFrame) – Uncertainty values to plot on each data point. Should match summary metric columns.
run_label_map (mapping) – A python mapping between canonical run names and run labels as they should appear on plot labels. Use of this option is discouraged, because it makes it harder to match plots to data. run_label_map could be created by archive.get_runs().loc[these_runs][‘brief’]
metric_label_map (mapping) – A python mapping between canonical metric names and metric labels as they should appear on plot labels. Use this option carefully, because it makes it harder to match plots to metric calculation code.. metric_label_map could be equivalent to metric_subset[‘short_name’]
metric_subset (pandas.DataFrame) – Metric metadata as returned by archive.get_metric_subsets
ax (matplotlib.axes.Axes) – The axes on which to plot the data.
cmap (matplotlib.colors.ColorMap) – The color map to use for point colors.
linestyles (list) – A list of matplotlib linestyles to use to connect the lines
markers (list, opt) – A list of matplotlib markers to use to represent the points

Returns:

fig (matplotlib.figure.Figure) – The plot figure.
ax (matplotilb.axes.Axes) – The plot axes.

The run order and metric order (imposed into the summary dataframe passed here as summary) are important and preserved in the plot. These should be set in the (subset) summary dataframe passed here; the metric_subset is available, but used for ‘invert’ and plot styling and alternate labels.

rubin_sim.maf.run_comparison.radar(df, rgrids=[0.7, 1.0, 1.3, 1.6], colors=None, alpha=0.1, legend=True, figsize=(8.5, 5), fill=False, bbox_to_anchor=(1.6, 0.5), legend_font_size=None)[source]¶: make a radar plot!

rubin_sim.maf.run_comparison.write_metric_subsets(metric_subset_file, metric_subsets)[source]¶

Write an updated metric_subset dataframe to disk.

Parameters:

metric_subset_file (str) – Output file name.
metric_subsets (pandas.DataFrame) – Metric_subset dataframe, as defined in get_metric_subsets