Slicers

class rubin_sim.maf.slicers.BaseSlicer(verbose=False, badval=-666)[source]

Bases: object

Base class for all slicers: sets required methods and implements common functionality.

After first construction, the slicer should be ready for setup_slicer which defines slice_points, allowing the slicer to “slice” data and generate plots. After init after a restore: everything necessary for using slicer for plotting or saving/restoring metric data should be present, although the slicer does not need to be able to slice data again and generally will not be able to do so.

Parameters:

  • verbose (bool, optional) – True/False flag to send extra output to screen.

  • badval (int or float, optional) – The value the Slicer uses to fill masked metric data values

__eq__(other_slicer)[source]

Evaluate if two slicers are equivalent.

__iter__()[source]

Iterate over the slices.

__len__()[source]

Return nslice, the number of slice_points in the slicer.

__ne__(other_slicer)[source]

Evaluate if two slicers are not equivalent.

__next__()[source]

Returns results of self._slice_sim_data when iterating over slicer.

Results of self._slice_sim_data should be dictionary of {‘idxs’: the data indexes relevant for this slice of the slicer, ‘slice_point’: the metadata for the slice_point, which always includes ‘sid’ key for ID of slice_point.}

get_slice_points()[source]

Return the slice_point metadata, for all slice points.

output_json(metric_values, metric_name='', sim_data_name='', info_label='', plot_dict=None)[source]

Send metric data to JSON streaming API, along with a little bit of metadata.

This method will only work for metrics where the metricDtype is float or int, as JSON will not interpret more complex data properly. These values can’t be plotted anyway though.

Parameters:

  • metric_values (np.ma.MaskedArray or np.ndarray) – The metric values.

  • metric_name (str, optional) – The name of the metric.

  • sim_data_name (str, optional) – The name of the simulated data source.

  • info_label (str, optional) – Some additional information about this metric and how it was calculated.

  • plot_dict (dict, optional.) – The plot_dict for this metric bundle.

Returns:

io – StringIO object containing a header dictionary with metric_name/metadata/sim_data_name/slicer_name, and plot labels from plot_dict, and metric values/data for plot. if oneDSlicer, the data is [ [bin_left_edge, value], [bin_left_edge, value]..]. if a spatial slicer, the data is [ [lon, lat, value], [lon, lat, value] ..].

Return type:

StringIO

read_backwards_compatible(restored, infilename)[source]

Read pre v1.0 metric files.

read_data(infilename)[source]

Read metric data from disk, along with the info to rebuild the slicer (minus new slicing capability).

Parameters:

infilename (str) – The filename containing the metric data.

Returns:

  • metric_values, slicer, header (np.ma.MaskedArray,)

  • rubin_sim.maf.slicer, dict – MetricValues stored in data file, the slicer basis for those metric values, and a dictionary containing header information (run_name, metadata, etc.).

setup_slicer(sim_data, maps=None)[source]

Set up Slicer for data slicing.

Set up internal parameters necessary for slicer to slice data and generates indexes on sim_data. Also sets _slice_sim_data for a particular slicer.

Parameters:

  • sim_data (np.recarray) – The simulated data to be sliced.

  • maps (list of rubin_sim.maf.maps objects, optional.) – Maps to apply at each slice_point, to add to the slice_point metadata.

write_data(outfilename, metric_values, metric_name='', sim_data_name='', constraint=None, info_label='', plot_dict=None, display_dict=None, summary_values=None)[source]

Save metric values along with the information required to re-build the slicer.

Parameters:

  • outfilename (str) – The output file name.

  • metric_values (np.ma.MaskedArray or np.ndarray) – The metric values to save to disk.

  • metric_name (str) – Name of the metric as configured when run

  • sim_data_name (str) – Name of the simulation metric run on

  • constraint (str) – Constraint used to subselect data

  • info_label (str) – Descriptive additional information

  • plot_dict (dict) – Dictionary of plotting parameters

  • display_dict (dict) – Dictionary of display parameters, including caption

  • summary_values (dict) – Dictionary of summary statistics

class rubin_sim.maf.slicers.BaseSpatialSlicer(lon_col='fieldRA', lat_col='fieldDec', rot_sky_pos_col_name='rotSkyPos', lat_lon_deg=True, verbose=True, badval=-666, leafsize=100, radius=2.45, use_camera=True, camera_footprint_file=None)[source]

Bases: BaseSlicer

Base spatial slicer object, contains additional functionality for spatial slicing, including setting up and traversing a kdtree containing the simulated data points.

Parameters:

  • lon_col (str, optional) – Name of the longitude (RA equivalent) column.

  • lat_col (str, optional) – Name of the latitude (Dec equivalent) column.

  • rot_sky_pos_col_name (str, optional) – Name of the rotSkyPos column in the input data. Only used if use_camera is True. Describes the orientation of the camera orientation on the sky.

  • lat_lon_deg (bool, optional) – Flag indicating whether lat and lon values from input data are in degrees (True) or radians (False).

  • verbose (bool, optional) – Verbosity flag - noisy or quiet.

  • badval (float, optional) – Bad value flag, relevant for plotting.

  • leafsize (int, optional) – Leafsize value for kdtree.

  • radius (float, optional) – Radius for matching in the kdtree. Equivalent to the radius of the FOV, in degrees.

  • use_camera (bool, optional) – Flag to indicate whether to use the LSST camera footprint or not.

  • camera_footprint_file (str, optional) – Name of the camera footprint map to use. Can be None, which will use the default file.

setup_slicer(sim_data, maps=None)[source]

Use sim_data[self.lon_col] and sim_data[self.lat_col] (in radians) to set up KDTree.

Parameters:

  • sim_data (numpy.ndarray) – The simulated data, including the location of each pointing.

  • maps (list of rubin_sim.maf.maps objects, optional) – List of maps (such as dust extinction) that will run to build up additional data at each slice_point. This additional data is available to metrics via the slice_point dictionary.

class rubin_sim.maf.slicers.BlockIntervalSlicer(mjd_column_name='observationStartMJD', duration_column_name='visitTime', note_column_name='note', badval=nan, verbose=False)[source]

Bases: TimeIntervalSlicer

Slices into intervals with common “note” values and no long gaps.

Parameters:

  • mjd_column_name (str) – Name of column on which to slice, must be in units of days

  • duration_column_name (str) – Name of column with the duration of each visit (in seconds)

  • note_column_name (str) – Name of column with the visit note.

  • badval (float) – Value to use for bad values in slice

  • verbose (bool) – Print extra information?

setup_slicer(sim_data, maps=None)[source]

Set up Slicer for data slicing.

Set up internal parameters necessary for slicer to slice data and generates indexes on sim_data. Also sets _slice_sim_data for a particular slicer.

Parameters:

  • sim_data (np.recarray) – The simulated data to be sliced.

  • maps (list of rubin_sim.maf.maps objects, optional.) – Maps to apply at each slice_point, to add to the slice_point metadata.

class rubin_sim.maf.slicers.HealpixComCamSlicer(nside=128, lon_col='fieldRA', lat_col='fieldDec', lat_lon_deg=True, verbose=True, badval=nan, use_cache=True, leafsize=100, use_camera=False, rot_sky_pos_col_name='rotSkyPos', mjdColName='observationStartMJD', chipNames=['R:2,2 S:0,0', 'R:2,2 S:0,1', 'R:2,2 S:0,2', 'R:2,2 S:1,0', 'R:2,2 S:1,1', 'R:2,2 S:1,2', 'R:2,2 S:2,0', 'R:2,2 S:2,1', 'R:2,2 S:2,2'], side_length=0.7)[source]

Bases: HealpixSlicer

Slicer that uses the ComCam footprint to decide if observations overlap a healpixel center

setup_slicer(sim_data, maps=None)[source]

Use sim_data[self.lon_col] and sim_data[self.lat_col] (in radians) to set up KDTree.

Parameters:

  • sim_data (numpy.recarray) – The simulated data, including the location of each pointing.

  • maps (list of rubin_sim.maf.maps objects, optional) – List of maps (such as dust extinction) that will run to build up additional metadata at each slice_point. This additional metadata is available to metrics via the slice_point dictionary. Default None.

class rubin_sim.maf.slicers.HealpixSDSSSlicer(nside=128, lon_col='RA1', lat_col='Dec1', verbose=True, use_cache=True, radius=0.2833333333333333, leafsize=100, **kwargs)[source]

Bases: HealpixSlicer

For use with SDSS stripe 82 square images

setup_slicer(sim_data, maps=None)[source]

Use sim_data[self.lon_col] and sim_data[self.lat_col] (in radians) to set up KDTree.

class rubin_sim.maf.slicers.HealpixSlicer(nside=128, lon_col='fieldRA', lat_col='fieldDec', lat_lon_deg=True, verbose=True, badval=nan, use_cache=True, leafsize=100, radius=2.45, use_camera=True, camera_footprint_file=None, rot_sky_pos_col_name='rotSkyPos')[source]

Bases: BaseSpatialSlicer

A spatial slicer that evaluates pointings on a healpix-based grid.

Note that a healpix slicer is intended to evaluate the sky on a spatial grid. Usually this grid will be something like RA as the lon_col and Dec as the lat_col. However, it could also be altitude and azimuth, in which case altitude as lat_col, and azimuth as lon_col. An additional alternative is to use HA/Dec, with lon_col=HA, lat_col=Dec.

When plotting with RA/Dec, the default HealpixSkyMap can be used, corresponding to {‘rot’: (0, 0, 0), ‘flip’: ‘astro’}. When plotting with alt/az, either the LambertSkyMap can be used (if Basemap is installed) or the HealpixSkyMap can be used with a modified plot_dict, {‘rot’: (90, 90, 90), ‘flip’: ‘geo’}. When plotting with HA/Dec, only the HealpixSkyMap can be used, with a modified plot_dict of {‘rot’: (0, -30, 0), ‘flip’: ‘geo’}.

Parameters:

  • nside (int, optional) – The nside parameter of the healpix grid. Must be a power of 2. Default 128.

  • lon_col (str, optional) – Name of the longitude (RA equivalent) column to use from the input data. Default fieldRA

  • lat_col (str, optional) – Name of the latitude (Dec equivalent) column to use from the input data. Default fieldDec

  • lat_lon_deg (bool, optional) – Flag indicating whether the lat and lon values in the input data are in degrees (True) or radians (False). Default True.

  • verbose (bool, optional) – Flag to indicate whether or not to write additional information to stdout during runtime. Default True.

  • badval (float, optional) – Bad value flag, relevant for plotting. Default the np.nan value (in order to properly flag bad data points for plotting with the healpix plotting routines). This should not be changed.

  • use_cache (bool, optional) – Flag allowing the user to indicate whether or not to cache (and reuse) metric results calculated with the same set of simulated data pointings. This can be safely set to True for slicers not using maps and will result in increased speed. When calculating metric results using maps, the metadata at each individual ra/dec point may influence the metric results and so use_cache should be set to False. Default True.

  • leafsize (int, optional) – Leafsize value for kdtree. Default 100.

  • radius (float, optional) – Radius for matching in the kdtree. Equivalent to the radius of the FOV. Degrees. Default 2.45.

  • use_camera (bool, optional) – Flag to indicate whether to use the LSST camera footprint or not. Default True.

  • camera_footprint_file (str, optional) – Name of the camera footprint map to use. Can be None, which will use the default.

  • rot_sky_pos_col_name (str, optional) – Name of the rotSkyPos column in the input data. Only used if use_camera is True. Describes the orientation of the camera orientation compared to the sky. Default rotSkyPos.

__eq__(other_slicer)[source]

Evaluate if two slicers are equivalent.

class rubin_sim.maf.slicers.HealpixSubsetSlicer(nside, hpid, lon_col='fieldRA', lat_col='fieldDec', lat_lon_deg=True, verbose=True, badval=nan, use_cache=True, leafsize=100, radius=2.45, use_camera=True, camera_footprint_file=None, rot_sky_pos_col_name='rotSkyPos')[source]

Bases: HealpixSlicer

A spatial slicer that evaluates pointings on a subset of a healpix-based grid. The advantage of using this healpixSubsetSlicer (rather than just putting the RA/Dec values into the UserPointsSlicer, which is another valid approach) is that you preserve the full healpix array. This means you could do things like calculate the power spectrum and plot without remapping into healpixels first. The downside is that you must first (externally) define the healpixels that you wish to use - the rubin_sim.featureScheduler.footprints is a useful add-on here.

When plotting with RA/Dec, the default HealpixSkyMap can be used, corresponding to {‘rot’: (0, 0, 0), ‘flip’: ‘astro’}.

Parameters:

  • nside (int) – The nside parameter of the healpix grid. Must be a power of 2.

  • hpid (np.ndarray) – The subset of healpix id’s to use to calculate the metric. Because the hpid should be defined based on a particular nside, these first two arguments are not optional for this slicer.

  • lon_col (str, optional) – Name of the longitude (RA equivalent) column to use from the input data. Default fieldRA

  • lat_col (str, optional) – Name of the latitude (Dec equivalent) column to use from the input data. Default fieldDec

  • lat_lon_deg (bool, optional) – Flag indicating whether the lat and lon values in the input data are in degrees (True) or radians (False). Default True.

  • verbose (bool, optional) – Flag to indicate whether or not to write additional information to stdout during runtime. Default True.

  • badval (float, optional) – Bad value flag, relevant for plotting. Default the np.nan value (in order to properly flag bad data points for plotting with the healpix plotting routines). This should not be changed.

  • use_cache (bool, optional) – Flag allowing the user to indicate whether or not to cache (and reuse) metric results calculated with the same set of simulated data pointings. This can be safely set to True for slicers not using maps and will result in increased speed. When calculating metric results using maps, the metadata at each individual ra/dec point may influence the metric results and so use_cache should be set to False. Default True.

  • leafsize (int, optional) – Leafsize value for kdtree. Default 100.

  • radius (float, optional) – Radius for matching in the kdtree. Equivalent to the radius of the FOV. Degrees. Default 1.75.

  • use_camera (bool, optional) – Flag to indicate whether to use the LSST camera footprint or not. Default False.

  • camera_footprint_file (str, optional) – Name of the camera footprint map to use. Can be None, which will use the default.

  • rot_sky_pos_col_name (str, optional) – Name of the rotSkyPos column in the input data. Only used if use_camera is True. Describes the orientation of the camera orientation compared to the sky. Default rotSkyPos.

__eq__(other_slicer)[source]

Evaluate if two slicers are equivalent.

__iter__()[source]

Iterate over the slices.

__next__()[source]

Returns results of self._slice_sim_data when iterating over slicer.

Results of self._slice_sim_data should be dictionary of {‘idxs’: the data indexes relevant for this slice of the slicer, ‘slice_point’: the metadata for the slice_point, which always includes ‘sid’ key for ID of slice_point.}

setup_slicer(sim_data, maps=None)[source]

Use sim_data[self.lon_col] and sim_data[self.lat_col] (in radians) to set up KDTree.

Parameters:

  • sim_data (numpy.recarray) – The simulated data, including the location of each pointing.

  • maps (list of rubin_sim.maf.maps objects, optional) – List of maps (such as dust extinction) that will run to build up additional metadata at each slice_point. This additional metadata is available to metrics via the slice_point dictionary. Default None.

class rubin_sim.maf.slicers.HourglassSlicer(verbose=True, badval=-666)[source]

Bases: UniSlicer

Slicer to make the filter hourglass plots

read_metric_data(infilename)[source]

Override base read method to ‘pass’: we don’t save or read hourglass metric data.

The data volume is too large.

write_data(outfilename, metric_values, metric_name='', **kwargs)[source]

Override base write method: we don’t want to save hourglass metric data.

The data volume is too large.

class rubin_sim.maf.slicers.MoObjSlicer(h_range=None, verbose=True, badval=0)[source]

Bases: BaseSlicer

Slice moving object _observations_, per object and optionally clone/per H value.

Iteration over the MoObjSlicer will go as: * iterate over each orbit; * if Hrange is not None, for each orbit, iterate over Hrange.

Parameters:

h_range (numpy.ndarray or None) – The H values to clone the orbital parameters over. If Hrange is None, will not clone orbits.

__eq__(other_slicer)[source]

Evaluate if two slicers are equal.

__iter__()[source]

Iterate through each of the ssoIds.

__next__()[source]

Returns result of self._getObs when iterating over moSlicer.

read_obs(obs_file)[source]

Read observations of the solar system objects (such as created by sims_movingObjects).

Parameters:

obs_file (str) – The file containing the observation information.

setup_slicer(orbit_file, delim=None, skiprows=None, obs_file=None)[source]

Set up the slicer and read orbit_file and obs_file from disk.

Sets self.orbits (with orbit parameters), self.all_obs, and self.obs self.orbit_file and self.obs_file

Parameters:

  • orbit_file (str) – The file containing the orbit information. This is necessary, in order to be able to generate plots.

  • obs_file (str, optional) – The file containing the observations of each object, optional. If not provided (default, None), then the slicer will not be able to ‘slice’, but can still plot.

subset_obs(pandas_constraint=None)[source]

Choose a subset of all the observations, such as those in a particular time period.

class rubin_sim.maf.slicers.MovieSlicer(slice_col_name=None, slice_col_units=None, bins=None, bin_min=None, bin_max=None, bin_size=None, verbose=True, badval=0, cumulative=True, force_no_ffmpeg=False)[source]

Bases: BaseSlicer

__eq__(other_slicer)[source]

Evaluate if slicers are equivalent.

make_movie(outfileroot, sliceformat, plot_type, fig_format, out_dir='Output', ips=10.0, fps=10.0)[source]

Takes in metric and slicer metadata and calls ffmpeg to stitch together output files.

setup_slicer(sim_data, maps=None)[source]

Set up bins in slicer.

class rubin_sim.maf.slicers.NDSlicer(slice_col_list=None, bins_list=100, verbose=True)[source]

Bases: BaseSlicer

Nd slicer (N dimensions)

Parameters:

  • slice_col_list (list of str) – Names of the data columns for slicing (e.g. ‘airmass`, etc)

  • bins_list (int or list of int or np.ndarray, opt) – Single integer (for same number of slices in each dimension) or a list of integers (matching slice_col_list) or list of arrays. Default 100, in all dimensions.

  • ([a (All bins are half-open)

  • b)).

__eq__(other_slicer)[source]

Evaluate if grids are equivalent.

setup_slicer(sim_data, maps=None)[source]

Set up bins.

class rubin_sim.maf.slicers.OneDSlicer(slice_col_name=None, slice_col_units=None, bins=None, bin_min=None, bin_max=None, bin_size=None, verbose=True, badval=0)[source]

Bases: BaseSlicer

OneD Slicer allows the ‘slicing’ of data into bins in a single dimension.

Parameters:

  • slice_col_name (str) – The name of the data column to base slicing on (i.e. ‘airmass’, etc.)

  • slice_col_units (str, optional) – Set a name for the units of the sliceCol. Used for plotting labels. Default None.

  • bins (np.ndarray, optional) – The data will be sliced into ‘bins’: this can be defined as an array here. Default None.

  • bin_min (float, optional)

  • bin_max (float, optional)

  • bin_size (float, optional) – If bins is not defined, then bin_min/bin_max/bin_size can be chosen to anchor the slice points. Default None. Priority goes: bins >> bin_min/bin_max/bin_size >> data values (if none of the above are chosen).

  • ([a (b)) while the last bin is)

  • ([a

  • b]).

__eq__(other_slicer)[source]

Evaluate if slicers are equivalent.

setup_slicer(sim_data, maps=None)[source]

Set up bins in slicer. This happens AFTER sim_data is defined, thus typically in the MetricBundleGroup. This maps data into the bins; it’s not a good idea to reuse a OneDSlicer as a result.

exception rubin_sim.maf.slicers.SlicerNotSetupError[source]

Bases: Exception

Thrown when a slicer is not setup for the method called.

class rubin_sim.maf.slicers.SlicerRegistry(name, bases, dict)[source]

Bases: type

Meta class for slicers, to build a registry of slicer classes.

class rubin_sim.maf.slicers.TimeIntervalSlicer(interval_seconds=90, mjd_column_name='observationStartMJD', badval=nan, verbose=False)[source]

Bases: BaseSlicer

Base for all time interval slicers.

Slices in constant time intervals.

Parameters:

  • interval_seconds (int) – Duration of slice time intervals, in seconds

  • mjd_column_name (str) – Name of column on which to slice, must be in units of days

  • badval (float) – Value to use for bad values in slice

  • verbose (bool) – Print extra information?

__eq__(other_slicer)[source]

Evaluate if slicers are equivalent.

setup_slicer(sim_data, maps=None)[source]

Set up Slicer for data slicing.

Set up internal parameters necessary for slicer to slice data and generates indexes on sim_data. Also sets _slice_sim_data for a particular slicer.

Parameters:

  • sim_data (np.recarray) – The simulated data to be sliced.

  • maps (list of rubin_sim.maf.maps objects, optional.) – Maps to apply at each slice_point, to add to the slice_point metadata.

class rubin_sim.maf.slicers.UniSlicer(verbose=True, badval=-666)[source]

Bases: BaseSlicer

UniSlicer.

__eq__(other_slicer)[source]

Evaluate if slicers are equivalent.

setup_slicer(sim_data, maps=None)[source]

Use sim_data to set indexes to return.

class rubin_sim.maf.slicers.UserPointsSlicer(ra, dec, lon_col='fieldRA', lat_col='fieldDec', lat_lon_deg=True, verbose=True, badval=-666, leafsize=100, radius=2.45, use_camera=True, camera_footprint_file=None, rot_sky_pos_col_name='rotSkyPos')[source]

Bases: BaseSpatialSlicer

A spatial slicer based on a user-provided list of points. The data_slices returned are the visit pointings which overlap each of these points.

Parameters:

  • ra (list or numpy.ndarray) – User-selected RA points, in degrees. Stored internally in radians.

  • dec (list or numpy.ndarray) – User-selected Dec points, in degrees. Stored internally in radians.

  • lon_col (str, optional) – Name of the longitude (RA equivalent) column in the input data. Default fieldRA

  • lat_col (str, optional) – Name of the latitude (Dec equivalent) column in the input data. Default fieldDec

  • latLonDeg (bool, optional) – Flag indicating whether the lon and lat values will be in degrees (True) or radians (False). Default True.

  • verbose (bool, optional) – Flag to indicate whether or not to write additional information to stdout during runtime. Default True.

  • badval (float, optional) – Bad value flag, relevant for plotting. Default -666.

  • leafsize (int, optional) – Leafsize value for kdtree. Default 100.

  • radius (float, optional) – Radius for matching in the kdtree. Equivalent to the radius of the FOV. Degrees. Default 2.45.

  • use_camera (bool, optional) – Flag to indicate whether to use the LSST camera footprint or not. Default True.

  • camera_footprint_file (str, optional) – Name of the camera footprint map to use. Can be None, which will use the default.

  • rotSkyPosColName (str, optional) – Name of the rotSkyPos column in the input data. Only used if use_camera is True. Describes the camera orientation compared to the sky. Default rotSkyPos.

__eq__(other_slicer)[source]

Evaluate if two slicers are equivalent.

class rubin_sim.maf.slicers.VisitIntervalSlicer(mjd_column_name='observationStartMJD', duration_column_name='visitTime', extra_column_names=(), badval=nan, verbose=False)[source]

Bases: TimeIntervalSlicer

Slices into intervals each of which contain one visit

Parameters:

  • mjd_column_name (str) – Name of column on which to slice, must be in units of days

  • duration_column_name (str) – Name of column with the duration of each visit (in seconds)

  • badval (float) – Value to use for bad values in slice

  • verbose (bool) – Print extra information?

setup_slicer(sim_data, maps=None)[source]

Set up Slicer for data slicing.

Set up internal parameters necessary for slicer to slice data and generates indexes on sim_data. Also sets _slice_sim_data for a particular slicer.

Parameters:

  • sim_data (np.recarray) – The simulated data to be sliced.

  • maps (list of rubin_sim.maf.maps objects, optional.) – Maps to apply at each slice_point, to add to the slice_point metadata.