deepSIP.utils¶
-
deepSIP.utils.
load_txt_spectrum
(filename)¶ load spectrum from txt file with first two columns as wave flux
Parameters: - filename : str
name of text file to read spectrum from
Returns: - wave : np.array
wavelength grid
- flux : np.array
fluxes on wavelength grid
-
deepSIP.utils.
savenet
(network, filename)¶ save network parameters
Parameters: - network : DropoutCNN or torch network
network to save
- filename : str
name of file to save network to
-
deepSIP.utils.
loadnet
(network, filename, device='cpu')¶ load network parameters
Parameters: - network : DropoutCNN
network to load weights and biases into
- filename : str
name of file to load network from
- device : str, optional
device to load network onto (‘cpu’ by default)
-
deepSIP.utils.
deredshift
(wave, z)¶ transform wavelength to rest frame
Parameters: - wave : np.array
wavelength grid
- z : float
redshift of SN
Returns: - np.array
rest-frame wavelength grid
-
deepSIP.utils.
smooth
(flux, window, order)¶ smooth using a savitzky-golay filter
Parameters: - flux : np.array
fluxes
- window : int
window size in angstroms for smoothing
- smoothing : int
polynomial order for smoothing
Returns: - np.array
smoothed spectrum
-
deepSIP.utils.
get_continuum
(flux, window, order)¶ model continuum from heavily smoothed spectrum
Notes
see documentiation for smooth, which this function simply wraps
-
deepSIP.utils.
log_wave
(bounds, n_bins)¶ compute logarithmic wavelength grid
Parameters: - bounds : tuple, list, or other iterable of length 2
lower and upper limit of logarithmic wavelength array
- n_bins : int
number of bins in logarithmic wavelength grid
Returns: - np.array
logarithmic wavelength grid
-
deepSIP.utils.
log_bin
(wave, flux, bounds, n_bins)¶ rebin onto logarithmic wavelength grid (adapted from astrodash/SNID)
Parameters: - wave : np.array
wavelength grid
- flux : np.array
fluxes on wavelength grid
- bounds : tuple, list, or other iterable of length 2
lower and upper limit of logarithmic wavelength array
- n_bins : int
number of bins in logarithmic wavelength grid
Returns: - lwave : np.array
logarithmic wavelength grid
- lflux : np.array
rebinned fluxes on grid
- valid_mask : boolean array
selection mask for signal on grid
-
deepSIP.utils.
normalize_flux
(flux)¶ normalize flux range of 1 centered at 0
Parameters: - flux : np.array
fluxes
Returns: - np.array
normalized flux
-
deepSIP.utils.
apodize
(flux, end_pct)¶ taper spectrum at each end to zero using hanning window
Parameters: - flux : np.array
fluxes
- end_pct : float
percentage to taper at each end
Returns: - np.array
apodized flux
-
deepSIP.utils.
redshift
(wave, z)¶ transform wavelength to observer frame
Parameters: - wave : np.array
rest wavelength grid
- z : float
redshift of SN relative to observer
Returns: - np.array
observer-frame wavelength grid
-
deepSIP.utils.
drop_ends
(wave, flux, max_drop_frac)¶ drop random fraction of spectrum from ends
Parameters: - wave : np.array
wavelength grid
- flux : np.array
fluxes on wavelength grid
- max_drop_frac : float
maximum fraction to be dropped from each end
Returns: - np.array, np.array
truncated wave and flux arrays
-
deepSIP.utils.
reset_state
(seed=100)¶ set all seeds for reproducibility
Parameters: - seed : int, optional
seed for random number generator
Notes
seeds are set for torch (including cuda), numpy, random, and PHYTHONHASH; this appears to be sufficient to ensure reproducibility
-
class
deepSIP.utils.
VoidScaler
(*args, **kwargs)¶ Bases:
object
sklearn conforming scaler that has no effect (i.e. identity transformation)
Parameters: - *args
arbitrary arguments with no effect
- **kwargs
arbitrary keyword arguments with no effect
Methods
fit fit_transform inverse_transform transform -
fit
(self, X)¶
-
transform
(self, X)¶
-
inverse_transform
(self, X)¶
-
fit_transform
(self, X)¶
-
class
deepSIP.utils.
LinearScaler
(minimum=0.0, maximum=1.0)¶ Bases:
deepSIP.utils.VoidScaler
sklearn conforming that behaves as MinMaxScaler but min and max are fixed
Parameters: - [min,max]imum : float or np.array
[min,max]imum value(s) used for transformation
Attributes: - [min,max] : float or np.array
see Parameters
Methods
fit fit_transform inverse_transform transform -
transform
(self, X)¶
-
inverse_transform
(self, X)¶
-
deepSIP.utils.
torch2numpy
(*tensors)¶ convert input torch tensors to numpy ndarrays
Parameters: - *tensors
torch tensors to convert to corresponding numpy ndarrays
Returns: - ndarrays: np.ndarrays
numpy ndarrays corresponding to input torch tensors
-
deepSIP.utils.
count_params
(network)¶ count trainable parameters in network
Parameters: - network : torch network
network to count trainable parameters for
Returns: - int
number of trainable parameters in input network
-
deepSIP.utils.
init_weights
(network_component)¶ initialize weights (normal) and biases (slight positive)
Parameters: - network_component : component torch network
component to initialize
-
deepSIP.utils.
stochastic_predict
(network, X, mcnum=75, sigmoid=False, seed=None, scaler=None, status=False)¶ perform mcnum stochastic forward passes, return mean and std np.arrays
Parameters: - network : torch network
network to generate predictions with
- X : torch tensor
inputs to network
- mcnum : int, optional
number of stochastic forward passes to perform
- sigmoid : bool, optional
apply sigmoid to outputs
- seed : int, optional
seed for random number generator
- scaler : sklearn conforming scaler
scaler to inverse transform outputs with
- status : bool, optional
show status bars
Returns: - mean : np.ndarray
mean prediction per input
- std : np.ndarray
std of predictions per input
Notes
Does mcnum forward passes on each row of input, one at a time, with re-seeding between each row. This ensures that predictions on a given input will be the same regardless of the order of inputs. It is possible the algorithm could be optimized for speed, but in tests it performs comparably to the previous implementation that suffered from modest reproducibility issues.
-
class
deepSIP.utils.
WrappedModel
(network)¶ Bases:
torch.nn.modules.module.Module
wrapper to seamlessly load network on cpu from dicts saved with DataParallel
Parameters: - network : torch network
network to wrap
Attributes: - module : torch network
see Parameters
Methods
__call__
(self, *input, **kwargs)Call self as a function. add_module
(self, name, module)Adds a child module to the current module. apply
(self, fn)Applies fn
recursively to every submodule (as returned by.children()
) as well as self.buffers
(self[, recurse])Returns an iterator over module buffers. children
(self)Returns an iterator over immediate children modules. cpu
(self)Moves all model parameters and buffers to the CPU. cuda
(self[, device])Moves all model parameters and buffers to the GPU. double
(self)Casts all floating point parameters and buffers to double
datatype.eval
(self)Sets the module in evaluation mode. extra_repr
(self)Set the extra representation of the module float
(self)Casts all floating point parameters and buffers to float datatype. forward
(self, *x)pass arguments to forward of module half
(self)Casts all floating point parameters and buffers to half
datatype.load_state_dict
(self, state_dict[, strict])Copies parameters and buffers from state_dict
into this module and its descendants.modules
(self)Returns an iterator over all modules in the network. named_buffers
(self[, prefix, recurse])Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself. named_children
(self)Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself. named_modules
(self[, memo, prefix])Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself. named_parameters
(self[, prefix, recurse])Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself. parameters
(self[, recurse])Returns an iterator over module parameters. register_backward_hook
(self, hook)Registers a backward hook on the module. register_buffer
(self, name, tensor)Adds a persistent buffer to the module. register_forward_hook
(self, hook)Registers a forward hook on the module. register_forward_pre_hook
(self, hook)Registers a forward pre-hook on the module. register_parameter
(self, name, param)Adds a parameter to the module. state_dict
(self[, destination, prefix, …])Returns a dictionary containing a whole state of the module. to
(self, *args, **kwargs)Moves and/or casts the parameters and buffers. train
(self[, mode])Sets the module in training mode. type
(self, dst_type)Casts all parameters and buffers to dst_type
.zero_grad
(self)Sets gradients of all model parameters to zero. share_memory -
forward
(self, *x)¶ pass arguments to forward of module
-
class
deepSIP.utils.
VoidLRScheduler
¶ Bases:
object
torch conforming LR scheduler with no effect
Methods
step -
step
(self, *args)¶
-
-
deepSIP.utils.
rmse
(y, yhat)¶ root-mean-square error
Parameters: - y : np.ndarray
true labels for each unique target
- yhat : np.ndarray
predictions for each unique target
Returns: - rmse : np.ndarray
root-mean-square error for each unique target
-
deepSIP.utils.
wrmse
(y, yhat, sigma_hat)¶ inverse predicted variance weighted root-mean-square error
Parameters: - y : np.ndarray
true labels for each unique target
- yhat : np.ndarray
predictions for each unique target
- sigma_hat : np.ndarray
predicted uncertainties for each unique target
Returns: - wrmse : np.ndarray
weighted root-mean-square error for each unique target
-
deepSIP.utils.
outlier
(y, yhat)¶ largest absolute residual
Parameters: - y : np.ndarray
true labels for each unique target
- yhat : np.ndarray
predictions for each unique target
Returns: - mr : np.ndarray
maximum absolute residual for each unique target
-
deepSIP.utils.
slope
(y, yhat)¶ slope of linear fit to yhat as a function of y
Parameters: - y : np.array
true labels
- yhat : np.array
predictions
Returns: - slope : float
slope of fitted line to yhat vs y
-
deepSIP.utils.
regression_metrics
(y, yhat, sigma_hat, key_prepend='')¶ compute all regression-related metrics
Parameters: - y : np.ndarray
true labels for each unique target
- yhat : np.ndarray
predictions for each unique target
- sigma_hat : np.ndarray
predicted uncertainties for each unique target
- key_prepend : str, optional
label to prepend output dict keys with
Returns: - metrics : dict
all computed regression metrics for each unique target
-
deepSIP.utils.
classify
(p, threshold=0.5)¶ given probabilities, perform binary classification using threshold
Parameters: - p : np.array
probabilities for binary classification
- threshold : float, optional
decision threshold for binary classification
Returns: - np.array
binary classifications
-
deepSIP.utils.
classification_metrics
(y, p, threshold=0.5, key_prepend='')¶ compute all classification related metrics
Parameters: - y : np.array
true labels
- p : np.array
probabilities for binary classification
- threshold : float, optional
decision threshold for binary classification
- key_prepend : str, optional
label to prepend output dict keys with
Returns: - metrics : dict
all computed classification metrics