deepSIP.utils

deepSIP.utils.load_txt_spectrum(filename)

load spectrum from txt file with first two columns as wave flux

Parameters:
filename : str

name of text file to read spectrum from

Returns:
wave : np.array

wavelength grid

flux : np.array

fluxes on wavelength grid

deepSIP.utils.savenet(network, filename)

save network parameters

Parameters:
network : DropoutCNN or torch network

network to save

filename : str

name of file to save network to

deepSIP.utils.loadnet(network, filename, device='cpu')

load network parameters

Parameters:
network : DropoutCNN

network to load weights and biases into

filename : str

name of file to load network from

device : str, optional

device to load network onto (‘cpu’ by default)

deepSIP.utils.deredshift(wave, z)

transform wavelength to rest frame

Parameters:
wave : np.array

wavelength grid

z : float

redshift of SN

Returns:
np.array

rest-frame wavelength grid

deepSIP.utils.smooth(flux, window, order)

smooth using a savitzky-golay filter

Parameters:
flux : np.array

fluxes

window : int

window size in angstroms for smoothing

smoothing : int

polynomial order for smoothing

Returns:
np.array

smoothed spectrum

deepSIP.utils.get_continuum(flux, window, order)

model continuum from heavily smoothed spectrum

Notes

see documentiation for smooth, which this function simply wraps

deepSIP.utils.log_wave(bounds, n_bins)

compute logarithmic wavelength grid

Parameters:
bounds : tuple, list, or other iterable of length 2

lower and upper limit of logarithmic wavelength array

n_bins : int

number of bins in logarithmic wavelength grid

Returns:
np.array

logarithmic wavelength grid

deepSIP.utils.log_bin(wave, flux, bounds, n_bins)

rebin onto logarithmic wavelength grid (adapted from astrodash/SNID)

Parameters:
wave : np.array

wavelength grid

flux : np.array

fluxes on wavelength grid

bounds : tuple, list, or other iterable of length 2

lower and upper limit of logarithmic wavelength array

n_bins : int

number of bins in logarithmic wavelength grid

Returns:
lwave : np.array

logarithmic wavelength grid

lflux : np.array

rebinned fluxes on grid

valid_mask : boolean array

selection mask for signal on grid

deepSIP.utils.normalize_flux(flux)

normalize flux range of 1 centered at 0

Parameters:
flux : np.array

fluxes

Returns:
np.array

normalized flux

deepSIP.utils.apodize(flux, end_pct)

taper spectrum at each end to zero using hanning window

Parameters:
flux : np.array

fluxes

end_pct : float

percentage to taper at each end

Returns:
np.array

apodized flux

deepSIP.utils.redshift(wave, z)

transform wavelength to observer frame

Parameters:
wave : np.array

rest wavelength grid

z : float

redshift of SN relative to observer

Returns:
np.array

observer-frame wavelength grid

deepSIP.utils.drop_ends(wave, flux, max_drop_frac)

drop random fraction of spectrum from ends

Parameters:
wave : np.array

wavelength grid

flux : np.array

fluxes on wavelength grid

max_drop_frac : float

maximum fraction to be dropped from each end

Returns:
np.array, np.array

truncated wave and flux arrays

deepSIP.utils.reset_state(seed=100)

set all seeds for reproducibility

Parameters:
seed : int, optional

seed for random number generator

Notes

seeds are set for torch (including cuda), numpy, random, and PHYTHONHASH; this appears to be sufficient to ensure reproducibility

class deepSIP.utils.VoidScaler(*args, **kwargs)

Bases: object

sklearn conforming scaler that has no effect (i.e. identity transformation)

Parameters:
*args

arbitrary arguments with no effect

**kwargs

arbitrary keyword arguments with no effect

Methods

fit  
fit_transform  
inverse_transform  
transform  
fit(self, X)
transform(self, X)
inverse_transform(self, X)
fit_transform(self, X)
class deepSIP.utils.LinearScaler(minimum=0.0, maximum=1.0)

Bases: deepSIP.utils.VoidScaler

sklearn conforming that behaves as MinMaxScaler but min and max are fixed

Parameters:
[min,max]imum : float or np.array

[min,max]imum value(s) used for transformation

Attributes:
[min,max] : float or np.array

see Parameters

Methods

fit  
fit_transform  
inverse_transform  
transform  
transform(self, X)
inverse_transform(self, X)
deepSIP.utils.torch2numpy(*tensors)

convert input torch tensors to numpy ndarrays

Parameters:
*tensors

torch tensors to convert to corresponding numpy ndarrays

Returns:
ndarrays: np.ndarrays

numpy ndarrays corresponding to input torch tensors

deepSIP.utils.count_params(network)

count trainable parameters in network

Parameters:
network : torch network

network to count trainable parameters for

Returns:
int

number of trainable parameters in input network

deepSIP.utils.init_weights(network_component)

initialize weights (normal) and biases (slight positive)

Parameters:
network_component : component torch network

component to initialize

deepSIP.utils.stochastic_predict(network, X, mcnum=75, sigmoid=False, seed=None, scaler=None, status=False)

perform mcnum stochastic forward passes, return mean and std np.arrays

Parameters:
network : torch network

network to generate predictions with

X : torch tensor

inputs to network

mcnum : int, optional

number of stochastic forward passes to perform

sigmoid : bool, optional

apply sigmoid to outputs

seed : int, optional

seed for random number generator

scaler : sklearn conforming scaler

scaler to inverse transform outputs with

status : bool, optional

show status bars

Returns:
mean : np.ndarray

mean prediction per input

std : np.ndarray

std of predictions per input

Notes

Does mcnum forward passes on each row of input, one at a time, with re-seeding between each row. This ensures that predictions on a given input will be the same regardless of the order of inputs. It is possible the algorithm could be optimized for speed, but in tests it performs comparably to the previous implementation that suffered from modest reproducibility issues.

class deepSIP.utils.WrappedModel(network)

Bases: torch.nn.modules.module.Module

wrapper to seamlessly load network on cpu from dicts saved with DataParallel

Parameters:
network : torch network

network to wrap

Attributes:
module : torch network

see Parameters

Methods

__call__(self, *input, **kwargs) Call self as a function.
add_module(self, name, module) Adds a child module to the current module.
apply(self, fn) Applies fn recursively to every submodule (as returned by .children()) as well as self.
buffers(self[, recurse]) Returns an iterator over module buffers.
children(self) Returns an iterator over immediate children modules.
cpu(self) Moves all model parameters and buffers to the CPU.
cuda(self[, device]) Moves all model parameters and buffers to the GPU.
double(self) Casts all floating point parameters and buffers to double datatype.
eval(self) Sets the module in evaluation mode.
extra_repr(self) Set the extra representation of the module
float(self) Casts all floating point parameters and buffers to float datatype.
forward(self, *x) pass arguments to forward of module
half(self) Casts all floating point parameters and buffers to half datatype.
load_state_dict(self, state_dict[, strict]) Copies parameters and buffers from state_dict into this module and its descendants.
modules(self) Returns an iterator over all modules in the network.
named_buffers(self[, prefix, recurse]) Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
named_children(self) Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
named_modules(self[, memo, prefix]) Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
named_parameters(self[, prefix, recurse]) Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
parameters(self[, recurse]) Returns an iterator over module parameters.
register_backward_hook(self, hook) Registers a backward hook on the module.
register_buffer(self, name, tensor) Adds a persistent buffer to the module.
register_forward_hook(self, hook) Registers a forward hook on the module.
register_forward_pre_hook(self, hook) Registers a forward pre-hook on the module.
register_parameter(self, name, param) Adds a parameter to the module.
state_dict(self[, destination, prefix, …]) Returns a dictionary containing a whole state of the module.
to(self, *args, **kwargs) Moves and/or casts the parameters and buffers.
train(self[, mode]) Sets the module in training mode.
type(self, dst_type) Casts all parameters and buffers to dst_type.
zero_grad(self) Sets gradients of all model parameters to zero.
share_memory  
forward(self, *x)

pass arguments to forward of module

class deepSIP.utils.VoidLRScheduler

Bases: object

torch conforming LR scheduler with no effect

Methods

step  
step(self, *args)
deepSIP.utils.rmse(y, yhat)

root-mean-square error

Parameters:
y : np.ndarray

true labels for each unique target

yhat : np.ndarray

predictions for each unique target

Returns:
rmse : np.ndarray

root-mean-square error for each unique target

deepSIP.utils.wrmse(y, yhat, sigma_hat)

inverse predicted variance weighted root-mean-square error

Parameters:
y : np.ndarray

true labels for each unique target

yhat : np.ndarray

predictions for each unique target

sigma_hat : np.ndarray

predicted uncertainties for each unique target

Returns:
wrmse : np.ndarray

weighted root-mean-square error for each unique target

deepSIP.utils.outlier(y, yhat)

largest absolute residual

Parameters:
y : np.ndarray

true labels for each unique target

yhat : np.ndarray

predictions for each unique target

Returns:
mr : np.ndarray

maximum absolute residual for each unique target

deepSIP.utils.slope(y, yhat)

slope of linear fit to yhat as a function of y

Parameters:
y : np.array

true labels

yhat : np.array

predictions

Returns:
slope : float

slope of fitted line to yhat vs y

deepSIP.utils.regression_metrics(y, yhat, sigma_hat, key_prepend='')

compute all regression-related metrics

Parameters:
y : np.ndarray

true labels for each unique target

yhat : np.ndarray

predictions for each unique target

sigma_hat : np.ndarray

predicted uncertainties for each unique target

key_prepend : str, optional

label to prepend output dict keys with

Returns:
metrics : dict

all computed regression metrics for each unique target

deepSIP.utils.classify(p, threshold=0.5)

given probabilities, perform binary classification using threshold

Parameters:
p : np.array

probabilities for binary classification

threshold : float, optional

decision threshold for binary classification

Returns:
np.array

binary classifications

deepSIP.utils.classification_metrics(y, p, threshold=0.5, key_prepend='')

compute all classification related metrics

Parameters:
y : np.array

true labels

p : np.array

probabilities for binary classification

threshold : float, optional

decision threshold for binary classification

key_prepend : str, optional

label to prepend output dict keys with

Returns:
metrics : dict

all computed classification metrics