Imputation models¶
Imputation models and the amputation methods used to train and evaluate them. See Imputation.
icu_benchmarks.imputation.amputations
¶
This file implements amputation mechanisms (MCAR, MAR (logisitc) and MNAR (logistic)) for missing data generation. It was inspired from: https://rmisstastic.netlify.app/how-to/python/generate_html/how%20to%20generate%20missing%20values Original code: https://github.com/BorisMuzellec/MissingDataOT/blob/master/utils.py
Functions:
| Name | Description |
|---|---|
MCAR_mask |
Missing completely at random mechanism. |
BO_mask |
Black out missing mechanism. Removes values across dimensions. |
MAR_logistic_mask |
Missing at random mechanism with a logistic masking model. First, a subset of variables with no missing values is |
MNAR_logistic_mask |
Missing not at random mechanism with a logistic masking model. It implements two mechanisms: |
ampute_data |
Generate missing values for specifics missing-data mechanism and proportion of missing values. |
MCAR_mask
¶
Missing completely at random mechanism.
Parameters¶
X : torch.FloatTensor, shape (n, d) Data for which missing values will be simulated. p : float Proportion of missing values to generate for variables which will have missing values.
Returns¶
mask : torch.BoolTensor Mask of generated missing values (True if the value is missing).
Source code in icu_benchmarks/imputation/amputations.py
BO_mask
¶
Black out missing mechanism. Removes values across dimensions.
Parameters¶
X : torch.FloatTensor, shape (n, d) Data for which missing values will be simulated. p : float Proportion of missing values to generate for variables which will have missing values.
Returns¶
mask : torch.BoolTensor Mask of generated missing values (True if the value is missing).
Source code in icu_benchmarks/imputation/amputations.py
MAR_logistic_mask
¶
Missing at random mechanism with a logistic masking model. First, a subset of variables with no missing values is randomly selected. The remaining variables have missing values according to a logistic model with random weights, re-scaled so as to attain the desired proportion of missing values on those variables.
Parameters¶
X : torch.FloatTensor, shape (n, d) Data for which missing values will be simulated. p : float Proportion of missing values to generate for variables which will have missing values. p_obs : float Proportion of variables with no missing values that will be used for the logistic masking model.
Returns¶
mask : torch.BoolTensor Mask of generated missing values (True if the value is missing).
Source code in icu_benchmarks/imputation/amputations.py
MNAR_logistic_mask
¶
Missing not at random mechanism with a logistic masking model. It implements two mechanisms: (i) Missing probabilities are selected with a logistic model, taking all variables as inputs. Hence, values that are inputs can also be missing. (ii) Variables are split into a set of inputs for a logistic model, and a set whose missing probabilities are determined by the logistic model. Then inputs are then masked MCAR (hence, missing values from the second set will depend on masked values. In either case, weights are random and the intercept is selected to attain the desired proportion of missing values.
Parameters¶
X : torch.FloatTensor, shape (n, d) Data for which missing values will be simulated. p : float Proportion of missing values to generate for variables which will have missing values. p_params : float Proportion of variables that will be used for the logistic masking model (only if exclude_inputs). exclude_inputs : boolean, default=True True: mechanism (ii) is used, False: (i)
Returns¶
mask : torch.BoolTensor Mask of generated missing values (True if the value is missing).
Source code in icu_benchmarks/imputation/amputations.py
ampute_data
¶
Generate missing values for specifics missing-data mechanism and proportion of missing values.
Parameters¶
data : DataFrame Data for which missing values will be simulated. mechanism : str, Indicates the missing-data mechanism to be used. ("MCAR", "MAR" or "MNAR") p_miss : float Proportion of missing values to generate for variables which will have missing values. p_obs : float If mecha = "MAR" or "MNAR", proportion of variables with no missing values that will be used for the logistic masking model.
Returns¶
imputed_data: DataFrame The data with the generated missing values.
Source code in icu_benchmarks/imputation/amputations.py
icu_benchmarks.imputation.baselines
¶
Baseline imputation methods. These methods imported from other frameworks and are used as baselines for comparison.
Classes:
| Name | Description |
|---|---|
KNNImputation |
Imputation using Scikit-Learn K-Nearest Neighbour. |
MICEImputation |
Imputation using Scikit-Learn MICE. |
MeanImputation |
Mean imputation using Scikit-Learn SimpleImputer. |
MedianImputation |
Median imputation using Scikit-Learn SimpleImputer. |
ZeroImputation |
Zero imputation using Scikit-Learn SimpleImputer. |
MostFrequentImputation |
Most frequent imputation using Scikit-Learn SimpleImputer. |
BRITSImputation |
Bidirectional Recurrent Imputation for Time Series (BRITS) imputation using PyPots package. |
SAITSImputation |
Self-Attention based Imputation for Time Series (SAITS) imputation using PyPots package. |
AttentionImputation |
Attention based Imputation (Transformer) imputation using PyPots package. |
Functions:
| Name | Description |
|---|---|
import_optional |
Import a package from the optional imputation extra (pip install "yaib[imputation]"). |
pypots_fit |
PyPOTS models train on a dict holding the (stays, time steps, features) array with missing values. |
pypots_impute |
Imputes a batch with a fitted PyPOTS model, keeping the batch shape and device. |
KNNImputation
¶
Bases: ImputationWrapper
Imputation using Scikit-Learn K-Nearest Neighbour.
Source code in icu_benchmarks/imputation/baselines.py
MICEImputation
¶
Bases: ImputationWrapper
Imputation using Scikit-Learn MICE.
Source code in icu_benchmarks/imputation/baselines.py
MeanImputation
¶
Bases: ImputationWrapper
Mean imputation using Scikit-Learn SimpleImputer.
Source code in icu_benchmarks/imputation/baselines.py
MedianImputation
¶
Bases: ImputationWrapper
Median imputation using Scikit-Learn SimpleImputer.
Source code in icu_benchmarks/imputation/baselines.py
ZeroImputation
¶
Bases: ImputationWrapper
Zero imputation using Scikit-Learn SimpleImputer.
Source code in icu_benchmarks/imputation/baselines.py
MostFrequentImputation
¶
Bases: ImputationWrapper
Most frequent imputation using Scikit-Learn SimpleImputer.
Source code in icu_benchmarks/imputation/baselines.py
BRITSImputation
¶
Bases: ImputationWrapper
Bidirectional Recurrent Imputation for Time Series (BRITS) imputation using PyPots package.
Source code in icu_benchmarks/imputation/baselines.py
SAITSImputation
¶
SAITSImputation(*args, input_size, epochs, n_layers, d_model, d_inner, n_head, d_k, d_v, dropout, **kwargs)
Bases: ImputationWrapper
Self-Attention based Imputation for Time Series (SAITS) imputation using PyPots package.
Source code in icu_benchmarks/imputation/baselines.py
AttentionImputation
¶
AttentionImputation(*args, input_size, epochs, n_layers, d_model, d_inner, n_head, d_k, d_v, dropout, **kwargs)
Bases: ImputationWrapper
Attention based Imputation (Transformer) imputation using PyPots package.
Source code in icu_benchmarks/imputation/baselines.py
import_optional
¶
Import a package from the optional imputation extra (pip install "yaib[imputation]").
Source code in icu_benchmarks/imputation/baselines.py
pypots_fit
¶
PyPOTS models train on a dict holding the (stays, time steps, features) array with missing values.
Source code in icu_benchmarks/imputation/baselines.py
pypots_impute
¶
Imputes a batch with a fitted PyPOTS model, keeping the batch shape and device.
Source code in icu_benchmarks/imputation/baselines.py
icu_benchmarks.imputation.mlp
¶
Classes:
| Name | Description |
|---|---|
MLPImputation |
Imputation model based on a Multi-Layer Perceptron (MLP). |
MLPImputation
¶
Bases: ImputationWrapper
Imputation model based on a Multi-Layer Perceptron (MLP).
Source code in icu_benchmarks/imputation/mlp.py
icu_benchmarks.imputation.rnn
¶
Classes:
| Name | Description |
|---|---|
RNNImputation |
Imputation model with Gated Recurrent Units (GRU) or Long-Short Term Memory Network (LSTM). Defaults to GRU. |
BRNNImputation |
Imputation model with Bidirectional Gated Recurrent Units (GRU) or Long-Short Term Memory Network (LSTM). Defaults to |
RNNImputation
¶
Bases: ImputationWrapper
Imputation model with Gated Recurrent Units (GRU) or Long-Short Term Memory Network (LSTM). Defaults to GRU.
Source code in icu_benchmarks/imputation/rnn.py
BRNNImputation
¶
BRNNImputation(*args, input_size, hidden_size=64, state_init='zero', dropout=0.0, cell='gru', **kwargs)
Bases: ImputationWrapper
Imputation model with Bidirectional Gated Recurrent Units (GRU) or Long-Short Term Memory Network (LSTM). Defaults to GRU.
Source code in icu_benchmarks/imputation/rnn.py
icu_benchmarks.imputation.np
¶
Classes:
| Name | Description |
|---|---|
NPImputation |
Imputation using Neural Processes. Implementation adapted from https://github.com/EmilienDupont/neural-processes/. |
NeuralProcess |
Class that implements neural processes. |
NPImputation
¶
NPImputation(input_size, encoder_layers, encoder_h_dim, decoder_layers, decoder_h_dim, r_dim, z_dim, train_sample_times, val_sample_times, test_sample_times, predict_sample_times, *args, **kwargs)
Bases: ImputationWrapper
Imputation using Neural Processes. Implementation adapted from https://github.com/EmilienDupont/neural-processes/. Provides imputation wrapper for NeuralProcess class.
Source code in icu_benchmarks/imputation/np.py
NeuralProcess
¶
NeuralProcess(x_dim, y_dim, encoder_layers, encoder_h_dim, decoder_layers, decoder_h_dim, r_dim, z_dim)
Bases: Module
Class that implements neural processes.
Source code in icu_benchmarks/imputation/np.py
icu_benchmarks.imputation.csdi
¶
Classes:
| Name | Description |
|---|---|
CSDI |
Conditional Score-based Diffusion Models for Imputation (CSDI) of Time Series. See https://arxiv.org/abs/2107.03502 |
CSDI
¶
CSDI(input_size, time_step_embedding_size, feature_embedding_size, unconditional, target_strategy, num_diffusion_steps, diffusion_step_embedding_dim, n_attention_heads, num_residual_layers, noise_schedule, beta_start, beta_end, n_samples, conv_channels, *args, **kwargs)
Bases: ImputationWrapper
Conditional Score-based Diffusion Models for Imputation (CSDI) of Time Series. See https://arxiv.org/abs/2107.03502 for details.
Source code in icu_benchmarks/imputation/csdi.py
icu_benchmarks.imputation.diffusion
¶
Classes:
| Name | Description |
|---|---|
SimpleDiffusionModel |
Simple Diffusion Model for Imputation. See https://arxiv.org/abs/2006.11239 for more details. |
SimpleDiffusionModel
¶
SimpleDiffusionModel(input_size, n_onedirectional_conv, T, min_noise, max_noise, noise_scheduler, *args, **kwargs)
Bases: ImputationWrapper
Simple Diffusion Model for Imputation. See https://arxiv.org/abs/2006.11239 for more details.
Methods:
| Name | Description |
|---|---|
predict_step |
Impute the missing values by running the reverse diffusion from pure noise. At each timestep, the |
backward_diffusion_sample |
Function that takes a noised image at some timestep t and the noise prediction and tries to compute the original |
Source code in icu_benchmarks/imputation/diffusion.py
predict_step
¶
Impute the missing values by running the reverse diffusion from pure noise. At each timestep, the observed values are replaced by their forward diffusion sample for that timestep.
Source code in icu_benchmarks/imputation/diffusion.py
backward_diffusion_sample
¶
Function that takes a noised image at some timestep t and the noise prediction and tries to compute the original sample. The t needs to be one specific timestamp -> always the same value. It does not have to be like this in the forward diffusion sample function
Source code in icu_benchmarks/imputation/diffusion.py
icu_benchmarks.imputation.simple_diffusion
¶
Classes:
| Name | Description |
|---|---|
SimpleDiffusionModel |
Imputation model based on a Simple Diffusion Model. |
SimpleDiffusionModel
¶
Bases: ImputationWrapper
Imputation model based on a Simple Diffusion Model. Adapted from https://colab.research.google.com/drive/1sjy9odlSSy0RBVgMTgP7s99NXsqglsUL.
Methods:
| Name | Description |
|---|---|
get_index_from_list |
Returns a specific index t of a passed list of values vals |
forward_diffusion_sample |
Takes an image and a timestep as input and |
predict_step |
Impute the missing values by running the reverse diffusion from pure noise. At each timestep, the |
Source code in icu_benchmarks/imputation/simple_diffusion.py
get_index_from_list
¶
Returns a specific index t of a passed list of values vals while considering the batch dimension.
Source code in icu_benchmarks/imputation/simple_diffusion.py
forward_diffusion_sample
¶
Takes an image and a timestep as input and returns the noisy version of it
Source code in icu_benchmarks/imputation/simple_diffusion.py
predict_step
¶
Impute the missing values by running the reverse diffusion from pure noise. At each timestep, the observed values are replaced by their forward diffusion sample for that timestep.
Source code in icu_benchmarks/imputation/simple_diffusion.py
icu_benchmarks.imputation.diffwave
¶
Classes:
| Name | Description |
|---|---|
DiffWaveImputer |
Imputation model based on DiffWave (https://arxiv.org/abs/2009.09761). Adapted from |
Functions:
| Name | Description |
|---|---|
calc_diffusion_step_embedding |
Embed a diffusion step $t$ into a higher dimensional space |
calc_diffusion_hyperparams |
Compute diffusion process hyperparameters |
std_normal |
Generate the standard Gaussian variable of a certain size |
DiffWaveImputer
¶
DiffWaveImputer(in_channels, res_channels, skip_channels, out_channels, num_res_layers, dilation_cycle, diffusion_step_embed_dim_in, diffusion_step_embed_dim_mid, diffusion_step_embed_dim_out, diffusion_time_steps, beta_0, beta_T, *args, **kwargs)
Bases: ImputationWrapper
Imputation model based on DiffWave (https://arxiv.org/abs/2009.09761). Adapted from https://github.com/AI4HealthUOL/SSSD/blob/main/src/imputers/DiffWaveImputer.py
Methods:
| Name | Description |
|---|---|
sampling |
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t) |
Source code in icu_benchmarks/imputation/diffwave.py
sampling
¶
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t)
Parameters: net (torch network): the wavenet model size (tuple): size of tensor to be generated, usually is (number of audios to generate, channels=1, length of audio) diffusion_hyperparams (dict): dictionary of diffusion hyperparameters returned by calc_diffusion_hyperparams note, the tensors need to be cuda tensors
Returns: the generated audio(s) in torch.tensor, shape=size
Source code in icu_benchmarks/imputation/diffwave.py
calc_diffusion_step_embedding
¶
Embed a diffusion step $t$ into a higher dimensional space E.g. the embedding vector in the 128-dimensional space is [sin(t * 10^(04/63)), ... , sin(t * 10^(634/63)), cos(t * 10^(04/63)), ... , cos(t * 10^(634/63))] Parameters: diffusion_steps (torch.long tensor, shape=(batchsize, 1)): diffusion steps for batch data diffusion_step_embed_dim_in (int, default=128): dimensionality of the embedding space for discrete diffusion steps
Returns: the embedding vectors (torch.tensor, shape=(batchsize, diffusion_step_embed_dim_in)):
Source code in icu_benchmarks/imputation/diffwave.py
calc_diffusion_hyperparams
¶
Compute diffusion process hyperparameters
Parameters: T (int): number of diffusion steps beta_0 and beta_T (float): beta schedule start/end value, where any beta_t in the middle is linearly interpolated
a dictionary of diffusion hyperparameters including: T (int), Beta/Alpha/Alpha_bar/Sigma (torch.tensor on cpu, shape=(T, )) These cpu tensors are changed to cuda tensors on each individual gpu
Source code in icu_benchmarks/imputation/diffwave.py
std_normal
¶
icu_benchmarks.imputation.sssds4
¶
Classes:
| Name | Description |
|---|---|
SSSDS4 |
Implements the SSSD model from https://arxiv.org/abs/2208.09399. |
Functions:
| Name | Description |
|---|---|
std_normal |
Generate the standard Gaussian variable of a certain size |
calc_diffusion_step_embedding |
Embed a diffusion step $t$ into a higher dimensional space |
calc_diffusion_hyperparams |
Compute diffusion process hyperparameters |
SSSDS4
¶
SSSDS4(input_size, res_channels, skip_channels, num_res_layers, diffusion_step_embed_dim_in, diffusion_step_embed_dim_mid, diffusion_step_embed_dim_out, s4_lmax, s4_d_state, s4_dropout, s4_bidirectional, s4_layernorm, diffusion_time_steps, beta_0, beta_T, *args, **kwargs: str)
Bases: ImputationWrapper
Implements the SSSD model from https://arxiv.org/abs/2208.09399.
Methods:
| Name | Description |
|---|---|
sampling |
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t) |
Source code in icu_benchmarks/imputation/sssds4.py
sampling
¶
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t)
Parameters: net (torch network): the wavenet model size (tuple): size of tensor to be generated, usually is (number of audios to generate, channels=1, length of audio) diffusion_hyperparams (dict): dictionary of diffusion hyperparameters returned by calc_diffusion_hyperparams note, the tensors need to be cuda tensors
Returns: the generated audio(s) in torch.tensor, shape=size
Source code in icu_benchmarks/imputation/sssds4.py
std_normal
¶
calc_diffusion_step_embedding
¶
Embed a diffusion step $t$ into a higher dimensional space E.g. the embedding vector in the 128-dimensional space is [sin(t * 10^(04/63)), ... , sin(t * 10^(634/63)), cos(t * 10^(04/63)), ... , cos(t * 10^(634/63))] Parameters: diffusion_steps (torch.long tensor, shape=(batchsize, 1)): diffusion steps for batch data diffusion_step_embed_dim_in (int, default=128): dimensionality of the embedding space for discrete diffusion steps
Returns: the embedding vectors (torch.tensor, shape=(batchsize, diffusion_step_embed_dim_in)):
Source code in icu_benchmarks/imputation/sssds4.py
calc_diffusion_hyperparams
¶
Compute diffusion process hyperparameters
Parameters: T (int): number of diffusion steps beta_0 and beta_T (float): beta schedule start/end value, where any beta_t in the middle is linearly interpolated
a dictionary of diffusion hyperparameters including: T (int), Beta/Alpha/Alpha_bar/Sigma (torch.tensor on cpu, shape=(T, )) These cpu tensors are changed to cuda tensors on each individual gpu
Source code in icu_benchmarks/imputation/sssds4.py
icu_benchmarks.imputation.sssdsa
¶
Classes:
| Name | Description |
|---|---|
SSSDSA |
"SaShiMi model backbone. Adapted from https://github.com/AI4HealthUOL/SSSD/blob/main/src/imputers/SSSDSAImputer.py |
DownPool |
|
UpPool |
|
FFBlock |
|
ResidualBlock |
|
Functions:
| Name | Description |
|---|---|
std_normal |
Generate the standard Gaussian variable of a certain size |
calc_diffusion_step_embedding |
Embed a diffusion step $t$ into a higher dimensional space |
largets_component |
returns the prime number that divides number the most times. |
next_power |
returns the next power of 2. |
calc_diffusion_hyperparams |
Compute diffusion process hyperparameters |
SSSDSA
¶
SSSDSA(d_model, n_layers, pool, expand, ff, glu, unet, dropout, in_channels, out_channels, diffusion_step_embed_dim_in, diffusion_step_embed_dim_mid, diffusion_step_embed_dim_out, label_embed_dim, label_embed_classes, bidirectional, s4_lmax, s4_d_state, s4_dropout, s4_bidirectional, diffusion_time_steps, beta_0, beta_T, input_size, *args, **kwargs)
Bases: ImputationWrapper
"SaShiMi model backbone. Adapted from https://github.com/AI4HealthUOL/SSSD/blob/main/src/imputers/SSSDSAImputer.py
SaShiMi model backbone.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d_model
|
dimension of the model. We generally use 64 for all our experiments. |
required | |
n_layers
|
number of (Residual (S4) --> Residual (FF)) blocks at each pooling level. We use 8 layers for our experiments, although we found that increasing layers even further generally improves performance at the expense of training / inference speed. |
required | |
pool
|
pooling factor at each level. Pooling shrinks the sequence length at lower levels. We experimented with a pooling factor of 4 with 1 to 4 tiers of pooling and found 2 tiers to be best. It's possible that a different combination of pooling factors and number of tiers may perform better. |
required | |
expand
|
expansion factor when pooling. Features are expanded (i.e. the model becomes wider) at lower levels of the architecture.We generally found 2 to perform best (among 2, 4). |
required | |
ff
|
expansion factor for the FF inverted bottleneck. We generally found 2 to perform best (among 2, 4). |
required | |
bidirectional
|
use bidirectional S4 layers. Bidirectional layers are suitable for use with non-causal models such as diffusion models like DiffWave. |
required | |
glu
|
use gated linear unit in the S4 layers. Adds parameters and generally improves performance. |
required | |
unet
|
use a unet-like architecture, adding (Residual (S4) --> Residual (FF)) layers before downpooling. All else fixed, this slows down inference (and slightly slows training), but generally improves performance. We use this variant when dropping in SaShiMi into diffusion models, and this should generally be preferred for non-autoregressive models. |
required | |
dropout
|
dropout rate. Default to 0.0, since we haven't found settings where SaShiMi overfits. |
required |
Methods:
| Name | Description |
|---|---|
step |
input: (batch, d_input) |
setup_rnn |
Convert the SaShiMi model to a RNN for autoregressive generation. |
sampling |
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t) |
Source code in icu_benchmarks/imputation/sssdsa.py
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | |
step
¶
input: (batch, d_input) output: (batch, d_output)
Source code in icu_benchmarks/imputation/sssdsa.py
setup_rnn
¶
Convert the SaShiMi model to a RNN for autoregressive generation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mode
|
S4 recurrence mode. Using |
'dense'
|
Source code in icu_benchmarks/imputation/sssdsa.py
sampling
¶
Perform the complete sampling step according to p(x_0|x_T) = prod_{t=1}^T p_{ heta}(x_{t-1}|x_t)
Parameters: net (torch network): the wavenet model size (tuple): size of tensor to be generated, usually is (number of audios to generate, channels=1, length of audio) diffusion_hyperparams (dict): dictionary of diffusion hyperparameters returned by calc_diffusion_hyperparams note, the tensors need to be cuda tensors
Returns: the generated audio(s) in torch.tensor, shape=size
Source code in icu_benchmarks/imputation/sssdsa.py
DownPool
¶
UpPool
¶
Bases: Module
Methods:
| Name | Description |
|---|---|
step |
x: (..., H) |
Source code in icu_benchmarks/imputation/sssdsa.py
step
¶
x: (..., H)
Source code in icu_benchmarks/imputation/sssdsa.py
FFBlock
¶
Bases: Module
Feed-forward block.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d_model
|
dimension of input |
required | |
expand
|
expansion factor for inverted bottleneck |
2
|
|
dropout
|
dropout rate |
0.0
|
Source code in icu_benchmarks/imputation/sssdsa.py
ResidualBlock
¶
ResidualBlock(d_model, layer, dropout, diffusion_step_embed_dim_out, in_channels, label_embed_dim, stride)
Bases: Module
Residual S4 block.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d_model
|
dimension of the model |
required | |
bidirectional
|
use bidirectional S4 layer |
required | |
glu
|
use gated linear unit in the S4 layer |
required | |
dropout
|
dropout rate |
required |
Methods:
| Name | Description |
|---|---|
forward |
Input x is shape (B, d_input, L) |
Source code in icu_benchmarks/imputation/sssdsa.py
forward
¶
Input x is shape (B, d_input, L)
Source code in icu_benchmarks/imputation/sssdsa.py
std_normal
¶
calc_diffusion_step_embedding
¶
Embed a diffusion step $t$ into a higher dimensional space E.g. the embedding vector in the 128-dimensional space is [sin(t * 10^(04/63)), ... , sin(t * 10^(634/63)), cos(t * 10^(04/63)), ... , cos(t * 10^(634/63))] Parameters: diffusion_steps (torch.long tensor, shape=(batchsize, 1)): diffusion steps for batch data diffusion_step_embed_dim_in (int, default=128): dimensionality of the embedding space for discrete diffusion steps
Returns: the embedding vectors (torch.tensor, shape=(batchsize, diffusion_step_embed_dim_in)):
Source code in icu_benchmarks/imputation/sssdsa.py
largets_component
¶
returns the prime number that divides number the most times.
next_power
¶
calc_diffusion_hyperparams
¶
Compute diffusion process hyperparameters
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
T
|
int
|
number of diffusion steps. |
required |
beta_0 beta_T (float)
|
beta schedule start/end value, where any beta_t in the middle is linearly interpolated |
required |
a dictionary of diffusion hyperparameters including: T (int), Beta/Alpha/Alpha_bar/Sigma (torch.tensor on cpu, shape=(T, )) These cpu tensors are changed to cuda tensors on each individual gpu