network

Neural network for MALA.

class FeedForwardNet(params: Parameters)[source]

Bases: Network

Initialize this network as a feed-forward network.

forward(inputs)[source]

Perform a forward pass through the network.

Parameters:: inputs (torch.Tensor) – Input array for which the forward pass is to be performed.
Returns:: predicted_array – Predicted outputs of array.
Return type:: torch.Tensor

class GRU(params: Parameters)[source]

Bases: LSTM

Initialize this network as a GRU network.

forward(x)[source]

Perform a forward pass through the network.

Parameters:: x (torch.Tensor) – Input array for which the forward pass is to be performed.
Returns:: predicted_array – Predicted outputs of array.
Return type:: torch.Tensor.

init_hidden()[source]

Initialize hidden state to zero when called and assigns specific sizes.

Returns:: Hidden state – initialised to zeros.
Return type:: torch.Tensor

class LSTM(params: Parameters)[source]

Bases: Network

Initialize this network as a LSTM network.

forward(x)[source]

Perform a forward pass through the network.

Parameters:: x (torch.Tensor) – Input array for which the forward pass is to be performed.
Returns:: predicted_array – Predicted outputs of array.
Return type:: torch.Tensor

init_hidden()[source]

Initialize hidden state and cell state to zero when called.

Also assigns specific sizes.

Returns:: Hidden state and cell state – initialised to zeros.
Return type:: torch.Tensor

class Network(params: Parameters)[source]

Bases: Module

Central network class for this framework, based on pytorch.nn.Module.

The correct type of neural network will automatically be instantiated by this class if possible. You can also instantiate the desired network directly by calling upon the subclass.

Parameters:: params (mala.common.parametes.Parameters) – Parameters used to create this neural network.

calculate_loss(output, target)[source]

Calculate the loss for a predicted output and target.

Parameters:

output (torch.Tensor) – Predicted output.
target (torch.Tensor.) – Actual output.

Returns:

loss_val – Loss value for output and target.

Return type:

float

do_prediction(array)[source]

Predict the output values for an input array..

Interface to do predictions. The data put in here is assumed to be a scaled torch.Tensor and in the right units. Be aware that this will pass the entire array through the network, which might be very demanding in terms of RAM.

Parameters:: array (torch.Tensor) – Input array for which the prediction is to be performed.
Returns:: predicted_array – Predicted outputs of array.
Return type:: torch.Tensor

abstract forward(inputs)[source]: Abstract method. To be implemented by the derived class.

classmethod load_from_file(params, file)[source]

Load a network from a file.

Parameters:

params (mala.common.parameters.Parameters) – Parameters object with which the network should be created. Has to be compatible to the network architecture. This is usually enforced by using the same Parameters object (and saving/loading it to)
file (string or ZipExtFile) – Path to the file from which the network should be loaded.

Returns:

loaded_network – The network that was loaded from the file.

Return type:

Network

save_network(path_to_file)[source]

Save the network.

This function serves as an interfaces to pytorchs own saving functionalities AND possibly own saving needs.

Parameters:: path_to_file (string) – Path to the file in which the network should be saved.

class PositionalEncoding(*args: Any, **kwargs: Any)[source]

Bases: Module

Injects some information of relative/absolute position of a token.

Parameters:

d_model (int) – input dimension of the model
dropout (float) – dropout rate
max_len (int) – maximum length of the input sequence

forward(x)[source]: Perform a forward pass through the network.

class TransformerNet(params: Parameters)[source]

Bases: Network

Initialize this network as the transformer net.

Parameters:: params (mala.common.parametes.Parameters) – Parameters used to create this neural network.

forward(x)[source]: Perform a forward pass through the network.

static generate_square_subsequent_mask(size)[source]

Generate a mask so that only the current / previous tokens are visible.

Parameters:: size (int) – size of the mask

init_weights()[source]

Initialise weights with a uniform random distribution.

Distribution will be in the range (-initrange, initrange).