multi_lazy_load_data_loader

Class for loading multiple data sets with pre-fetching.

class MultiLazyLoadDataLoader(datasets, **kwargs)

Bases: object

Class for loading multiple data sets with pre-fetching.

Parameters:

datasets (list) – A list of data sets to be used in this class.

cleanup()

Deallocate arrays still left in memory.

static load_snapshot_to_shm(snapshot, descriptor_calculator, target_calculator, input_shm_name, output_shm_name)

Load a snapshot into shared memory.

Limited to numpy files for now.

Parameters:

• snapshot (mala.datahandling.snapshot.Snapshot) – Snapshot to be loaded.

• descriptor_calculator (mala.descriptors.descriptor.Descriptor) – Used to do unit conversion on input data.

• target_calculator (mala.targets.target.Target or derivative) – Used to do unit conversion on output data.

• input_shm_name (str) – Name of the input shared memory buffer.

• output_shm_name (str) – Name of the output shared memory buffer.