dqa.connectors.importer.PickleImporter#

class dqa.connectors.importer.PickleImporter(filename: str, **kwargs)#

Imports a dataset dictionary from a pickle file.

This complements the dqa.connectors.exporter.PickleExporter class.

Parameters:

filename (str) – Name of the input file.

Methods

filter_filenames(files, start, end, step)

For a list of available filenames, take a subset specified by start, end, step.

filter_filenames_regex(files)

Filter filenames by the regular expression specified in the constructor.

finish()

Can perform actions that are required to clean up after the task has finished, e.g.

get_import_parameters(folder_index)

Returns filename, folder_name, start, end, step for the folder with index folder_index.

get_start_end_step(start, end, step, ...)

Adjusts start, end, step if the num_files parameter is set.

import_file(folder, filename, parameters, ...)

Imports a single file.

import_files(folder, filenames, parameters)

Imports all files given by a list of filenames.

import_folder(folder[, start, end, step, ...])

Imports all files from a folder.

list_folder(folder)

Lists all files in a folder. By default, this lists the files on the file system that are relevant (by checking file ending or regex pattern). For other purposes this could also be used, for example, to list the contents of a database. :param folder: The name of the folder to be listed. :return: A tuple containing: - A list of all relevant filenames. - A list containing a dictionary of paramters for each filename. By default, these are obtained by parsing the named groups in the filename regex.

import_file2

in_out_default

input_output_dataset

input_output_machine

input_output_mode

input_output_name

list_batch_ids

list_total

log

modify_data_row

modify_dataset

modify_dataset_dict

modify_machine

modify_measurement

set_logging_level

transfer_metadata