datman.exporters module

Functions to export data into different file formats and organizations.

To allow datman to export to a new format make a subclass of SessionExporter or SeriesExporter depending on whether the new format requires data from a complete scan session or a single series, respectively. The new subclass should implement all abstract methods, including ‘export’ which does the actual work of generating outputs.

Also, ensure that subclasses define the ‘type’ attribute to be a short unique key that can be referenced in config files (e.g. ‘nii’).

class datman.exporters.BidsExporter(config, session, experiment, bids_opts=None, **kwargs)[source]

Bases: datman.exporters.SessionExporter

An exporter that runs dcm2bids.

add_repeat_num()[source]

export(raw_data_dir, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

get_sidecars()[source]

needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

type = 'bids'

class datman.exporters.DBExporter(config, session, experiment, **kwargs)[source]

Bases: datman.exporters.SessionExporter

Add a datman-style session and its contents to datman’s QC dashboard.

add_tech_notes(session)[source]

Add the path to a scan session’s tech notes to the database.

Parameters: session (dashboard.models.Session) – A valid QC dashboard scan session.

errors_outdated(scan, fname)[source]

export(*args, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

get_bids_name(dm_name, session)[source]

Get BIDS style scan name from a datman style nifti.

Returns

A valid bids style file name or an empty string if one: cannot be found.

Return type

str

classmethod get_output_dir(session)[source]: Retrieve the exporter’s output dir without needing an instance.

make_scan(file_stem)[source]

Add a single scan to datman’s QC dashboard.

Parameters: file_stem (str) – A valid datman-style file name.

make_session()[source]

Add the current session to datman’s QC database.

Returns: The created scan session or None.
Return type: dashboard.models.Session

property names

Gets list of valid datman-style scan names for a session.

Returns

A dictionary of datman style scan names mapped to: the bids style name if one can be found, otherwise, an empty string.

Return type

dict

needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

type = 'db'

class datman.exporters.DcmExporter(output_dir, fname_root, echo_dict=None, dry_run=False, **kwargs)[source]

Bases: datman.exporters.SeriesExporter

Export a single dicom from a scan.

export(raw_data_dir, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

ext = '.dcm'

type = 'dcm'

class datman.exporters.Exporter[source]

Bases: abc.ABC

An abstract base class for all Exporters.

abstract export(raw_data_dir, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

classmethod get_output_dir(session)[source]: Retrieve the exporter’s output dir without needing an instance.

make_output_dir()[source]

Creates the directory where the Exporter’s outputs will be stored.

Returns: True if directory exists (or isn’t needed), False otherwise.
Return type: bool

abstract needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

abstract outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

type = None

class datman.exporters.NiiExporter(output_dir, fname_root, echo_dict=None, dry_run=False, **kwargs)[source]

Bases: datman.exporters.SeriesExporter

Export a series to nifti format with datman-style names.

export(raw_data_dir, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

ext = '.nii.gz'

move_file(gen_file)[source]

Move the temp outputs of dcm2niix to the intended output directory.

Parameters: gen_file (str) – The full path to the generated nifti file to move.

report_issues(stem, messages)[source]

Write an error log if dcm2niix had errors during conversion.

Parameters

stem (stem) – A valid datman-style file name (minus extension).
messages (str) – Error messages to write.

type = 'nii'

class datman.exporters.NiiLinkExporter(config, session, experiment, **kwargs)[source]

Bases: datman.exporters.SessionExporter

Populates a study’s nii folder with symlinks pointing to the bids dir.

belongs_to_session(nifti_path)[source]

Check if a nifti belongs to this repeat or another for this session.

Parameters

nifti_path (str) – A nifti file name from the bids folder (minus extension).

Returns

True if the nifti file belongs to this particular: repeat. False if it belongs to another repeat.

Return type

bool

clear_errors(dm_file)[source]

Remove an error file from a previous BIDs export issue.

Parameters: dm_file (str) – A valid datman file name.

export(*args, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

ext = '.nii.gz'

get_bids_niftis()[source]

Get all nifti files from a BIDS session.

Returns

A list of full paths (minus the file extension) to: each bids format nifti file in the session.

Return type

list

get_dm_names()[source]

Get the datman-style scan names for an entire XNAT experiment.

Returns

A list of datman-style names for all scans found: for the session on XNAT.

Return type

list

get_error_file(dm_file)[source]

classmethod get_output_dir(session)[source]: Retrieve the exporter’s output dir without needing an instance.

get_series_num(side_car)[source]

Find the correct series number for a scan.

Most JSON side car files have the correct series number already. However, series that are split during nifti conversion (e.g. FMAP-AP/-PA) end up with one of the two JSON files having a modified series number. This function will default to the XNAT series number whenever possible, for accuracy.

Parameters: side_car (dict) – A dictionary containing the contents of a scan’s JSON side car file.
Returns: The most accurate series number found for the scan.
Return type: str

make_datman_name(bids_path, scan_tag)[source]

Create a Datman-style file name for a bids file.

Parameters

bids_path (str) – The full path (+/- extension) of a bids file to create a datman name for.
scan_tag (str) – A datman style tag to apply to the bids scan.

Returns

A valid datman style file name (minus extension).

Return type

str

make_link(dm_file, bids_file)[source]

Create a symlink in the datman style that points to a bids file.

Parameters

dm_file (str) – A valid datman file name.
bids_file (str) – The full path to a bids file (minus extension.)

match_dm_to_bids(dm_names, bids_names)[source]

Match each datman file name to its BIDS equivalent.

Parameters

dm_names (list) – A list of all valid datman scan names found for this session on XNAT.
bids_names (list) – A list of all bids files (minus extensions) that exist for this session.

Returns

A dictionary matching the intended datman file name to: the full path (minus extension) of the same series in the bids folder. If no matching bids file was found, it will instead be matched to the string ‘missing’.

Return type

dict

needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

report_errors(dm_file)[source]

Create an error file to report probable BIDS conversion issues.

Parameters: dm_file (str) – A valid datman file name.

type = 'nii_link'

class datman.exporters.SeriesExporter(output_dir, fname_root, echo_dict=None, dry_run=False, **kwargs)[source]

Bases: datman.exporters.Exporter

A base class for exporters that take a single series as input.

ext = None

needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

class datman.exporters.SessionExporter(config, session, experiment, dry_run=False, **kwargs)[source]

Bases: datman.exporters.Exporter

A base class for exporters that take an entire session as input.

Subclasses should override __init__ (without changing basic input args) and call super().__init__(config, session, experiment, **kwargs).

The init function for SessionExporter largely exists to define expected input arguments and set some universally needed attributes.

class datman.exporters.SharedExporter(config, session, experiment, bids_opts=None, **kwargs)[source]

Bases: datman.exporters.SessionExporter

Export an XNAT ‘shared’ experiment.

export(*args, **kwargs)[source]

Exports raw data to the current Exporter’s format.

Parameters: raw_data_dir (str) – The directory that contains the downloaded raw data.

ext = '.nii.gz'

find_bids_files(name_map=None)[source]

Find all bids files that have been created for the source session.

Parameters

name_map (dict, optional) – A dictionary that may contain other discovered files and their output names. Default None.

Returns

A dictionary of source session files that have been: found, mapped to their full path under the shared/alias ID.

Return type

dict

find_dm_files(dir_type, name_map=None)[source]

Find datman-style source files in all listed directory types.

Parameters

dir_type (list) – A list of paths on the source session to search through. All entries should be valid path types defined for the datman.scan.Scan object.
name_map (dict, optional) – A dictionary of other discovered source files mapped to their aliases. Default None.

Returns

A dictionary of source session files that have been: found, mapped to their full path under the shared/alias ID.

Return type

dict

find_resource_files(name_map=None)[source]

Find all source session resources files.

Parameters

name_map (dict, optional) – A dictionary of any previously found source files mapped to their aliases.

Returns

A dictionary of source session files that have been: found, mapped to their full path under the shared/alias ID.

Return type

dict

find_source_session(config, experiment)[source]

Find the original data on the filesystem.

Parameters

config (datman.config.config) – The datman config object for the study that the shared experiment belongs to.
experiment (datman.xnat.XNATExperiment) – The experiment object for the shared session on XNAT.

Returns

The scan object for the source dataset: as previously exported to the filesystem.

Return type

datman.scan.Scan

classmethod get_output_dir(session)[source]: Retrieve the exporter’s output dir without needing an instance.

make_name_map(dm_dirs, use_bids=False)[source]

Create a dictionary of source files to their ‘shared’ alias.

Parameters

dm_dirs (list) – A list of datman-style paths on the source session’s datman.scan.Scan object to search for files.
use_bids (any, optional) – Whether or not to search for bids files. Any input equivalent to boolean True will be taken as ‘True’. Default False.

Returns

A dictionary mapping the full path to each discovered source: file to the full path to the output alias name/symlink.

Return type

dict

needs_raw_data()[source]

Whether raw data must be downloaded for the Exporter.

Returns

True if raw data must be given, False otherwise. Note that: False may be returned if outputs already exist.

Return type

bool

outputs_exist()[source]

Whether outputs have already been generated for this Exporter.

Returns: True if all expected outputs exist, False otherwise.
Return type: bool

type = 'shared'

datman.exporters.get_exporter(key, scope='series')[source]

Find an exporter class for a given key identifier.

Parameters

key (str) – The ‘type’ identifier of a defined exporter (e.g. ‘nii’).
scope (str, optional) – Whether to search for a series or session exporter. Defaults to ‘series’.

Returns

The Exporter subclass for the type,: if one is defined, or else None.

Return type

datman.exporters.Exporter