Models

Module contents

Submodules

dashboard.models.emails module

Email notifications sent from models.py

Email functions used by models are likely to be submitted to the scheduler and so special care should be taken to ensure their input arguments are always JSON serializable types (see here for info on serializable types: https://docs.python.org/3/library/json.html#json.JSONEncoder).

Additionally, because these functions are needed within models.py, you can’t query the database within an email function without risking circular import issues. Any info needed to generate the email should therefore be provided as an argument when the function is called.

dashboard.models.emails.account_activation_email(oauth_uname, user_email, num_studies)

Notify a user that their dashboard access has been approved.

Parameters:

  • oauth_uname (str) – The OAuth provider account name that the new user will access the dashboard through.

  • user_email (str) – The email account associated with the user.

  • num_studies (int) – The number of studies this user has been granted access to.

dashboard.models.emails.account_rejection_email(user_id, user_email)

Notify a potential user that they will not be granted dashboard access.

Parameters:

  • user_id (int) – The id for the database record storing the user’s info.

  • user_email (str) – The email address to notify.

dashboard.models.emails.account_request_email(name)

Notify admins that a new user is requesting dashboard access.

Parameters:

name (str) – The real name of the user requesting an account

dashboard.models.emails.qc_notification_email(user_name, dest_email, study, current_tp, remain_tp=None)

Notify QCers that there is a new session to review.

Parameters:

  • user_name (str) – A user’s real name.

  • dest_email (str) – Email address to contact.

  • study (str) – The name of the study that has received data.

  • current_tp (str) – The name of the timepoint that needs review.

  • remain_tp (list of str) – A list of names of timepoints from this study that are still awaiting quality control.

dashboard.models.models module

Database models + relations

class dashboard.models.models.AccountRequest(user_id)

Bases: TableMixin, Model

approve()
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

reject()
user
user_id
class dashboard.models.models.AltStudyCode(**kwargs)

Bases: Model

code
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

property site
site_id
property study
study_id
study_site
property uses_redcap
class dashboard.models.models.Analysis(**kwargs)

Bases: Model

analysis_comments
description
id
name
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

software
class dashboard.models.models.AnalysisComment(**kwargs)

Bases: Model

analysis
analysis_id
comment
excluded
id
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scan
scan_id
user
user_id
class dashboard.models.models.AnonymousUser

Bases: PermissionMixin, AnonymousUserMixin

account_active = False
dashboard_admin = False
first_name = 'Guest'
get_disabled_sites()
get_sites()
get_studies()
id = -1
class dashboard.models.models.EmptySession(name, num, user_id, comment)

Bases: Model

This table exists solely so QCers can dismiss errors about empty sessions and comment on any follow up they’ve performed.

comment
date_added
name
num
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

reviewer
session
user_id
class dashboard.models.models.ExpectedScan(study, site, scantype, count=0, pha_count=0)

Bases: TableMixin, Model

count
pha_count
scantype
scantype_id
site_id
standards
study_id
class dashboard.models.models.GoldStandard(study, gs_json)

Bases: Model

expected_scan
id
json_contents
json_created
json_path
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scans = ObjectAssociationProxyInstance(AssociationProxy('scan_gold_standard', 'scan'))
site
study
tag
class dashboard.models.models.IncidentalFinding(user_id, timepoint_id, description)

Bases: Model

description
id
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

session
timepoint_id
timestamp
user
user_id
class dashboard.models.models.MetricValue(**kwargs)

Bases: Model

id
metrictype
metrictype_id
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scan
scan_id
property value

Returns the value field from the database. The value is stored as a string. If the value contains ‘::’ character this will convert it to a list, otherwise it will attempt to cast to Float. Failing that the value is returned as a string.

class dashboard.models.models.Metrictype(**kwargs)

Bases: Model

id
metric_values
name
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scantype
scantype_id
class dashboard.models.models.PermissionMixin

Bases: object

Adds methods needed for user authentication.

does_qc(study, site=None)
abstract get_disabled_sites()
abstract get_sites()
abstract get_studies()
has_study_access(study, site=None)
is_kimel_contact(study, site=None)
is_primary_contact(study, site=None)
is_study_RA(study, site=None)
is_study_admin(study, site=None)
class dashboard.models.models.PipelineScope(**kwargs)

Bases: Model

query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scope
class dashboard.models.models.RedcapConfig(project, instrument, url, version=None)

Bases: TableMixin, Model

comment_field
completed_field
property completed_value
date_field
event_ids
get_config(project=None, instrument=None, url=None, version=None, create=False)
id
instrument
project
records
redcap_version
session_id_field
token
url
user_id_field
class dashboard.models.models.RedcapRecord(record, config_id, date, redcap_version=None)

Bases: Model

comment
config
date
event_id
form_config
id
property instrument
property is_shared
property project
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

record
property redcap_version
sessions
property url
user
class dashboard.models.models.Scan(name, timepoint, repeat, series, tag, description=None, source_id=None)

Bases: TableMixin, Model

property active_gold_standard
add_bids(name)
add_checklist_entry(signing_user, comment=None, sign_off=None)
add_error(error_message)
add_json(json_file, timestamp=None)
analysis_comments
bids_name
blacklisted()
conv_errors
description
flagged()
get_checklist_entry()
get_comment()
get_header_diffs()
get_study(study_id=None)
property gold_standards
header_diffs
id
is_linked()
is_new()
is_outdated_header_diffs()

Reports whether an update to the header diffs is needed.

json_contents
json_created
json_path
length
list_children()
metric_values
name
qc_review
property qc_type
repeat
scantype
series
session
signed_off()
source_data
source_id
tag
timepoint
update_header_diffs(standard=None, ignore=None, tolerance=None)
class dashboard.models.models.ScanChecklist(scan_id, user_id, comment=None, approved=False)

Bases: TableMixin, Model

approved
comment
id
scan
scan_id
property timestamp
update_entry(user_id, comment=None, status=None)
user
user_id
class dashboard.models.models.ScanGoldStandard(scan_id, gold_standard_id, diffs, gold_version, scan_version, timestamp=None)

Bases: Model

date_added
diffs
gold_standard
gold_standard_id
gold_version
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

scan
scan_id
scan_version
property timestamp
class dashboard.models.models.Scantype(tag)

Bases: TableMixin, Model

expected_scans
metrictypes
pha_type
qc_type
scans
studies
tag
class dashboard.models.models.Session(name, num, date=None, signed_off=False, reviewer_id=None, review_date=None)

Bases: TableMixin, Model

add_redcap(record_num, date, project=None, url=None, instrument=None, config=None, rc_user=None, comment=None, event_id=None, redcap_version=None)
add_scan(name, series, tag, description=None, source_id=None)
add_task(file_path, name=None)
date
delete()

This will also delete anything referencing the current session (i.e. any scans, redcap comments, blacklist entries or dismissed ‘missing scans’ errors)

delete_scan(name)
empty_session
expects_notes(study=None)
get_blacklist_entries()

Returns all ScanChecklist entries for all blacklisted scans in this session.

get_expected_scans(study_id=None)
get_study(study_id=None)
is_new()
is_qcd()
kcni_name
missing_scans()
name
num
redcap_record
review_date
reviewer
reviewer_id
scans
sign_off(user_id)
signed_off
property site
task_files
tech_notes
timepoint
class dashboard.models.models.SessionRedcap(name, num, record_id=None)

Bases: Model

name
num
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

record
record_id
session
share_record(target_session)

Shares an existing redcap record with another session that represents an alternate ID for the original participant

class dashboard.models.models.Site(site_name, description=None)

Bases: TableMixin, Model

description
name
studies
timepoints
class dashboard.models.models.Study(study_id, full_name=None, description=None, read_me=None, is_open=None)

Bases: TableMixin, Model

add_gold_standard(gs_file)
add_pipeline(pipeline_key, view, name, scope)
add_timepoint(timepoint)
choose_staff_contact()
delete()
delete_scantype(site_id, scantype)
delete_site(site)
description
email_qc
get_QCers()
get_RAs(site=None, unique=False)

Get a list of all RAs for the study, or all RAs for a given site.

The ‘unique’ flag can be used to ensure RAs are only in the list once if they happen to be an RA for multiple sites. Pretty much only useful for printing the list.

get_blacklisted_scans()
get_flagged_scans()
get_missing_redcap()

Returns a list of all non-phantoms for the current study that are expecting a redcap survey but dont yet have one.

get_missing_scans()

Returns a list of session names + repeat numbers for sessions in this study that have a scan completed survey but no scan data yet. Excludes session IDs that have explicitly been marked as never expecting data

get_new_sessions()
get_pipelines(scope=None)
get_primary_contacts()
get_qced_scans()
get_sessions_using_redcap()

Returns a query that will find all sessions in the current study that expect a redcap survey (whether or not we’ve already received one for them)

Doing this manually rather than letting SQLAlchemy handle it behind the scenes because it was firing off one query per session every time a study page loaded. So it’s way uglier, but way faster. Sorry :(

get_staff_contacts()
get_tag_counts(site, pha=False)

Return the number of scans expected for each tag.

Parameters:

  • site (str) – The name of a site for this study.

  • pha (bool, optional) – Whether to look up the number expected for phantoms.

id
is_open
name
num_timepoints(type='')
outstanding_issues()
read_me
scantypes
select_next(user_list)

Selects the next user from a study. This can be used to choose the next QC-er to assign, or the next staff member to assign to an issue, etc.

Current strategy is to pick a random person. This may be changed in the future.

sites
standards
timepoints
update_scantype(site_id, scantype, num=None, pha_num=None, create=False)

Update the number of scans expected for each site / tag combination.

Parameters:

  • site_id (str or Site) – The site to configure.

  • scantype (str or Scantype) – The scan type to configure.

  • num (int, optional) – The number of scans expected with this tag for human subjects at this site. Only updates if value given. Defaults to None.

  • pha_only (int, optional) – The number of scans with this tag expected for phantoms at this site. Only updates if value given. Defaults to None.

  • create (bool, optional) – Whether to add a new record for the study/site/tag combination if one doesnt exist.

Raises:

InvalidDataException – If a site is given that isn’t configured for this study, if the scan type does not exist, or if an error occurs during database update.

update_site(site_id, redcap=None, notes=None, code=None, xnat_archive=None, xnat_convention='KCNI', xnat_credentials=None, xnat_url=None, create=False)

Update a site configured for this study (or configure a new one).

Parameters:

  • site_id (str or Site) – The ID of a site associated with this study or its record from the database.

  • redcap (bool, optional) – True if redcap scan completed records are used by this site. Updates only if value provided. Defaults to None.

  • notes (bool, optional) – True if tech notes are provided by this site. Updates only if value provided. Defaults to None.

  • code (str, optional) – The study code used for IDs for this study and site combination. Updates only if value provided. Defaults to None.

  • xnat_archive (str, optional) – The name of the archive on XNAT where data for this site is stored. Updates only if value provided. Defaults to None.

  • xnat_convention (str, optional) – The naming convention used on the XNAT server for this site. Defaults to ‘KCNI’.

  • xnat_credentials (str, optional) – The full path to the credentials file to read when accessing this site’s XNAT archive. Defaults to None.

  • xnat_url (str, optional) – The URL to use when accessing this site’s XNAT archive. Defaults to None.

  • create (bool, optional) – Whether to create the site and add it to this study if it isnt already associated. Defaults to False.

Raises:

InvalidDataException – If the site doesnt exist or isn’t associated with this study (and create wasnt given) or if the update fails.

users
class dashboard.models.models.StudyPipeline(**kwargs)

Bases: TableMixin, Model

Define QC pipelines enabled for a study.

name
pipeline_id
scope
study_id
view
class dashboard.models.models.StudySite(study_id, site_id, uses_redcap=False, uses_notes=None, code=None)

Bases: TableMixin, Model

alt_codes
code
download_script
expected_scans
post_download_script
site
site_id
study
study_id
users
uses_notes
uses_redcap
xnat_archive
xnat_convention
xnat_credentials
xnat_url
class dashboard.models.models.StudyUser(study_id, user_id, site_id=None, admin=False, is_primary_contact=False, is_kimel_contact=False, is_study_RA=False, does_qc=False)

Bases: Model

does_qc
id
is_admin
kimel_contact
primary_contact
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

site_id
study
study_RA
study_id
user
user_id
class dashboard.models.models.TableMixin

Bases: object

Adds simple methods commonly needed for tables.

delete()
save()
class dashboard.models.models.TaskFile(timepoint, repeat, file_path, file_name=None)

Bases: Model

file_name
file_path
id
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

repeat
session
timepoint
class dashboard.models.models.Timepoint(name, site, is_phantom=False)

Bases: TableMixin, Model

accessible_study(user)

Returns a study that the timepoint belongs to and the user has access to if one exists.

add_bids(name, session)
add_comment(user_id, text)
add_session(num, date=None)
belongs_to(study)

Returns true if the data in this Timepoint is considered part of the given study

bids_name
bids_session
comments
delete()

This will cascade and also delete any records that reference the current timepoint, so be careful :)

delete_comment(comment_id)
dismiss_redcap_error(session_num)
expects_notes(study=None)
expects_redcap(study=None)
get_blacklist_entries()

Returns any ScanChecklist entries for blacklisted scans for every session belonging to this timepoint.

get_comment(comment_id)
get_study(study_id=None)

Most timepoints only ever have one study and this will just return the first one found. If ‘id’ is given it will either return the study object or raise an exception if this timepoint doesnt belong to that study

ignore_missing_scans(session_num, user_id, comment)
incidental_findings
is_phantom
is_qcd()
kcni_name
missing_scans()
name
needs_redcap_survey(study_id)
report_incidental_finding(user_id, comment)
property reviewer

Returns the name of the first session’s qc reviewer as this timepoint’s reviewer

sessions
site
site_id
studies
update_comment(user_id, comment_id, new_text)
class dashboard.models.models.TimepointComment(timepoint_id, user_id, comment)

Bases: Model

comment
id
modified
query: t.ClassVar[Query]

A SQLAlchemy query for a model. Equivalent to db.session.query(Model). Can be customized per-model by overriding query_class.

Warning

The query interface is considered legacy in SQLAlchemy. Prefer using session.execute(select()) instead.

timepoint
timepoint_id
property timestamp
update(new_text)
user
user_id
class dashboard.models.models.User(first, last, username=None, provider='github', email=None, position=None, institution=None, phone=None, ext=None, alt_phone=None, alt_ext=None, picture=None, dashboard_admin=False, account_active=False)

Bases: PermissionMixin, UserMixin, TableMixin, Model

property account_provider
add_studies(study_ids)

Enable study access for this user.

This will add a StudyUser record with the default permissions for each given study. Restrict access to specific sites by mapping the study ID to a list of allowed site names. An empty list means ‘grant the user access to all sites’.

For example:

study_ids = {‘OPT’: [‘UT1’, ‘UT2’],

‘PRELAPSE’: []}

This would grant the user access to two sites (UT1, UT2) in OPT and all sites in PRELAPSE.

Parameters:

study_ids (dict) – A dictionary of study IDs to grant user access to. Each study ID key should map to a list of sites to enable. Use the empty list to indicate global study access.

Raises:

InvalidDataException – If the format of arg study_ids is incorrect or one or more records cannot be added.

alt_ext
alt_phone
analysis_comments
dashboard_admin
email
ext
first_name
get_disabled_sites()

Get a dict of study IDs mapped to sites this user cant access

Returns:

A dictionary mapping each study ID to a list of its sites that this user does not have access to. Studies where the user has full access will be omitted entirely.

Return type:

dict

get_sites()

Get a list of sites the user has even partial access to.

Caution: If using this to determine data access you must still restrict by study.

Returns:

A list of string site names.

Return type:

list

get_studies()

Get a list of studies that user has even partial access to

Returns:

A list of Study objects, one for each study where the user has at least partial (site based) access.

Return type:

list

id
incidental_findings
institution
is_active
last_name
num_requests()

Returns a count of all pending user requests that need to be reviewed

pending_approval
phone
picture
position
remove_studies(study_ids)

Disable study access for this user.

This removes any StudyUser records that match the given study IDs. Access to specific sites within a study can be disabled by mapping the study ID to a list of site names. An empty list means ‘disable user access to this study completely’.

For example:

study_ids = {‘OPT’: [‘UT1’],

‘PRELAPSE’: []}

This would remove user access to site ‘UT1’ for OPT and totally disable user access to PRELAPSE

Parameters:

study_ids (dict) – A dictionary of study IDs to disable access to. Each ID should map to a list of sites to disable. The empty list indicates complete access restriction

Raises:

InvalidDataException – If the format of arg study_ids is incorrect or one or more records cannot be removed.

request_account(request_form)
scan_comments
sessions_reviewed
studies
timepoint_comments
update_avatar(url=None)
update_username(new_name, provider='github')
property username

dashboard.models.utils module

Functions and classes for use in dashboard.models

class dashboard.models.utils.DictListCollection(key)

Bases: KeyFuncDict

Allows a relationship to be organized into a dictionary of lists

list_mod()

Allows sqlalchemy manage changes to the contents of the lists

dashboard.models.utils.async_xnat_update(xnat_url, user, password, xnat_archive, exp_name, series_num, comment, quality)

Push usability data into XNAT.

This will update XNAT in a separate thread to reduce waiting time for the user. Because it will run in another thread, database objects cannot be passed in as arguments.

Parameters:

  • xnat_url (str) – The full URL to use for the XNAT server.

  • user (str) – The user to log in as.

  • password (str) – The password to log in with.

  • xnat_archvie (str) – The name of the XNAT archive that contains the experiment.

  • exp_name (str) – The name of the experiment on XNAT.

  • series_num (int) – The series number of the file to update.

  • comment (str) – The user’s QC comment.

  • quality (str) – The quality label to apply based on whether data has been flagged, blacklisted or approved.

dashboard.models.utils.file_timestamp(file_path)
dashboard.models.utils.get_software_version(json_contents)
dashboard.models.utils.get_xnat_credentials(site_settings, app_config)

Retrieve the xnat username and password for a given study/site.

Parameters:

  • site_settings (dashboard.models.StudySite) – A StudySite record

  • app_config (dict) – Configuration of the current app instance, as retrieved from current_app.config

dashboard.models.utils.read_json(json_file)
dashboard.models.utils.schedule_email(email_func, input_args, input_kwargs=None)

Send an email from the server side.

If an automated email is fired from code that may be run on the client side then it should be wrapped by this function. This will add a scheduler job that fires instantly to ensure it executes server side (or just send it if it’s already server side).

NOTE: We’re using scheduler.add_job directly and not dashboard.monitors.add_monitor because monitors often need classes from the models and using add_monitor would introduce circular dependencies.

dashboard.models.utils.update_xnat_usability(scan, app_config)

Update XNAT usability data for a scan.

Parameters:

  • scan (dashboard.models.Scan) – The series to push QC data for.

  • app_config (dict) – Configuration of the current app instance, as retrieved from current_app.config