What Data can be Migrated?

Here is an overview of data that can be migrated:

The types of customer data which can be migrated include the following

Volunteer Data

This is the information about the volunteer, such as contact, and medical history.

Appointments

Past or future appointments.

Study Participation

Summary info about a volunteer’s previous study participation. This is extremely limited, does not include clinical data, and is meant to support advanced volunteer search.

Configuration

Configurations required to support migrated data include initialization of lists of supported medications, substance uses, and medical conditions.

Often these appear in source systems as data drop-downs.

What are the mappings required to migrate any source data into ClinSpark?

Some data maps cleanly into ClinSpark, like DOB. Any transformations required must be noted in the specification, such as from source system race definitions to those categories used in ClinSpark.

An example is volunteer phone data. Migration is an opportunity to clean source data. We need very explicit instructions with example errors on the source side, and a definition of what transformations are requested.

Another common example is medical history. Substance use data for instance frequently needs to be “cleaned” so that it can be transformed into the ClinSpark representations. To document these transformations between data representations in the source DB and the desired representation in ClinSpark, we suggest using a spreedsheet or alternative method of presenting tabular data.

This approach is ideal since it can be ingested programmatically by the transformation program.

Migration Specification

This document identifies the places in ClinSpark which can receive migrated customer data. For each type of data, it is the customer’s responsibility to identify where in their source system this data can be found.

This document is intended to be used as a worksheet, specifying all migration work which must be done. Any work to be performed by IQVIA engineering must first be clearly specified in this document. Upon completion, this document will serve as a record of migration work which has been performed, and as a guide for user testing and validation.

Collaborative Document

The specification document is collaboratively filled out over the course of the migration process. It is not unusual to see multiple editors working on different sections of the document. And a critical part of the workflow is inline comments which can be assigned to readers and support threaded discussions about particular portions of the document.

To support this, the specification itself needs to be on a collaboration platform. We may be able to offer e.g. Teams / Sharepoint / Excel as a collaboration platform to certain customers, and if not, we may be able to use yours.

Specification Example

The following example specification has been designed to be read by a script during the migration process…

Specification Sections

The specification is broken up into sections (tabs in the workbook), where each section is a different place in the ClinSpark UI that can potentially receive volunteer information.

For each section, the following areas must be included:

Migration Specification

List the following:

Source System Table/Screen
Source System Column/Atttibute
CS Table/Screen
CS Column/Atttibute
Migrate (Y/N)?
Status (Dev Ready/Dev Complete/Validated)
Migration Instructions/Rules
Comments

Then for each data type (Medication Form, Medication Target, Medication Frequency etc.) all the possible options for Source must be listed against their intended target.

Screenshots

Screenshots of target and source should not normally be needed unless we run into difficulties…

SQL

It will not normally be needed to provide the SQL for each field in case we run into difficulties…

If required, provide the SQL in a textual form which can be copied and later executed by engineers as examples. Provide the execution output showing the test data highlighted in this section’s example to demonstrate it is correct.

Iterations of specifications and implementations

Each iteration starts with the completion of the dedicated specification document. The first iteration tends to be by far the biggest, since this is where the full scope of the migration is spec’d out and a first comprehensive implementation is attempted.

The output of this implementation is then reviewed by the customer. Any gaps, errors or other changes are identified in the review of the iteration deliverable. And then all of these are then updated in the specification document for iteration 2.

The specification changes for later iterations should focus on particular issues, and they should clearly articulate the change and most importantly, reference test cases with specific examples from the test data.

Data Without an Existing Field in ClinSpark

Frequently a source system will have a field for volunteer data which doesn’t have an explicit field in ClinSpark.

For these one option is to create within ClinSpark custom volunteer questions, and migrate the source data into these questions. For example here are some custom questions from a demo system:

Project Tracking

In addition to the specification, each migration effort will be managed and tracked (questions, issues, decisions etc.) in a Basecamp sub-project…