Migration Specification

This document identifies the places in ClinSpark which can receive migrated customer data. For each type of data, it is the customer’s responsibility to identify where in their source system this data can be found.

This document is intended to be used as a worksheet, specifying all migration work which must be done. Any work to be performed by FH engineering must first be clearly specified in this document. Upon completion, this document will serve as a record of migration work which has been performed, and as a guide for validation.

About the Specification and Process

Collaborative Document

The specification document is collaboratively filled out over the course of the migration process. It is not unusual to see multiple editors working on different sections of the doc. And a critical part of the workflow is inline comments which can be assigned to readers and support threaded discussions about particular portions of the doc.

To support this, the spec itself needs to be on a collaboration platform. Foundry Health can provide the spec using the Foundry G Suite docs app, which meets this requirement. If for some reason this is not acceptable to your organization, if there’s another collaborative platform such as Office 365, then it is fine to use that. As long as all collaborators have access to a commenting-enabled collaborative platform things will work fine.

Iterations of specifications and implementations

Each iteration starts with a dedicated specification document. The first iteration tends to be by far the biggest, since this is where the full scope of the migration is spec’d out and a first comprehensive implementation is attempted.

...

The spec documents for later iterations do not need to adhere to the format of the primary specification. They tend to be focused on particular issues, and they should clearly articulate the change and most importantly, show test cases with specific examples from the test volunteers.

Spec Design

Spec Templates

You can download the Migration Spec Template as a MS Word document here.

This should be copied to a collaborative platform such as G Suite (Foundry can provide this) or one chosen by the customer.

About the Requirements

What Data can be Migrated

Here is an overview of data that can be migrated:

...

Often these appear in source systems as data drop downs.

Specification Sections

The specification is broken up into sections, where each section is a different place in the ClinSpark UI that can potentially receive volunteer information.

For each section, the following areas must be documented:

ClinSpark UI

You’ll see that this is already populated in the specification. A portion of volunteer data will be taken from the ClinSpark UI, and annotated with numbered dots which identify individual fields.

...

This defines the scope of this section of the spec. The remainder of this section will cover other details about these fields.

Customer Source UI

For the numbered dots defined above in ClinSpark, show where in the source UI this data lives.

...

Trace Fields to the Customer Source DB

For each field we need to know how the data shown in the source UI is represented in the source DB.

...

To demonstrate that this is correct, provide the sql in a textual form which can be copied and later executed by engineers as examples. Provide the execution output showing the test data highlighted in this section’s example to demonstrate it is correct.

...

What are the mappings required to migrate any source data into ClinSpark?

Some data maps cleanly into ClinSpark, like DOB. Any transformations required must be noted in the specification, such as from source system race definitions to those categories used in ClinSpark.

...

Another common example is medical history. Substance use data for instance frequently needs to be “cleaned” so that it can be transformed into the ClinSpark representations. To document these transformations between data representations in the source DB and the desired representation in ClinSpark, we suggest using a google sheet. Here is an example Customer Medical History Mapping Definition Template. This approach is ideal since it can be ingested programmatically by the transformation program.

Questions, Issues, Decisions

Place topics for discussion here. Add comments and assign them as appropriate to members of either the Foundry or customer team.

Implementation Evidence

Once implementation is done by Foundry Health, we will place screenshots showing what this looks like here.

Data Without an Existing Field in ClinSpark

Frequently a source system will have a field for volunteer data which doesn’t have an explicit field in ClinSpark.

...