© 2024 IQVIA - All Rights Reserved

Data Cleaning Progress (Report)

Owned by Toma Holtzer
Last updated: 21-Aug-2023
16 min read

Data cleaning is the process of ensuring that all data is entered and correct. It also covers review of that data and ultimately the locking of cleaned data.  This report demonstrates the progress of data cleaning as it takes place, particularly as it relates to completed data fields, and review of that data.

If this report is not available in a given customer environment, it can be made available upon request via service desk ticket.

 

Summary

Requirements

#

Requirement

User Story

Acceptance Criteria

Jira Issue

Notes

#

Requirement

User Story

Acceptance Criteria

Jira Issue

Notes

1

Display progress breakdown

As a data manager, I want to see a breakdown of data cleaning progress by subject and item group level, because this provides me with a logical breakdown of data still outstanding or under review.

Display progress of data cleaning at both subject and item group level

FHSD-942

Subject view shows things at a ‘birds eye’ view, but doesn’t show what is outstanding.
Item group view breaks down what data is actually ‘outstanding’ and requiring review.

2

Review progress per subject

As a data manager, I want to see the progress of data cleaning per subject, so get an understanding of how far along each subject is with data collection and data review.

Track the percentage of completed data
Track review progress from one-to-many reviewers, including review by a given department.For example, monitors or data managers.

Track percentage of locked data per subject
Track subject lock status, including lock date, when applicable
Track PI signoff status (Y/N) and date of signoff, when applicableDriven by use of ‘PISIGN’ alias on item intended to track PI signature.

FHSD-942

Alias information for the PI Signature item:
Alias Name: PISIGN
Alias Context: does not matter how context is defined

Design

The report will be generated as an XLS file containing two tabs, containing subject level and item level details (respectively).

Filters

The Filters allow to include Randomized Subjects only or all Subjects.

  • Only Subjects on active Cohort Assignments will be taken into account.

  • Randomized Subjects need to be on an active Cohort Assignment on a “Randomization” type Cohort.

It is also possible to select one or more Subjects from the list.

Example Reports

Data Cleaning Progress Report Example:

Data Review Locked Status Report:

Role Dependencies

Certain areas of this report are dependent on strict Role definitions and make an assumption about how certain Roles are named in customer environments. Certain data may not be produced in these sections if the Role names do not match what is defined in the reporting configuration logic.

The standard report logic looks for the following Roles:

  • Data Manager or DM for data review

  • Monitor or CRA for data review

  • PI for the “PI Signature” fields on the Subject tab

If customers want to utilize this report but follow different or additional naming conventions for these Roles, they should reach out to Foundry Health via service desk ticket for assistance.

Data Mapping

TAB 1 - ‘Subject Level’ Tab

The subject level tab is intended to track data cleaning progress on a per-subject basis.  For each subject who has participated in the trial, the following status fields help to track progress toward clean data:

  • Subject Status

  • Completed Item Group Data (Count and %)

  • Canceled Item Group Data

  • User Review of Item Group Data (Count and %)

  • Data Management Review of Item Group Data (Count and %)

  • Monitor Review of Item Group Data (Count and %)

  • Locked Item Group Data (Count and %)

  • Subject Lock Status and Date

  • PI Signature Status and Date

Study Name

Name of the study displayed to users when navigating ClinSpark.  Defined in the Study > Configure component.

ClinSpark

Report

Protocol Name

Protocol name of the study reported during data exporting.  Defined in the Study > Configure component.

ClinSpark

Report

Volunteer ID

This is the unique identifier for each volunteer that follows the volunteer across studies.

ClinSpark

Report

Randomization Number

Randomization (or “enrollment”) number is the subject number under which a subject has dosed or otherwise enrolled into the trial.  These numbers are established within ‘Randomization’ style cohorts in the Study > Configure component.

ClinSpark

Report

Lead In Number

Lead-In number is the subject number under which a subject has participated in Day -1 of the trial.  These numbers are established within ‘Lead-In’ style cohorts in the Study > Configure component.

ClinSpark

Report

Screening Number

Screening number is the subject number under which a subject has screened for a trial.  These numbers are established within ‘Screening’ style cohorts in the Study > Configure component.

ClinSpark

Report

Subject Status

Subject status tracks the state or outcome of a subject’s participation in a trial.  Options include Active, Completed, Terminated, and Withdrawn.  These statuses are established within the Study > Subjects component. 

ClinSpark

Report

Locked Subject (Y/N)

This field tracks whether data for an entire subject has been locked.  Typically, subjects are locked as one of the final steps in data cleaning.  Once a subject is locked, none of that subject’s data may be edited.  Locked subjects are displayed with a lock icon next to the ‘subject’ hyperlink in the Study > Data component.

ClinSpark

Report

Existing Item Group Data

Existing item group data is a count of all item groups associated as part of a subject’s database, regardless of the status of the item group (i.e. unsaved, incomplete, complete, etc.).  Within the Study > Data component, item groups are separated by light/dark shading.

ClinSpark

Report

Completed Item Group Data

Count of completed item groups for each subject.  An item group is considered complete once all mandatory data fields have been entered and are conformant.  In the Study > Data component, complete item groups are marked with a green check box.

ClinSpark

Report

Cancelled Item Group Data

Count of canceled item groups for each subject.  An item group may be canceled if it is redundant or not needed for any other reason.  Canceled data is excluded from progress % calculations.  In the Study > Data component, canceled item groups are marked with a red circle with a line through it.

Please note that the Item Groups do not inherit the Cancelled Status of a Form. The Item Group will need to be cancelled individually.

ClinSpark

Report

Item Group Data Progress %

Progress percent is defined as the percentage of existing item group data fields (excluding cancelled Item groups) that are marked as complete.  The formula for this calculation is as follows:  

Progress % = (Completed Item Group Data) / (Existing Item Group Data - Cancelled Item Group Data) 

ClinSpark

N/A

Report 

Locked Item Group Data

This field tracks the count of item groups for each subject which have been individually locked.  Once an item group has been locked, the contents of the item group may no longer be edited.  Item groups which have been locked display with a ‘locked’ icon in the Study > Data module.

ClinSpark

Report

Locked Item Group Data %

This field tracks the percentage of existing item groups that have been individually locked.  The formula for this calculation is as follows:

Lock % = (Locked Item Group Data) / (Existing Item Group Data - Cancelled Item Group Data)

ClinSpark

N/A

Report 

DM Reviewed Item Group Data

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks the number of item groups that have been reviewed by any user with a ‘DM’ or ‘Data Manager’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details. 

ClinSpark

Report

DM Reviewed Item Group Data %

This field tracks the percentage of existing item groups that have been reviewed by any user with a ‘DM’ or ‘Data Manager’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details. The formula for this calculation is as follows:

Review % = (Reviewed Item Group Data by DM Users) / (Existing Item Group Data - Cancelled Item Group Data)

ClinSpark

N/A

Report 

CRA Reviewed Item Group Data

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks the number of item groups that have been reviewed by any user with a ‘CRA’ or ‘Monitor’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

CRA Reviewed Item Group Data %

This field tracks the percentage of existing item groups that have been reviewed by any user with a ‘CRA’ or ‘Monitor’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details. The formula for this calculation is as follows:

Review % = (Reviewed Item Group Data by CRA Users) / (Existing Item Group Data - Cancelled Item Group Data)

ClinSpark

N/A

Report

PI Signature (Y/N)

This field tracks whether a PI has signed off on study data for a given subject.  The logic for registering ‘Yes’ for a PI signoff is as follows:

  • The applicable ‘PI Signoff’ item that registers a PI’s acknowledgement of data approval contains an alias with the following defined field:

    • Name = PISIGN

    • Context is ignored for this alias 

  • The Item Group is Role Restricted to “PI” Role

  • The applicable “PI Signoff” item that registers a PI’s acknowledgement of data approval is marked as complete. 

Further notes about the Form setup for PI Signature:

  • Name the Form / Item Group / Item whatever you like.

  • The Data Type for the Item can be anything you like (a Yes/No Code List, a boolean tick checkbox or any other option that you chose.

  • The Form does not need to contain a datetime field, the date of the PI signature will be taken from the audit trail.

  • The Form can be built into an Activity Plan as a scheduled activity or it can be added as an adhoc Form on the Data Collection page.

  • The Report will not return the value recorded in the Item. For example, if you set it up as “PI Review” with Code listed answer “Yes” and “No”, it will not populate the column with the Yes or No value. The system will only look at the Item with the relevant Alias and if the Item has been completed during data collection, it will return PI Signature in the Report as “Yes”. Thus it is recommended to create the Item with only one possible option that reflects that the Review was done, for example, “PI Review Complete” with a boolean type checkbox for the PI to complete.

ClinSpark

Report

PI Signature Date

This field reflects the most recent date on which a PI completes the applicable ‘PI Signoff’ item that registers a PI’s acknowledgement of data approval.  This date may be accessed by clicking on the corresponding item prompt within the Study > Data component. 

ClinSpark

Report

User 1 Reviewed Item Group Data

Zero-to-many users may be permitted to mark item groups as reviewed.  The report will display the progress of all the users who are added as Reviewers in the Study > Configure > Data Reviewers module.

This field tracks the number of item groups that have been reviewed by the first reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

User 1 Reviewed Item Group Data %

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks the percentage of existing item groups that have been reviewed by the first reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.  The formula for this calculation is as follows:

Review % = (Reviewed Item Group Data by User 1) / (Existing Item Group Data - Cancelled Item Group Data)

ClinSpark

N/A

Report 

User 2 Reviewed Item Group Data

Zero-to-many users may be permitted to mark item groups as reviewed. The report will display the progress of all the users who are added as Reviewers in the Study > Configure > Data Reviewers module.

This field tracks the number of item groups that have been reviewed by the second reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

User 2 Reviewed Item Group Data %

Zero-to-many users may be permitted to mark item groups as reviewed. This field tracks the percentage of existing item groups that have been reviewed by the second reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details. The formula for this calculation is as follows:

Review % = (Reviewed Item Group Data by User 2) / (Existing Item Group Data - Cancelled Item Group Data)

ClinSpark

N/A

Report 

TAB 2 - ‘Item Group Level’ Tab

The item group level tab is intended to track data cleaning progress on a per-item group basis.  This tab is useful for determining which data requires action (completion, locking, or review) in order to obtain ‘clean’ data.  For each item group present in the trial database, the following status fields help to track progress toward clean data:

  • Item Group Name

  • Locked Item Group Status and Date

  • User Review of Each Item Group

  • Data Management Review of Each Item Group

  • Monitor Review of Each Item Group

Study Name

Name of the study displayed to users when navigating ClinSpark.  Defined in the Study > Configure component.

ClinSpark

Report

Protocol Name

Protocol name of the study reported during data exporting.  Defined in the Study > Configure component.

ClinSpark

Report

Volunteer ID

This is the unique identifier for each volunteer that follows the volunteer across studies.

ClinSpark

Report

Randomization Number

Randomization (or “enrollment”) number is the subject number under which a subject has dosed or otherwise enrolled into the trial.  These numbers are established within ‘Randomization’ style cohorts in the Study > Configure component.

ClinSpark

Report

Lead In Number

Lead-In number is the subject number under which a subject has participated in Day -1 of the trial.  These numbers are established within ‘Lead-In’ style cohorts in the Study > Configure component.

ClinSpark

Report

Screening Number

Screening number is the subject number under which a subject has screened for a trial.  These numbers are established within ‘Screening’ style cohorts in the Study > Configure component.

ClinSpark

Report

Subject Status

Subject status tracks the state or outcome of a subject’s participation in a trial.  Options include Active, Completed, Terminated, and Withdrawn.  These statuses are established within the Study > Subjects component. 

ClinSpark

Report

Locked Subject (Y/N)

This field tracks whether data for an entire subject has been locked.  Typically, subjects are locked as one of the final steps in data cleaning.  Once a subject is locked, none of that subject’s data may be edited.  Locked subjects are displayed with a lock icon next to the ‘subject’ hyperlink in the Study > Data component.

ClinSpark

Report

Study Event

This field tracks the visit (or “Study Event”) against which each item group is associated.  This helps users to quickly locate data that may require action (i.e. completion, review, or locking).  Visit information is readily displayed in the Study > Configure component.

ClinSpark

Report

Form

This field tracks the Form name in which each item group is located.

Timepoint

This field tracks the Timepoint that is defined in Timed Activity Plans for each Form.

Item Group Name

This field tracks the item group name for each item group available for each subject.  This helps users to quickly locate data that may require action (i.e. completion, review, or locking).  Item Group information is readily displayed in the Study > Configure component.

ClinSpark

Report

Repeat #

When Repeat Item Groups are used for data collection, the Report will track the number of each occurrence of the repeated Item Group.

Canceled (Y/N)

This field tracks if the Item Group has been canceled.

Locked (Y/N)

This field tracks the count of item groups for each subject which have been individually locked.  Once an item group has been locked, the contents of the item group may no longer be edited.  Item groups which have been locked display with a ‘locked’ icon in the Study > Data module.

ClinSpark

Report

Collection Status

This field shows the collection status of any given Item Group.

DM Review (Y/N)

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks whether each item group has been reviewed by any user with a ‘DM’ or ‘Data Manager’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

CRA Review (Y/N)

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks whether each item group has been reviewed by any user with a ‘CRA’ or ‘Monitor’ role.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

User 1 Review (Y/N)

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks whether each item group has been reviewed by the first reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

User 2 Review (Y/N)

Zero-to-many users may be permitted to mark item groups as reviewed.  This field tracks whether each item group has been reviewed by the second reviewer.  Within the Study > Data component, reviewed data is marked by a green eyeball icon.  Click into the Item Group hyperlink to access review details.

ClinSpark

Report

Exported and Printed Copies Are Uncontrolled