1 Overview
2 CDISC Standard Background
3 Methods vs Edit Checks
4 Formal Expressions are Javascript
5 Formal Expression Dev Tooling in ClinSpark
- 5.1 Obtaining ItemData IDs for testing execution
- 5.2 Implicit Methods
- 5.3 Implicit Data
  - 5.3.1 formJson
  - 5.3.2 findFormData - looking up previous formData
- 5.4 Guidelines and Suggestions
  - 5.4.1 Errors/warnings in the Formal Expressions syntax checker

Overview

ClinSpark supports customer-provided dynamic logic to perform calculations and validations at the time of data collection. Methods support setting itemData values dynamically based on arbitrary logic and subject data. Edit Checks support dynamic validation of individual itemData values.

The functionality is extremely powerful. It also requires a certain degree of technical skill in Javascript, and an understanding of the context within ClinSpark that this logic executes.

CDISC Standard Background

Much of the CDISC ODM defines a schema for structuring data, and this is of course the foundation for ClinSpark’s data and object model. In addition to this static data structure guidance, the 1.3.0 CDISC ODM specification also defines a number of optional elements which support specifying dynamic behaviors for the model. This addition supports powerful use cases beyond what is possible with purely static data structure definitions. The ODM specification and it identifies 3 types that may be supported:

Name	Description	Supported by ClinSpark?	Where Defined in CDISC ODM Spec

Name	Description	Supported by ClinSpark?	Where Defined in CDISC ODM Spec
MethodDef	A MethodDef describes how a data value can be obtained from a collection of other data values. This allows calculated or derived values to be dynamically generated during data capture.	Yes	Section 3.1.1.3.9
RangeCheck	A RangeCheck defines a constraint on the value of the enclosing item. It may defined as an explicit 1 or 2 sided range or as an expression that evaluates to True when the ItemData value is valid or False when the ItemData value is invalid.	Yes. We call them “Edit Checks”	Section 3.1.1.3.6.4
ConditionDef	Indicates that the referenced element may be omitted if the Condition evaluates to true.	No	Section 3.1.1.3.11

The ODM specification does not define how to implement these behaviors. It defines data fields called Formal Expressions to hold computer code and specifies inputs and outputs. But it does not require or even suggest an implementation model. It is up to the implementation to define how this Formal Expression instruction gets executed to perform the desired operation.

In light of this, ClinSpark has implemented the MethodDef and RangeCheck to leverage Formal Expressions written in Javascript. This Javascript code has access at execution time to the ItemData and its FormData context in the form of a JSON object. And this execution occurs on the server prior to persisting the data.

Methods vs Edit Checks

Methods are top-level CRF Design elements which can be attached to zero to many items, and the value returned by a Method is set as the value of the attached ItemData. Edit Checks are essentially validation scripts, and the return value can determine whether an ItemData can be Complete or nonconformant. Both Methods and Edit Checks execute once each time data is collected.

Type	Reusable within a study?	Outcome	Return Type	In CDISC

Type	Reusable within a study?	Outcome	Return Type	In CDISC
Method	Yes. Can be attached to multiple Items	Sets the value of the attached ItemData	Return type MUST match the datatype of the attached item.	MethodDef
Edit Check	No. Must be copy/pasted each time onto the appropriate Item	Decides whether an Item is conformant	Boolean. True = valid value, False = invalid.	RangeCheck

Formal Expressions are Javascript

ClinSpark supports “Formal Expressions” through Javascript. This is the Javascript code which gets executed to accomplish the task. It is the contents of a method, which makes some calculation, performs whatever logic is required to implement either the MethodDef or the RangeCheck.

ClinSpark currently uses the Mozilla Rhino Javascript engine bundled with Java 7. This implementation of Rhino is based on version 1.7R3 pre-release sources with Oracle modifications.
https://www.oracle.com/java/technologies/javase/7u1-relnotes.html

The ClinSpark product team is evaluating support for more modern Javascript engines in a future release. Customers that have feedback on this topic are encouraged to reach out via service desk with questions or concerns.

Here’s an example of a method in ClinSpark which calculates BMI. It is designed to be associated with a Item who has other Items named Height and Weight. It retrieves these values from JSON passed to the method, performs the calculation, then returns a JSON response which includes the units.

Note that the Context value in the ClinSpark implementation is used for reference only, it does not impact the execution of the expression in any way.

The return requirements for a Method are a value if the function does not need units, or a JSON object as shown above if units must be specified.

Formal Expression Dev Tooling in ClinSpark

To support self-serve formal expression development and testing by customers, an embedded editor and test execution harness exists within ClinSpark. Here is where it can be accessed:

This tooling applies to both Methods and Edit Checks. You will see that the same test harness is present in the editing UI for both of these features.

Here is an overview of the editor and testing harness:

Red dot	Description

Red dot	Description
1	This is an embedded Javascript editor. It supports appropriate highlighting and basic syntax checking. Note the warnings and errors in the left sidebar.
2	Guidance on using the logger function to gain insight into the inner workings of your code. This is crucial during development and troubleshooting.
3	To execute the expression against collected itemData, this field accepts the database ID of an itemData. Details to follow on how to obtain this value.
4	Executes the method from #1 against the itemData specified in #3 and displays the output along with any logger content. This allows testing and troubleshooting against actual data, without requiring additional collection activities.
5	Displays the available ClinSpark-provided Javascript functions which will be available to the expression at runtime.
6	Downloads the JSON contents of the itemData from #3 as a file. This may be useful, but note that you can also use logger(JSON.stringify(itemJson, null, 2)); and also logger(JSON.stringify(formJson, null, 2)); to see this output dynamically as well.

Obtaining ItemData IDs for testing execution

Expressions are run against actual itemData. The test harness requires as an input the database ID of a specific itemData of the user’s choice.

The first step in testing a method is to ensure that there is some collected itemData to test against. Once this is done, follow this walkthrough to see how to obtain the desired ID.

Red dot	Description

Red dot	Description
1	Search for a collected itemData which you would like to run your function against.
2	Hover over the name link.
3	Here you will see a numeric ID for that itemData you’re hovering over. THIS is the database ID, and this is what you can place into the ItemData ID field in the test harness.

Implicit Methods

As part of ClinSpark’s configuration, commonly useful utility methods are available to your code at runtime. These will be visible to your code at execution time and you can call them directly as needed. These methods assist in finding a certain item from a form for instance, ClinSpark-specific date utilities, etc. #5 from above will open a page showing all available methods.

Note that a useful technique when troubleshooting these methods is to copy a required function into your editor and add logger statements to help understand what is happening. For instance, you could see which itemGroups are inspected by adding the below logger statement to the standard method, assuming you have copied it into your editor.

function findFirstItemByName(formJson, itemName, sasFieldName, itemGroupRepeatKey) {
    var itemGroups = formJson.form.itemGroups;
    if (itemGroups && itemGroups.length) {
        for (var i = 0; i < itemGroups.length; i++) {
            var itemGroup = itemGroups[i];
            logger('inspecting itemGroup.name'+itemGroup.name);

Implicit Data

Two JSON objects are available to each expression execution, itemJson and formJson. The itemJson contains all of the data and metadata pertaining to the itemData which the expression is attached to. The formData is the complete formData graph, including all contained itemGroupData and itemData as wll as context info such as data about the subject/volunteer, cohort, study event etc.

You can use the logger to see the contents of these objects like this:

Red dot	Description

Red dot	Description
1	As per the highlighted help, add a logger entry to print the JSON object (itemJson or formJson) as a string.
2	Enter your chosen itemData target.
3	Click “Test”

The execution window output now displays the JSON object.

From there you can format it using the tool of your choice to see the output you’re looking for. A Google search for “prettify JSON” will help locate free online tools; the internal team uses VS Code to do this.

{
    "item": {
        "id": 136424,                      // Unique ID for the item data
        "name": "AE_START_DATE",           // Name of the item (e.g., 'Adverse Event Start Date')
        "dataType": "incompleteDatetime",  // Data type (e.g., 'string', 'date', 'incompleteDatetime')
        "dataCollectionStatus": "Complete",// Status: 'Complete', 'In Progress', etc.
        "sasFieldName": "AESTDTC",         // Corresponding SAS field name
        "value": "2024-08- T13:39: ",      // Captured value (can be incomplete)
        "codeListItems": [                 // Optional: List of allowed coded values
            {
                "codedValue": "CS",         // Coded value
                "decode": "Clinically Significant" // Meaning of coded value
            },
            {
                "codedValue": "NCS",
                "decode": "Not Clinically Significant"
            }
        ],
        "measurementUnits": [],        // Array of applicable measurement units
        "measurementUnit": null,       // Specific measurement unit (if used)
        "outOfRange": true,            // Whether the value is out of an acceptable range
        "nonconformantMessage": null,  // Message if the item is nonconformant
        "length": null,                // Optional: Length constraint for the value
        "significantDigits": null,     // Optional: Number of significant digits allowed
        "canceled": false              // Whether this item has been canceled
    }
}

formJson

The formJson object contains the complete formData along with all contained itemGroupData and their child itemData.

In addition, quite a bit of context metadata can be found within the formJson object:

{
    "form": {
        "name": "Form Name",                // Name of the form (e.g., 'Demographics')
        "studyEventName": "Event Name",      // Name of the study event
        "cohort": {                         // Cohort-related metadata
            "id": 88,
            "name": "Cohort Name",
            "epoch": {
                "id": 80,
                "name": "Screening"         // Epoch or phase name
            }
        },
        "timepoint": null,                   // Timepoint (if applicable)
        "canceled": false,                   // Whether the form is canceled
        "dataCollectionStatus": "Complete",  // Status: Complete, In Progress, etc.
        "subject": {                         // Information about the subject/volunteer
            "id": 165,
            "screeningNumber": "S440113001", // Subject screening number
            "randomizationNumber": "1302",   // Randomization number
            "subjectStudyStatus": "Active",  // Status of subject in study
            "volunteer": {                   // Volunteer details
                "id": 135,
                "initials": "M-M",
                "age": 30,
                "sexMale": true,
                "dateOfBirth": "1990-XX-05" // Supports incomplete dates
            }
        },
        "itemGroups": [                     // Array of item groups within the form
            {
                "itemGroupRepeatKey": 1,    // Identifier for repeated groups
                "items": [                  // Array of individual items
                    {
                        "name": "Item Name",   // Name of the item
                        "dataType": "string",  // Data type (e.g., string, date, etc.)
                        "value": "some value", // Collected data value
                        "canceled": false,     // Whether the item is canceled
                        "sasFieldName": "SAS_NAME", // SAS field name mapping
                        "codeListItems": []    // Optional: Code list if applicable
                    }
                ]
            }
        ]
    }
}

All of this data can be directly accessed from the expression.

findFormData - looking up previous formData

Sometimes you will need to lookup up other collected data for a subject. For instance this might allow a comparison of a value to a baseline, or retrieve other data imported from a volunteer record demographic form.

To see an example of using this method, search for the Triplicate Reading Average example in the Method library. This method retrieves 3 previously collected ECG readings and then produces an average.

findFormData supports querying for previously collected formData of this same subject only.

Guidelines and Suggestions

Define variables up top to support reusability across studies. See the examples in the Method Library section of this help site.

Errors/warnings in the Formal Expressions syntax checker

The syntax checker by default validates code as “strict mode” JavaScript. If the JavaScript code is not marked 'use strict' at the start of the file, you may get an error saying:
"Too many errors. (nn% Scanned)"

There are two ways to fix this.

1, Tell the syntax checker you are not using strict mode, paste this at the first line:
/* jshint strict: false */

2, If you want to write JavaScript in strict mode, and get strict mode syntax checking, paste this in the first two lines:
/* globals itemJson, formJson, findFormData, logger, customErrorMessage, findCompletedFormData, findFormData, getItemDataContext */
'use strict';

If you are using other implicit data/methods not included here and get “undefined variable” errors in those lines, add those variables/functions to the list of “globals”.

The list is needed because the editor would flag our implicit methods and implicit data as undefined variables.

Documentation

Creating Methods and Edit Checks