Technical Updates

Onboarding Outreach: Moving Data Into Production

Each month the NSSP onboarding team will contact site administrators to ask whether data being received in the Onboarding environment (used for staging) are ready for activation in the Production environment. This inquiry isn’t intended to replace onboarding activities that sites are implementing. Instead, it’s a collaborative approach that the onboarding team uses to move facilities through the onboarding process—from Staging to Production.

The onboarding team first checks whether facilities that actively send data to the Onboarding environment have data in XX_ST_Processed that meet minimum completeness requirements of 80% or greater across all Priority 1 data elements.

If completeness is good (above 80%), the onboarding team will ask site administrators if they are ready to advance these facilities to Active status in the Master Facility Table, provided the facilities can pass a validity review. (Sometimes site administrators have reasons for keeping facilities in an Onboarding status and decline to have these facilities reviewed further.)

If the site administrator responds “yes” and wants to move the facilities into Production, the onboarding team will

• Run full validation checks on all facilities based on site feedback and with site administrator approval.
• Promote all successfully validated facilities to Active status in the Master Facility Table.
• Follow up with site administrators to confirm all approved facilities’ data feeds are moved to the Production environment.

Additionally, the onboarding team will contact site administrators whose facilities were recently approved for Activation but have not begun sending data to Production. For each approved facility, the onboarding team will request

• an update on activities required for moving facility data to the production feed and
• an estimated time when the onboarding team should expect to see data arrive on production servers.

If you have questions about how to enlist the onboarding team’s help in moving facility data from the Onboarding environment (used for staging) to the Production environment, please submit a ticket to the NSSP Service Desk (an account is required).

New Laboratory Categories Available in ESSENCE

In October 2021, NSSP developers created a new calculated field in the laboratory A “by result” table to standardize the results for qualitative tests. The new categories include groupings for positive, negative, indeterminate, and test not performed, as well as groupings for detected microbes, detected drugs, and antibiotic resistance results. The field has been released as “C_Result_Test_Category” and is available for immediate use. All historic laboratory tests in ESSENCE have been updated with these categories.

The same rules apply for accessing ESSENCE laboratory data as for accessing emergency department data. By default, everyone in the same ESSENCE site should be able to see laboratory testing results when either the patient or provider ZIP code falls under that site’s purview. If you have questions about access, please contact your site administrator.

As a reminder, one major commercial laboratory transmits test orders and results every ten minutes. NSSP has labeled this commercial laboratory “Lab A.” Data are sent at time of order, and all result types (including positive and negative results) are included. Data are identified by specimen, and while patient demographics are available, no unique patient identifiers are available in ESSENCE at this time. Lab A data are available from early 2019.

Lab A data include all testing for any reportable disease in the United States. This means if any jurisdiction has a disease listed as a mandatory reportable disease, NSSP will receive testing results for that disease from across the nation. NSSP also receives data on specific drugs of abuse from Lab A, including, but not limited to, opioids, fentanyl, cocaine, amphetamines and methamphetamines, benzodiazepines, and cannabinoids. Furthermore, data for PCR and serology SARS-CoV-2 testing are available from six commercial laboratories, including Lab A. Laboratory data do not update on weekends or federal holidays and might be subject to occasional reporting lags based on events in a jurisdiction or a surge in testing.

Information about all laboratory fields available in ESSENCE can be found on tab 18 of the Commercial Laboratory Data Dictionary [Technical Publications and Standards | CDC]. Please contact nssp@cdc.gov with any additional questions.

New NSSP Data Dictionary Available

Version 41 of the NSSP Data Dictionary is now available on the NSSP Technical Publications and Standards website. This version provides additional information about the BioSense Platform Datamart database. The Indexes tab, which listed information about database indexes, has been deleted and two new tabs were added.

Performance of indexes in the Production and Onboarding environments was analyzed in the context of the queries being run, and the indexes were redesigned to be more efficient for current usage. The new tabs (Production Indexes, Staging Indexes) now list the indexes for tables in the Production and the Staging environments separately.

Thanks to Collaboration with Florida, NSSP Resolves
Issue Affecting Two Syndromes

Analysts with the Florida Department of Health found an error in the Chief Complaint Discharge Diagnosis (CCDD) categories for influenza-like illness (“ILI Syndrome Neg Coronavirus DD v1”) and e-cigarette or vaping-associated lung injury (“CDC EVALI v1-Manually limit to ages 11–34”). They alerted the NSSP team, and on October 20, 2021, NSSP notified site administrators.

Both categories use the logic “ISNULL,or,^,ANDNOT,” with CCDD category free-text values to negate other CCDD categories. Due to the processing logic, a visit could be labeled with either the ILI or EVALI categories before receiving another category label that should result in negation. The simplest example is a visit being categorized for “ILI Syndrome Neg Coronavirus DD v1” and “CDC Coronavirus DD v1.” The first syndrome is binned before the CDC Coronavirus DD v1 label can be applied, resulting in contradictory CCDD category labels.

Developers with the Johns Hopkins University Applied Physics Laboratory implemented a fix for these two CCDD categories on October 27, 2021, and several days were required to process corrections to the historic data. The corrections impacted all queries that used these categories. (Note: The fix corrected version 1 categories; version 2 categories were not created.)

Visit counts for both categories either decreased or remained unchanged. Our estimation based on 90 days of NSSP data indicated that nationally the ILI Syndrome Neg Coronavirus DD v1 would decrease by an average of 1,023 visits per day and that CDC EVALI v1-Manually limit to ages 11–34 would decrease, on average, by fewer than 7 visits per week. For ILI Syndrome Neg Coronavirus DD v1, the error with Coronavirus negations caused visits to roughly follow current trends in COVID-19 diagnosis codes.

Users may want to assess the impact of this fix on individual health jurisdiction data. To do so, users can run the following queries in the “CC and DD Category Free Text” field to quantify these changes. The returned visits will be those no longer receiving the labels of “ILI Syndrome Neg Coronavirus DD v1” or “CDC EVALI v1-Manually limit to ages 11–34.”

^;ILI Syndrome Neg Coronavirus DD v1;^,AND,^;CDC Coronavirus-DD v1;^

^EVALI^,AND,(,^Heroin^,or,^Opioid^,or,^Stimulant^,or,^Asthma^,or,^All traffic^,or,^legionell^,or,^Pertuss^,or,^Heat Related^,or,^Suicide^,or,^Dialys^,or,^Firearm^,or,^Diabetic^,)

Many thanks to our partners in Florida for bringing this error to our attention. For more details, please contact nssp@cdc.gov.