Importance of Continued Data Quality Assessment of Syndromic Production Data

Objective To explore the quality of data submitted once a facility is moved into an ongoing submission status and address the importance of continuing data quality assessments. Introduction Once a facility meets data quality standards and is approved for production, an assumption is made that the quality of data received remains at the same level. When looking at production data quality reports from various states generated using a SAS data quality program, a need for production data quality assessment was identified. By implementing a periodic data quality update on all production facilities, data quality has improved for production data as a whole and for individual facility data. Through this activity several root causes of data quality degradation have been identified, allowing processes to be implemented in order to mitigate impact on data quality. Methods Many jurisdictions work with facilities during the onboarding process to improve data quality. Once a certain level of data quality is achieved, the facility is moved into production. At this point the jurisdiction generally assumes that the quality of the data being submitted will remain fairly constant. To check this assumption in Kansas, a SAS Production Report program was developed specifically to look at production data quality. A legacy data set is downloaded from BioSense production servers by Earliest Date in order to capture all records for visits which occurred within a specified time frame. This data set is then run through a SAS data quality program which checks specific fields for completeness and validity and prints a report on counts and percentages of null and invalid values, outdated records, and timeliness of record submission, as well as examples of records from visits containing these errors. A report is created for the state as a whole, each facility, EHR vendor, and HIE sending data to the production servers, with examples provided only by facility. The facility, vendor, and HIE reports include state percentages of errors for comparison. The Production Report was initially run on Kansas data for the first quarter of 2016 followed by consultations with facilities on the findings. Monthly checks were made of data quality before and after facilities implemented changes. An examination of Kansas’ results showed a marked decrease in data quality for many facilities. Every facility had at least one area in need of improvement. The data quality reports and examples were sent to every facility sending production data during the first quarter attached to an email requesting a 30-60 minute call with each to go over the report. This call was deemed crucial to the process since it had been over a year, and in a few cases over two years, since some of the facilities had looked at data quality and would need a review of the findings and all requirements, new and old. Ultimately, over half of all production facilities scheduled a follow-up call. While some facilities expressed some degree of trepidation, most facilities were open to revisiting data quality and to making requested improvements. Reasons for data quality degradation included updates to EHR products, change of EHR product, work flow issues, engine updates, new requirements, and personnel turnover. A request was made of other jurisdictions (including Arizona, Nevada, and Illinois) to look at their production data using the same program and compare quality. Data was pulled for at least one week of July 2016 by Earliest Date. Results Monthly reports have been run on Kansas Production data both before and after the consultation meetings which indicate a marked improvement in both completeness of required fields and validity of values in those fields. Data for these monthly reports was again selected by Earliest Date. Conclusions In order to ensure production data continues to be of value for syndromic surveillance purposes, periodic data quality assessments should continue after a facility reaches ongoing submission status. Alterations in process include a review of production data at least twice per year with a follow up data review one month later to confirm adjustments have been correctly implemented.