Use Case 2 (UC2) Aim 3: Real-Time Validation of Code-Based Algorithms

This project is aim 3 of the overall UC2 project “Empirical Application of the Sentinel EHR and Claims Data Partner Network to Enhance ARIA Sufficient Inferential requests and Atypical Descriptive Requests”. 

One of the ways to enhance claims-based analyses is by validating and possibly improving the measurement characteristics of the code-based algorithms used to identify health outcomes of interest. Extensive time and resource allocation is typically required to carry out gold-standard label creations through manual chart review. In this specific aim, a process was tested that introduced efficiency into a validation study in two ways: 1) use of natural language processing (NLP) to reduce time spent by humans to review each chart, and 2) a multi-wave adaptive sampling approach with pre-defined criteria so the validation study could be stopped when performance characteristics were identified with sufficient precision. An example of an NLP tool that was developed to support efficient chart review was Clinical Optimized Record Annotation (CORA) which automatically searched and highlighted all synonyms in free-text notes based on the Unified Medical Language System (UMLS) for clinical terms relevant to the outcome of interest being validated. This allowed the human reviewers to focus on higher-yield information within notes that were likely to contain descriptions related to the underling clinical concept of interest and made the process of chart review more efficient. 

This process was tested in a case study that validated the performance of a claims-based outcome algorithm for intentional self-harm in patients with obesity.