Computable Phenotyping and Validation Studies: Venous Thromboembolism and Encapsulated Bacterial Infection

The Use Case 1 (UC1) Aim 1 (A Systematic Process for Assessing Fitness-for-Purpose of Health Outcomes for Computable Phenotyping with Electronic Health Records Data) project, led by Kaiser Permanente Washington Health Research Institute (KPWHRI) and the University of Washington, developed a systematic fitness-for-purpose (FFP) assessment process to implement concepts outlined in a previously described general framework for computable phenotyping incorporating Electronic Health Record (EHR) data. The process included verifying the Health Outcome of Interest (HOI) is well-defined, reviewing clinical information about the HOI, identifying existing algorithms and their performance, evaluating HOI clinical and data complexity, and determining an overall FFP conclusion and recommendation. This process was applied to ten HOIs lacking high-performing claims-based algorithms, selecting HOIs of public health importance that varied in clinical and data complexity. The aim of this project was to develop and validate computable phenotypes for two (2) HOIs, using the Sentinel Real World Evidence – Data Enterprise  (RWE-DE) data sources. The HOIs selected for this activity were 1) Venous Thromboembolism (VTE) and 2) Encapsulated Bacterial Infections (EBI).  

Two institutions led the development of these phenotypes: Brigham and Women's Hospital and Harvard Medical School primarily worked on VTE, while Vanderbilt University Medical Center (VUMC) primarily worked on EBI. For the VTE validation study, eligible patients included adults aged ≥18 years with at least 6 months of prior continuous enrollment and an MGB encounter on or after October 1, 2015. The final cohort included 23,537 patients with at least one ICD-10 code for VTE and 117,254 non-cases matched on age, sex, and care setting. For the EBI validation study, we relied on VUMC EHR data from 2016 through 2024. We identified candidate cases and controls using 244 ICD diagnosis codes, and index conditions were restricted to emergency department and inpatient encounters. The final cohort included 76,376 episodes from 59,097 patients, with 10,168 contributing more than one episode.

To support this validation, the study team used automated and semi-automated approaches to extract information from both structured and unstructed EHR data to validate the outcomes against ‘gold labels’, generated by expert review of patient charts. For the two selected HOIs, the study team carefully considered automated approaches that have shown promise in prior Sentinel Innovation Center (IC) studies. Specifically, PhenoSCALE (a semi-automated pipeline) and PheNorm (an automated pipeline), are two approaches that can use data driven feature identification as well as leverage training on computed ‘silver labels’ to significantly cut down on development timelines compared with the traditional manual approaches of phenotyping algorithm development. PhenoSCALE was applied for the phenotyping of VTE and PheNorm was applied for the phenotyping of EBI.

The chart review activities were conducted according to the HOIs selected, taking care to match the data source characteristics (such as inpatient or outpatient data, sample size) to the requirements of the HOI. The team included up to 150 charts in each of the (2) chart validation activities for VTE and EBI, each supporting one HOI. To support chart review of unstructured data, SOC/IC utilized an available NLP-supported chart abstraction tool which utilizes auto-highlighting of key words and synonyms to expedite chart review activities by human experts and was found to shorten review times by 40% in a recent IC study. 
Expanding on this project, VUMC will continue the effort developing and validating EBI. Computable phenotype algorithms and subsequent validations will be conducted using the VUMC EHR data. The team will expand on the list of bacterial species from the previous work, using a two-phase phenotyping approach. The aims of this follow up work include: 
Aim 1: Define and construct candidate episodes for clinically serious infection syndromes caused by selected encapsulated bacteria.
Aim 2: Extend the prior PheNorm-based EBI algorithm to develop hierarchical models for syndrome classification, serious bacterial infection, and organism attribution.
Aim 3: Evaluate phenotype performance against expert-adjudicated chart review and compare narrow, expanded, and prior phenotype definitions.