ECMO Predictive Scores, Past, Present, and Future

Abstract

Over the five decades since the first successful reports of extracorporeal membrane oxygenation (ECMO) use, ideal patient selection has been an ongoing question. This has led to the development of several prognostication tools aimed at identifying risk factors associated with poor outcomes. These have spanned neonatal, pediatric and adult patients supported on ECMO for cardiac or respiratory failure. The majority of these scores have focused on mortality as an objective poor outcome with only 2 adult scores looking at long-term neuropsychological outcomes in ECMO survivors. In the development of these scores the authors have mainly relied on registry style data with limited granularity and focused on immediate pre-ECMO data points without incorporation of the evolving patient trajectories leading up to ECMO cannulation. While such scores can be useful in both prognostication and as risk stratification and quality assessment tools, they all lack practicality on an individual patient level with regards to decision making, as these scores have all been developed on data from patients already supported on ECMO without a comparable control cohort, to truly mimic decision making at the bedside. In this chapter we review the currently available ECMO prognostication scores, their limitations and potential future directions.

1. Introduction

Throughout the history of extracorporeal membrane oxygenation (ECMO) from the first reported successful use in an adult patient in 1972 [1] and subsequently in neonates with respiratory failure [2], and then followed by the exponential increased use in adults following the CESAR trial [3], there has been interest in identifying patients who would benefit most from this high-risk resource-intensive therapy. Early efforts focused on using predictive scores of severe neonatal respiratory failure [4]; Newborn Pulmonary Insufficiency Index (a score developed by plotting serial inspired oxygen values with serial pH measurements in the first 24 hours of life) [5] and serial alveolar-arterial oxygen gradients (A-a DO2) [6], were deployed with mixed results. Since then, there have been extensive efforts at developing tools to aid in early identification of patients who would benefit most from timely institution of ECMO support and those with a high risk of mortality while being supported by ECMO. In this chapter, we provide a review of the currently available ECMO prediction tools, their development, validation and limitations and an outline of potential future directions of ECMO decision support tools.

2. ECMO outcomes prediction scores to date

The need for predictive scoring algorithms is particularly vital with deployment of a high resource therapy like extracorporeal life support. Over the last several decades various groups have published predictive scores typically focusing on mortality, with the hope to help guide clinical decision making for optimal patient selection prior to cannulation, and often to help stratify patient risk. The overall goal being to help identify patients most likely to benefit, and these scores have ranged in accuracy (measured by the Area Under the Receiver Operative Curve) from 0.65 to 0.89 [7].

To date, these scores have been focused on the broad pathophysiology of the disease necessitating ECMO support; respiratory failure requiring veno-venous (VV) ECMO and cardiac failure necessitating support by veno-arterial (VA) ECMO.

2.1 Respiratory failure

2.1.1 Neonatal - congenital diaphragmatic hernia

Congenital diaphragmatic hernia (CDH) continues to be one of the most common reasons for ECMO use in the neonatal age group [8]. This unique pathology is characterized by a failure of diaphragmatic development, associated lung hypoplasia and subsequent persistent pulmonary hypertension of the newborn (PPHN) and as such typically presents with isolated refractory respiratory or combined respiratory and cardiac failure shortly after birth [9].

As worsening hypoxia and hypercapnia after birth often exacerbate pulmonary hypertension in turn leading to worsening gas exchange and hemodynamics, the potential need exists for ECMO as a rescue therapy for those with severe CDH [9]. Prediction of CDH severity is often identified prenatally and is based off genetic testing, the sidedness of the CDH, liver position (proportion in the thorax), and observed to expected fetal lung volume by magnetic resonance imaging (MRI) [10]. Despite decades-long use of ECMO for CDH, there continue to ongoing controversies surrounding ECMO use in this unique population. Outcomes remain variable, controversy exists regarding if the optimal support modality is VV or VA ECMO and the timing of ECMO initiation, as well as if surgical repair is best performed early or late and finally if surgical repair should be performed while on ECMO support due to the increased possibility of associated bleeding complications [11, 12, 13, 14]. The CDH working groups for both the Extracorporeal Life Support Organization (ELSO) and its European chapter (EuroELSO) have published entry criteria for utilizing severity of hypoxemia, impaired ventilation and impaired tissue perfusion prior to ECMO as indications and prematurity, weight, comorbidities and duration of mechanical ventilation as relative contraindications [15, 16, 17].

Currently two CDH ECMO specific mortality risk prediction models exist utilizing data from the ELSO registry data from 2000 to 2015, including over 4000 ECMO supported neonates [18]. Using multivariable logistic regression analyses using both complete data sets and 10 imputed data sets, the authors developed two predictive models, a pre-ECMO mortality risk score and another on-ECMO mortality risk score. By dividing the studied cohort into a two thirds development cohort and a one third validation cohort, the authors included pre-ECMO ventilatory settings and adequacy of gas exchange as measured by blood pH in addition to CDH specific risk factors including CDH sidedness and repair on ECMO. The developed pre-ECMO model performed modestly with C statistics of 0.65. With the addition of on-ECMO complications; neurologic, renal and infectious, the on-ECMO Prediction model performance improved to C = 0.73 (Table 1). The authors recognized the value of these models specifically in research and quality improvement projects with cautious application in patient management.

Score	Year	Variables	Patient cohort
CDH; Pre-ECMO	2018	Prior CDH repair; Critical congenital heart disease; Perinatal infection; Weight; APGARs; Side of hernia; Pre ECMO-Arrest; pH; Ventilator settings	4374 Neonates with CDH from; ELSO registry; (2000–2015)
CDH; On-ECMO	2018	Pre-ECMO +; On-ECMO; ECMO settings (pump type); ECMO associated complications (hemorrhage, severe neurologic complication, elevated creatinine, dialysis, tamponade, CPR, sepsis)	4374 Neonates with CDH from; ELSO registry; (2000–2015)
PIPER	2016	Apgar at 5 minutes <7; Birth weight < 3 kg; Age > 10 days; CDH; MAP <49 mm Hg; pO2 < 34 mmHg; Patient not on iNO	5455 on VA ECMO, <30 days from; ELSO registry; (2000–2010)
Neo-RESCUERS	2016	Birth Weight; Gestational Age; Age; Gender; Primary Diagnosis; Comorbidity; Renal Failure; pH; PaO2/FiO2; iNO	4592 patients, <28 days; From ELSO registry; (2008–2013)
PED-RESCUERS	2016	Comorbidities; Primary diagnosis of Asthma, Bronchiolitis or Pertussis; pH; PaCO2; Ventilator settings; Duration of admission and MV prior to ECMO; Milrinone	2458 on ECMO for respiratory failure, 29 days to 18 years from; ELSO registry; (2009–2014)
P-PREP	2017	Gender; Age > 10; Year of ECMO support; Primary pulmonary diagnosis; Comorbidities; PF ratio; pH; VV vs. VA; Mechanical ventilation >14 days; HFOV; iNO; Neuromuscular blockade	4352 patients on ECMO for respiratory failure, >7 days to <18 years from ELSO registry; (2001–2013)
ECMONet	2013	PreECMO hospital length of stay; Mean Arterial pressure; Bilirubin; Creatinine; Hematocrit	60 adult influenza A patients with respiratory failure from multicenter data (2009 H1N1 Pandemic)
PRESERVE	2013	Age; BMI; Immunocompromised; SOFA>13; MV > 6 days; No prone positioning prior to ECMO; PEEP <10; Plateau Pressure > 30	140 adult ARDS patients from multicenter data (2008–2012)
RESP	2013	Age; Immunocompromised status; Mechanical ventilation prior to initiation of ECMO; Acute Respiratory Diagnosis; CNS dysfunction; Acute associated non-pulmonary infection; Cardiac Arrest prior to ECMO; PaCO2; Neuromuscular blockade prior to ECMO; iNO; Bicarb. Level; Peak inspiratory pressure	2355 adult patients with respiratory failure from; ELSO registry; (2000–2012)
Roch Score	2014	SOFA score; Age; Influenza Pneumonia	85 adult patients with ARDS from single center; (2009–2013)
VV	2016	Immunocompromised; SOFA score; Days of MV	116 adult patients with ARDS from single center; (2007–2015)
PRESET	2017	Hospital days pre ECMO; Mean arterial pressure; Lactate; pH; Platelet	108 adult patients with ARDS from single center; (2010–2015)

Table 1.

ECMO mortality prediction scores for respiratory failure.

ECMO = extracorporeal membrane oxygenation, CDH = congenital diaphragmatic hernia, MAP = mean arterial pressure, pO2 = partial pressure of oxygen, iNO = inhaled nitric oxide, VA = veno-arterial, ELSO = Extracorporeal Life Support Organization, CPR = cardiopulmonary resuscitation, PHIS = Pediatric Health Information System, MV = mechanical ventilation, VV = veno-venous, HFOV = high frequency oscillatory ventilation, CNS = central nervous system, iNO = inhaled nitric oxide, BMI = body mass index, bicarb = bicarbonate, SOFA = Sequential Organ Failure Assessment, ICU = intensive care unit, ARDS = acute respiratory distress syndrome, PTSD = post-traumatic stress disorder, HRQL = Health Related Quality of Life.

2.1.2 Neonatal

Two commonly utilized scores exist to predict ECMO mortality risk in the general neonatal population beyond CDH.

The Pittsburgh Index for Pre-ECMO Risk (PIPER) was developed utilizing 5455 neonatal respiratory VA-ECMO patients from the ELSO registry from 2000 to 2010 to predict survival to hospital discharge [19]. PIPER was developed on seven pre-ECMO variables including patient characteristics; age, weight and the diagnosis of CDH in addition to markers of hemodynamic compromise and severity of respiratory failure prior to ECMO initiation; mean arterial blood pressure (MAP), pH, partial pressure of oxygen (pO₂) and use of inhaled nitric oxide (iNO). The authors found that each increasing quartile had a 15% increased risk of mortality. Despite the focus on pre-ECMO risk, the authors also conducted further modeling to include on-ECMO variables such as ECMO duration and complications (hemorrhagic, mechanical, neurologic, pulmonary and renal), which increased the predictive power of the PIPER model from an area under the receiver operator curve (AUROC) of 0.74 to 0.79. While not developed to focus only on CDH patients, CDH was much more common in the highest PIPER quartiles. Older age at ECMO initiation was also found to be associated with decreased survival [19].

The Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support (Neo-RESCUERs) was similarly designed to predict mortality for neonates receiving ECMO respiratory support [20]. It was developed and validated on 4592 neonates in the ELSO registry between 2008 and 2013, with January 1st 2012 as the cutoff date between the initial derivation cohort and the subsequent internal validation dataset. Validation was performed on patients with complete data in addition to those with imputed data. The investigators included patient demographics (age, gestational age, birth weight, sex, diagnosis), markers of respiratory and cardiac failure (pH, partial pressure of carbon dioxide (PaCO₂), ratio of arterial partial pressure of oxygen to fraction of inspired oxygen (PF ratio), oxygenation index (OI), and history of cardiac arrest) in addition to renal failure and comorbid conditions prior to ECMO cannulation (Table 1). They demonstrated their lowest decile having a predicted mortality of 4.4% compared to an observed mortality of 7%, and their highest decile having a predicted mortality of 67.5% and observed mortality of 65.6%. CDH in this group had 11-fold higher adjusted odds for mortality compared to meconium aspiration, pre-ECMO renal failure also had a much higher odds of mortality. As is the case with similar registry-based scores, the authors acknowledge the significant limitations with reliance on only the available variables in addition to the retrospective nature of the study and the inclusion of only ECMO supported patients. As such, they recommend the use of Neo-RESCUERS as a benchmarking and risk stratification tool rather than an ECMO decision support tool.

2.1.3 Pediatric

Two common scores exist to predict mortality in pediatric respiratory failure. The Pediatric Risk Estimation Score for Children Using Extracorporeal Respiratory Support (Ped-RESCUERS) and the newer Pediatric Pulmonary Rescue with Extracorporeal Membrane Oxygenation Prediction (P-PREP) score.

Ped-RESCUERS was developed on patients aged 29 days to 18 years, utilizing the ELSO registry from 2009 to 2014, with the 2013 to 2014 cohort as the validation data set to predict survival to hospital discharge [21]. The model was developed and validated on 2458 pediatric patients undergoing ECMO for respiratory support, with an overall observed mortality rate was 39.8%. In addition to variables of severity of respiratory failure up to 6 hours prior to ECMO initiation (pH and PaCO₂), the authors also included the duration of mechanical ventilation and type of mechanical ventilatory support prior to cannulation and specific diagnoses such as pertussis, bronchiolitis and malignancy in the model development (Table 1). The model had modest performance with an AUROC of 0.69 in the development cohort and 0.63 in the validation data set. Similar to clinical experience, they found those with bronchiolitis and asthma had relatively better outcomes than those with cancer or pertussis, and those requiring ECMO later in their course had increased mortality. Interestingly higher pre-ECMO PaCO₂ was associated with less mortality, the authors speculate this may be due to the association with asthma and bronchiolitis and their subsequent survival benefit [21]. Similar to Neo-RESCUERS, the authors acknowledge the inability to use Ped-RESCUERS as a decision support tool on an individual patient level as the model was only developed on patients already supported on ECMO, but rather as a risk adjustment tool to help facilitate inter-institutional comparisons.

The P-PREP Score was developed and internally validated on 4352 children more than 7 days old to less than 18 years old requiring ECMO for respiratory failure in the ELSO registry between 2008 and 2013 and then externally validated in 2007 patients from the Pediatric Health Information System (PHIS) dataset [22]. Predictive variables included mode of ECMO support, and pre-ECMO variables such as length of mechanical ventilation, severity of hypoxia and diagnosis categories and comorbidities (Table 1). As the relative timing of comorbidities to ECMO initiation is not recorded in the ELSO registry, all comorbidities were assumed to be present at the time of ECMO cannulation. Of interest, the year of ECMO support was included in the final P-PREP model given the lower mortality rates in the 2009–2013 era compared to 2001–2009, although not assigned a score in the model calculation. Mode of ECMO support and severity of hypoxia were excluded from the external validation model as these variables are not collected in the PHIS database. The model had modest performance in the development, internal and external validation data sets (AUROC of 0.69, 0.66 and 0.69 respectively). Mortality was significantly higher in patients with two or more extrapulmonary organ system failures. Similar to previous models, P-PREP was limited to patients already supported on ECMO and as such could not be used as a ECMO decision making risk tool but the authors suggest it may aid in prognostication and family counseling for patients supported on ECMO for respiratory failure [22].

2.1.4 Adult

Since the 2009 H1N1 influenza A pandemic, there has been an exponential increase in ECMO use for adult respiratory failure [23] and as such several risk prediction scores have been developed for this population.

Perhaps the earliest widely used score was the ECMOnet score – developed on 60 patients from a multicenter 2009 H1N1 influenza A pandemic cohort [24]. The authors’ goal was to add consideration to extrapulmonary organ function, and not just the severity of respiratory failure in risk stratification for VV ECMO use in respiratory failure, aiming to aid in resource allocation and timing of ECMO initiation. Pre-ECMO predictors associated with mortality included hospital length of stay, bilirubin, creatinine, hematocrit and mean arterial pressure with a good model performance with an AUROC of 0.85 (Table 1). The results were then validated on an external dataset of 74 patients with acute respiratory distress syndrome (ARDS) supported on ECMO with an AUROC of 0.69. This tool provided early evidence of the importance of the consideration of extrapulmonary organ function at time of ECMO initiation.

The Predicting dEath for Severe ARDS on VV-ECMO (PRESERVE) score was developed on data from 140 patients with refractory ARDS in three French intensive care units from 2008 to 2012 to identify factors associated with death by 6 months post-ICU discharge [25]. A large portion of patients (26%) in this group also had H1N1 influenza. Eight pre-ECMO predictors were identified including age, body mass index, immunocompromised status, prone positioning, sepsis-related organ failure, days of pre-ECMO mechanical ventilation, plateau and positive end expiratory pressures (PEEP) (Table 1). Interestingly PF ratio was not found to be associated with mortality, but during enrolment of the development cohort increasing evidence had been mounting on the benefit of prone positioning. When the authors forced it into the model, PF ratio was found to be associated with lower mortality. The PRESERVE model performed well with an AUROC of 0.89 at predicting all cause 6-months post-ICU discharge mortality. The goal of the PRESERVE score was to help clinicians select appropriate candidates, and uniquely they provided details on health-related quality of life (HRQL) evaluation at 6 months, finding high levels of persistent physical and emotional difficulties, including anxiety, depression and post-traumatic stress disorder (PTSD) [24].

The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score is one of the most commonly clinically used and cited scores, developed from the ELSO registry data from 2000 to 2012 to predict survival to hospital discharge [26]. The authors also externally validated the score on the 140 patients used for the development of the PRESERVE score, using commonly available features in both datasets. Diagnostic groups, non-pulmonary infections and central nervous system dysfunction were significant predictors, again highlighting the importance of extrapulmonary organ function prior to ECMO initiation. The RESP score demonstrated the importance of ECMO specific mortality prediction scores as it performed better than simple severity of illness scores such as the simplified acute physiology score (SAPS II) and sequential organ failure assessment (SOFA). The RESP score includes five risk classes, with the lowest and highest classes having 8% and 82% mortality respectively. An important limitation of the score development was inability to incorporate prone positioning, as it was not reported in the ELSO registry data. As with the PRESERVE score, duration of mechanical ventilation above 7 days prior to ECMO initiation was found to be significantly associated with worse outcomes. The authors acknowledge the score limitation in only being developed on patients already supported on ECMO in addition to the lack of detailed biologic data in the ELSO registry beyond the immediate pre-ECMO blood gas values.

Since its development, the RESP score has been further externally validated in several independent studies [27, 28] and studies utilizing external databases comparing both the RESP and PRESERVE score have demonstrated similar accuracy [29, 30]. Though several of their variables overlap, which may explain their similar accuracy (Table 1). Interestingly, more recent data has shown that the RESP Score performed poorly at predicting mortality during the Coronavirus disease 2019 (COVID-19) pandemic – with patients in lower risk classes paradoxically having worse survival than those in higher risk classes [31].

The Roch Score was development to identify factors associated with in-hospital mortality in 85 ARDS patients treated with ECMO following referral to an ECMO center between 2009 and 2013 [32]. All patients were cannulated by a mobile ECMO unit prior to transfer to the referral center. ECMO decision making followed criteria with exclusion of patients with prolonged mechanical ventilation or ARDS over 7 days, age above 70 years and those with SOFA scores above 18. The uniqueness of this score’s development focusing on referral to an ECMO center, may support its utility in decision to transfer to ECMO centers. It included only limited variables (Table 1) including age, SOFA score and influenza diagnosis, making it suitable for timely decisions regarding transfer. The authors identified patients that were under 45 years of age and had a diagnosis of influenza had markedly better prognosis, independent of other organ dysfunctions [32].

Another score is the VV-ECMO mortality score [33], which was developed on a 116 adult patients single center cohort between 2007 and 2015. The authors included only patients with a PF ratio < 70, and independent predictors of in-hospital mortality included pre-ECMO variables such as SOFA score, length of mechanical ventilation and immunocompromised status, which were translated into a simple 3 binary variable predictor. The lowest score had a mortality of 18%, while the highest had a mortality of 88%, and again emphasized the importance of extra-pulmonary organ dysfunction and severity of illness. The authors’ goal was to develop a simple risk stratification tool to identify patients with the highest of a poor outcome with VV ECMO, for prognostication or consideration of alternative support modalities if possible.

The PREdiction of Survival on ECMO Therapy Score (PRESET) was derived on 108 ARDS patients between 2010 and 2015 [34]. In this work the authors analyzed the performance of 4 previous mortality risk scores; ECMOnet, RESP, PRESERVE and Roch with only the RESP and ECMOnet score demonstrating good accuracy. There was some evidence that the ECMOnet score was most appropriate only in cohorts of H1N1 patients. The authors then developed a new score, PRESET. The importance of extrapulmonary predictors such as mean arterial blood pressure, lactate, pH, platelets and pre-ECMO hospital length of stay were found. The lowest risk class had a 26% mortality rate, while the highest had a 93% mortality rate. Platelet count prior to ECMO initiation was found to be an important predictor, with a decrease of 100,000/ul associated with 30% increased mortality. These findings were similar to those reported in other critically ill patients [35]. A decrease in pH by 0.1 was also found to increase mortality by 40%. Recent studies have also demonstrated the accuracy of the PRESET score in COVID-19 patients [36].

Some studies have aimed at externally validating several of these scores with varying results, with slightly lower accuracy than initially published in general and with some studies showing superior performance of general severity of illness scores at predicting in-hospital mortality [37, 38, 39].

2.2 Cardiac failure

2.2.1 Pediatric

The Pediatric Extracorporeal Membrane Oxygenation Prediction (PEP) model was developed for risk stratifying mortality of all pediatric and neonatal patients requiring ECMO regardless of indication between 2012 and 2014 [40]. The authors included on data from 514 patients from the bleeding and thrombosis on extracorporeal membrane oxygenation (BATE) dataset to predict in-hospital mortality. Variables include indication for ECMO, age, CDH diagnosis, and laboratory markers in the 12 hours prior to ECMO initiation, utilizing the data points most proximal and prior to ECMO for analysis (Table 2). As with prior scores developed only on ECMO patients’ data, the authors acknowledged the inability to generalize the score for individual patient prediction or selection for ECMO support. The same group then performed an external validation of the PEP score on 4342 patients from the ELSO registry between 2012 and 2014 [41]. They found that the score AUROC decreased from 0.75 on the BATE data to 0.64 on the ELSO data with the most significant decrease in the highest risk deciles.

Score	Year	Variables	Patient cohort
PEP	2019	Age; Indication for ECMO; CDH; MAS; Baseline pH; PTT; INR; Documented blood infection prior to ECMO	514 patients, <19 years, from multicenter data (2012–2014)
Pedi-Save	2022	Age; Cardiac diagnostic category; Race; STAT category; pH; Acid buffer requirement prior to ECMO; Number of cardiac procedures; Failure to wean from CPB as ECMO indication	10,091 patients with a cardiac diagnosis, <18 years, from ELSO registry; (2001–2015)
SAVE	2015	Age; Weight; Pre-ECMO organ failure; Chronic Renal failure; Pre-ECMO cardiac arrest; DBP before ECMO >40 mmHg; PP before ECMO <20 mmHg; Bicarb. <15 mmol/L; Duration of intubation prior to initiation of ECMO; Peak inspiratory pressure < 20 cmH2O	3846 patients with cardiogenic shock from; ELSO registry; (2003–2013)
Encourage	2016	Age; Sex; BMI >25; GCS <6; Creatinine; Lactate; Prothrombin activity	138 patients with AMI patients from multicenter data (2008–2013)
PREDICT VA-ECMO	2018	On-ECMO; pH; Lactate; Bicarb. Level	205 VA-ECMO from single center; (2010–2015)
Simple VA ECMO score	2019	Age; Duration of intubation; Lactate; Platelets; Albumin	100 adult patients with cardiogenic shock or cardiac arrest from multicenter data; (2010–2017)
CASUS	2018	ICU duration; Lactate; Pressure adjusted heart rate; Intubation status; Renal function; Platelet count; Neurologic status	90 adult VA ECMO patients from single center (2011–2012)

Table 2.

ECMO mortality prediction scores for cardiac failure.

ECMO = extracorporeal membrane oxygenation, ELSO = Extracorporeal Life Support Organization, VA = veno-arterial, DBP = diastolic blood pressure, PP = pulse pressure, bicarb = bicarbonate, AMI = acute myocardial infarction, CDH = congenital diaphragmatic hernia, MAS = meconium aspiration syndrome, PTT = partial thromboplastin time, INR = international normalization ratio, STAT = The Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery, CPB = cardiopulmonary bypass, PP = pulse pressure, Bicarb = bicarbonate, BMI = body mass index, GCS = Glasgow coma scale.

The Pediatric Survival after Veno-arterial ECMO (Pedi-SAVE) score was recently developed utilizing pediatric cardiac VA-ECMO patients from 2001 to 2015 from the ELSO registry from birth to 18 years of age [42]. The study included 10,091 patients, and both pre- and post-cannulation models were developed. Lowest risk patients in the pre- and post-cannulation groups had a 65% and 74% chance of survival respectively, compared to the highest risk groups having 33% and 22% comparatively. Pre-cannulation factors included type of congenital heart disease with better outcomes in patients with non-single ventricle physiology, age, pH, requirement for acid-buffer administration, number of cardiac procedures, the Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery score (STAT), combined cardiopulmonary failure and failure to separate from cardiopulmonary bypass prior to ECMO initiation (Table 2). The post-cannulation model included pump flow rates in the first 24 hours on ECMO and on-ECMO complications [42]. The authors concluded that the developed scores could serve in risk adjustment, comparing outcomes between centers and across eras. They also acknowledge the limitations of a registry data-based predictive score with lack of granular patient physiological data, inability to control for data completeness and most importantly the inability to generalize the score to direct individual patient care or guide whether to provide or withdraw ECMO support on the individual patient level.

2.2.2 Adult

Several adult prognostic scores exist for ECMO use in the setting of cardiac failure exist. The survival after veno-arterial ECMO (SAVE) score sought to identify pre-ECMO factors which predict survival to hospital discharge, it utilized 3846 patients from the extracorporeal life support organization registry between 2003 and 2013 [43]. The authors performed further validation on an external dataset of 161 patients who underwent VA ECMO support at a single institution. The developed score included 12 variables including underlying diagnosis leading to cardiogenic shock, age, weight, pre-ECMO organ failure, duration of mechanical ventilation prior to ECMO, pre-ECMO cardiac arrest and hemodynamic profile and degree of metabolic acidosis (Table 2). The score gave 5 risk classes with survival percentage ranging from 18 to 75%. On internal validation, the score had an AUROC of 0.68 that increased to 0.9 on external validation. The high accuracy may have been due to the validation set being a participant in the original registry, and it being a high volume highly regimented center. Their findings highlighted the importance of timing for ECMO initiation, with too early use exposing patients to unnecessary complications and too late use proving futile. The SAVE score also outperformed the SOFA score at both cannulation and ICU admission and other ICU severity of illness scores at discriminating patients who did not survive to hospital discharge, once again highlighting the importance of ECMO specific scoring methods [43].

Two other common adult cardiac prognostic scores were developed with more limited populations in mind. The prEdictioN of Cardiogenic shock OUtcome foR AMI (Acute Myocardial Infarction) patients salvaGed by VA-ECMO (ENCOURAGE) score was developed to predict in-ICU mortality from data on 138 patients from 2 ICUs between 2008 and 2013 [44]. Similarly, to the PRESERVE score the authors conducted quality of life assessments on ICU survivors, and found high rates of anxiety, depression and PTSD. Seven pre-ECMO predictive variables were identified (Table 2) including age, Glasgow coma score, creatinine, lactate, prothrombin activity, body mass index and sex. The lowest risk group had an 80% survival compared to the highest score group having only a 7% survival. In this group of patients with acute myocardial infraction, the ENCOURAGE score had an AUROC of 0.84 outperforming the SAVE score, and myocardial infraction scores such as the GRACE model [45], Dutch University Hospital Model [46] or the SHOCK Trial and Registry Scoring system [47]. Limitations included that development heavily relied on data from 2 highly specialized and experienced centers, limiting its generalizability, in addition some of the VA ECMO patients having low markers of impaired end organ perfusion (lactate and pH), suggesting either early ECMO initiation or less severe disease at initiation of ECMO.

The PREDICT VA-ECMO score was developed on a single center derivation cohort of 205 all-comers who received VA ECMO from 2010 to 2015, and validated on a cohort from an independent center from 2010 to 2017 [48]. In this work the authors set out to develop a dynamic model to predict hospital survival using on-ECMO variables at multiple time points. Two models were thus developed; the 6-hour PREDICT-VA-ECMO score utilizing variables at 6-hours post ECMO cannulation and the 12-hour PREDICT-VA-ECMO incorporating variables at 1, 6 and 12 hours post cannulation. The PREDICT VA-ECMO score out performed common severity of illness scores, as well as the SAVE score, but with higher accuracy with the 12-hour model compared to the 6-hour model (AUROC of 0.823 and 0.839 respectively), with comparable performance in the external validation cohort. Variables included lactate, pH and bicarbonate (Table 2). It demonstrated good prediction accuracy utilizing only a few variables which are easily obtained, and importantly was able to include evaluation of extracorporeal cardiopulmonary resuscitation (eCPR) patients.

Other published scores include a simple scoring system developed on a retrospective cohort of 100 patients between 2010 and 2017 at three institutions, to predict survival to discharge. Five variables were ultimately included, lactate >10 mmol/L, albumin <3 gm/dL and platelets <180,000/uL as well as age and duration of pre-ECMO mechanical ventilation. The lowest score predicted a mortality of 10%, while the highest saw 100% expected mortality [49].

Some studies have also indicated that lactate and urine output are independent predictors of mortality in extracorporeal life support patients, and that the cardiac-surgery score (CASUS) has moderate accuracy compared to simpler severity of illness scores such as SOFA [50].

2.3 Limitations of current approaches

Several limitations exist when evaluating the current landscape of ECMO outcome prediction tools. Although there is good evidence that these scores perform better than standard severity of illness scores, external validation has often shown decreased accuracy. Concerningly many of these scores performed poorly during the novel COVID-19 pandemic, at a time when patient selection and resource allocation was integral [31, 36, 51].

The reasons for poor external validation are likely multifold, while scores developed on large international multicenter registry data should have excellent external validity, this ignores that these databases often have very poor granularity. They fail to account for large center variations which may account for mortality differences [52, 53], including variations in anticoagulation strategies [54], ventilator strategies, timing of ECMO initiation and expertise that may exist in high volume centers [52, 53, 55]. This is especially true when the underlying population receiving ECMO is evolving, as during the COVID-19 pandemic. Static scores lend themselves poorly to changing dynamics in population, as ECMO use has continued to expand since many of these scores have been developed. Furthermore, scores developed utilizing large international registry data and at high volume ECMO centers may translate poorly to centers outside the registry or lower volume or lower resource centers. Particularly concerning is the fact that many of the pediatric and neonatal scores lack any significant external validation, limiting confidence in their deployment.

Another explanation for many of the current generation of ECMO mortality scores’ modest performance may be the tendency to rely on accuracy as a measure of performance. Accuracy while made up of sensitivity and specificity, may be less reliable if an outcome of interest is less common. Furthermore, clinicians at the bedside frequently care about positive and negative predictive values, but these are based off prevalence. If centers have large variations in the prevalence of mortality, scores may experience wide variance in their ability to predict the outcome of interest.

Using data from both an institutional database of 15 hospitals in a quaternary referral center and a multinational dataset spanning 42 countries from the COVID-19 pandemic, we evaluated the performance of several ECMO mortality prediction scores; RESP, PRESERVE, Roch and PRESET [56]. In order to more comprehensively evaluate the scores’ performance, we reported both accuracy with AUROC and precision with area under the precision recall curve (AUPRC) (Table 3). Our results were consistent with previous reports during the pandemic with modest performance of all the studied scores, further emphasizing their limited clinical applicability, especially in the setting of global healthcare system resource limitation.

Dataset		ECMONet	RESP	PRESERVE	Roch	PRESET
Published Datasets	AUROC	0.61	0.64	0.69	0.63	0.66
	AUPRC
	N	404	601	792	430	388
International dataset	AUROC	0.57	—	—	0.63	0.55
	AUPRC	0.49	—	—	0.71	0.56
	N	328	—	—	214	352
Institutional Dataset	AUROC	0.54	0.53	0.59	0.53	0.61
	AUPRC	0.48	0.49	0.58	0.61	0.61
	N	60	54	21	50	50

Table 3.

Performance of commonly available ECMO mortality scores in the COVID-19 pandemic.

AUROC = area under receiver operator curve, AUPRC = area under precision recall curve.

The current generation of ECMO mortality scores also fails to provide guidance to clinicians in what may be the most vital decision – timing of ECMO initiation. The lack of data regarding the patients’ pre-ECMO evolving trajectory and reliance on static variables often immediately prior to ECMO cannulation, in addition to the lack of a true matched non-ECMO control cohort, significantly limit the utility of all the current scores as clinically applicable decision support tools.

Finally, only two of the scores (ENCOURAGE and PRESERVE) included any data on outcomes aside from mortality, providing information related to quality of life. As mortality improves it likely will become increasingly relevant to begin predicting on meaningful clinical outcomes like neurological function and quality of life.

3. Future directions

The continuously expanding use of ECMO internationally has been coupled by an evolution in biomedical informatics, artificial intelligence and machine learning methodology. This evolution has opened the possibility of exploring more complex machine learning methodologies to develop predictive models to better mirror the clinical decision-making dilemma.

The COVID-19 pandemic has the unmasked the collaborative potential across institutions and even nations to share healthcare data in real time. This opens the possibility to develop more granular multi-institutional databases that expand the currently available registry data. Such databases would open the potential to not only incorporate variables well in advance of ECMO cannulation to better correlate the outcomes with patient trajectories, but also allow the comparison with a propensity matched cohort of non-ECMO critically ill patients to mirror the influence of decision and timing of provision of ECMO support.

The advances in ECMO technology, expanding candidacy and improved outcomes, demonstrate that the use of mortality as an objective outcome has become insufficient. The current landscape of ECMO use mandates a transition from survival to functional neurological outcomes as the goal of predictive modeling. Such an approach requires agreement in the ECMO community on clearly defined definitions for goal neurological outcomes, a task well overdue. Such objective outcomes could then be used as goals in predictive model development to better understand the influence of timing and provision of such high-risk resource-intensive therapy on both individual and system levels.

4. Conclusions

Over the decades since the first deployment of ECMO its application has exponentially increased, and there has been growing interest in developing tools to guide clinical decision making, to help aid in patient selection and prognostication. While several scores have been developed, they share similar limitations secondary to the granularity of the available data, score development approaches, the focus on mortality as the main outcome of interest and reliance on data only from patients supported on ECMO. This has led to continued efforts to refine and a requirement to continuously update these scores. Future directions include a transition from a mortality focused approach to an approach focused on identifying objective short and long-term neurologic outcomes. Additionally, there is a need to develop tools capable of matching the studied ECMO cohort to a non-ECMO cohort of similar severity of illness and then develop tools capable of aiding in both patient selection and determination of the optimal ECMO initiation time to improve both mortality and neurologic outcomes.

Conflict of interest

The authors declare no conflict of interest.