Predicting Perioperative Mortality in Children: Academic Endeavor or Clinical Value?

Predicting the risks versus benefits of perioperative care associated with a surgical admission or intervention is a core component of both informed consent and medical decision-making, including preplanned allocation of resources and escalation of care such as intensive care admission. Therefore, research aimed to increase our ability to predict postoperative outcome with reasonable probability before anesthesia and surgery occupies one of the top positions on the priority list of healthcare providers involved with the perioperative care of patients. A substantial number of risk prediction scores have been developed for adults, and they are increasingly applied in these populations. While several prediction scores have also been described in the context of pediatric perioperative care, their use is still very limited.

In this issue of Anesthesiology, Tangel et al. provide us with a comprehensive review of the preoperative prediction scores assessing perioperative mortality in pediatric patients.¹  The authors conducted a systematic literature search and identified 10 studies reporting the development and/or validation of risk scores that predicted all-cause mortality up to 30 days postoperatively after anesthesia/surgery in pediatric populations. They assessed the quality of these studies in terms of (1) risk of bias using the Prediction Model of Risk of Bias Assessment Tool (PROBAST), an expert opinion-based tool for assessing the risk of bias and applicability of diagnostic and prognostic prediction model studies² ; (2) clinical applicability using the Grading and Assessment of Predictive Tools for Clinical decision support (GRASP)³ ; and (3) feasibility or ease of use in the clinical setting. This analysis revealed that while most scores are easy to use in the clinical setting and present good discrimination upon (mostly internal) validation, they all have an overall high or unclear risk of bias principally due to factors associated with analytic techniques. No single score emerged as qualitatively better than any other, and they are all still in the lowest phase of evaluation in terms of clinical applicability.

What do these findings teach us? Based on their thorough analysis, the authors’ conclusion is that none of the currently existing pediatric models can be recommended for use in everyday clinical practice to predict perioperative mortality in children. Does this scientifically justified conclusion mean that, at the current state of knowledge and while awaiting further model development, practicing pediatric anesthesiologists should dismiss pediatric mortality risk scores from their armamentarium? Before taking such a radical step, it may be worth reflecting upon the potential benefits that existing prediction scores can add to our practice, despite their well-documented weaknesses. While the risk of perioperative mortality may differ from one center to another, it is unlikely that the magnitude of these differences will fundamentally change decision-making and outcome when pediatric anesthesia and surgery are conducted with competency. Therefore, with all caveats acknowledged, data from already existing risk scores can be helpful when discussing probabilities of outcome with parents or health care professionals before anesthesia/surgery, especially in high-risk children with multiple risk factors.

Should we primarily focus on mortality when describing perioperative outcome in children? If not, what are the alternatives? To answer these questions, it is important to consider that development of reliable risk prediction scores must build upon clearly definable outcome measures that, ideally, occur at a reasonable frequency to allow validation with ease. These two prerequisites rarely coexist in pediatric perioperative care. There are basically three different sets of outcomes we could potentially consider when aiming for risk prediction models: mortality, morbidity, and adverse events. Among them, mortality is the best-defined outcome since it is binary and very important. Fortunately, perioperative mortality is very low in children (the unadjusted rate of mortality in pediatric risk prediction models ranges from 0.3 to 1.5%).¹  However, this low incidence rate makes risk prediction technically challenging. According to expert consensus on the development of risk prediction models, development of risk prediction scores requires a minimum of 20 events per predictor, while validation should be based on a minimum number of 100 events per predictor.²  Morbidity (i.e., the incidence/prevalence of disease), while less clearly defined than mortality, is another frequently used outcome measure of risk scores in adult populations. In children undergoing anesthesia/surgery, we know very little about the overall incidence of morbidities such as myocardial infarction, renal failure, stroke, or other. Therefore, while organ-specific morbidity may be an appropriate outcome measure of risk prediction in specific pediatric populations undergoing surgery (e.g., children with congenital heart disease), its inclusion in models applicable to general populations remains difficult. Most epidemiologic studies in pediatric perioperative care focus on adverse events. Adverse events are usually defined as “any unexpected medical occurrence.” The problem with this definition is that it is highly subjective since some healthcare provider may consider an “event” as unexpected, while for others, the occurrence of this same event is totally expected. Importantly, while there are some (causal) associations between organ morbidity and mortality, it remains to be determined how “adverse events” relate to clinically meaningful outcomes. Considering all these pitfalls and caveats related to different kinds of outcome measures, we believe that the development of prediction scores focusing on mortality is still the most feasible, reliable, and straightforward approach to predict perioperative risk in general pediatric populations.

When and how can we implement pediatric risk prediction scores into our clinical practice? Shall we create new ones? Stick to existing ones? If the latter, which one to choose? There are no easy and straightforward answers to these questions. The review by Tangel et al. draws careful attention to both the strengths and weaknesses of existing models and, thereby, helps to define the future research agenda. One of the principal messages of their analysis is the lack of independent external validation. This weakness should not be underestimated since an increasing number of studies demonstrates reduced accuracy of prediction rules when validated in new patients.⁴  If external validation is indeed a top priority of the research agenda, the next question is which of the nine so far available pediatric models should be validated. Again, there is no easy answer here. Models differ from each other, and appropriate external validation, in light of the low incidence of mortality, necessitates a huge and joint effort among multiple centers. One potential pragmatic approach, among others, would be to focus on the Pediatric Risk Assessment Score (PRAm), which is based on a powerful multivariable regression model and is easy to implement, and a nonindependent prospective external validation has already been performed.⁵  Ideally, this validation should be conducted in concert with several continents and settings. Assuming appropriate and satisfactory validation of existing models, do we have the place for creating (and validating) new ones? As we collect ever more data with increasing granularity, there is certainly a place for improvements in predictive models. Given the effort necessary for the validation of pediatric mortality prediction models, the practical question is whether we need to heavily invest in that direction if we already have satisfactory models at our disposition. In line with these thoughts, one should never forget that validation does not automatically mean clinical usefulness. Once validation of a model is reasonably complete, the next important step should be to evaluate if it indeed influences physician behavior and patient outcome. The final step would be to study the implementation of prediction rules in everyday clinical practice. These last two stages in the evaluation process of risk scores are of utmost importance but have not been addressed so far. The research agenda is thus well packed with straightforward goals. It is for us to decide whether we want to tackle them. The resulting answers will help us decide whether risk prediction scores are merely an academic endeavor, or they go beyond that to represent meaningful clinical value in the perioperative care of children.