Citing more data and better models, the Institute for Health Metrix and Evaluation has released an updated model for predicted COVID-19 deaths and hospital resources.
Key methodological and data updates are summarized below, as well as major results; more detailed information in the form of technical appendices will be published online by Tuesday, April 7.
PREDICTING COVID-19 DEATHS: WHAT HAS CHANGED AND WHY
Compared with our release on April 2, today’s predictions indicate the same peak date for daily COVID-19 deaths in the US – April 16 – but some states have seen shifts to earlier timing of COVID-19 peak deaths. At this peak date, the country’s daily COVID-19 deaths are now estimated to reach 3,130 (estimate range of 1,282 to 7,703) (see below) – again, similar to our last release. Our updated national projections for cumulative COVID-19 deaths through the first wave are lower than our April 2 model, with an estimate of 81,766 total COVID-19 deaths (estimate range 49,431 to 136,401).
Prediction measure for COVID-19 deaths | Predictions from our April 5 release (today) | Predictions from our April 2 release | Change of average values since the April 2 release* |
Peak date for daily deaths | April 16 | April 16 | 0 days |
Daily deaths at peak date | 3,130 (1,282 to 7,703) | 2,644 (1,216 to 4,136) | ↑ 485 deaths |
Cumulative deaths through first wave | 81,766 (49,431 to 136,401) | 93,531 (39,966 to 177,866) | ↓ 11,765 deaths |
*Change estimates do not include uncertainty; they are only based on the average value. If prediction values’ uncertainty intervals (the numbers reported in parentheses) overlap a lot across different releases, changes in these estimates are not considered substantively different.
Why did these changes occur? Even though at the national level the total number of deaths remains fairly unchanged, today’s release introduces substantial changes for several states. We present the major sources for these changes here.
Since our last release, IHME has made four substantial updates to the COVID-19 death model; we summarize them here.
Model update 1: Epidemic peak and its relationship to social distancing policies.
- At the time of our first release on March 26, the only location where the number of daily deaths had already peaked was Wuhan City. These data from Wuhan formed the basis of our estimation of the time from implementation of social distancing policies to the peak day of deaths. Since then, an additional seven locations in Italy and Spain with large coronavirus epidemics appear to have reached the peak number of daily deaths (see below): two in Spain (Castile-La Mancha and Madrid), and then five in Italy (Emilia-Romagna, Liguria, Lombardy, Piedmont, and Tuscany).
- With today’s update, we now estimate the time from implementation of social distancing policies to the peak of daily deaths using all eight locations where the number of daily deaths appears to be peaking or to have peaked. The time from implementation of social distancing to the peak of the epidemic in the Italy and Spain location is shorter than what was observed in Wuhan. As a result, in several states in the US, today’s updates show an earlier predicted date of peak daily deaths, even though at the national level the change is not very pronounced.
Model update 2: Weighting of social distancing policies in the COVID-19 death model.
- Our previous models had a composite measure for estimating the effects of social distancing policies on the course of the epidemic. Until today’s release, all types of mandate had been combined together by giving them equal weight. Of the four social distancing policies we currently track (i.e., school closures, stay-at-home orders, non-essential business closures, and travel limitations), no state had enacted a travel limitation mandate that met our criteria until quite recently (i.e., Alaska, implemented March 28 and incorporated into our April 2 release). Subsequently, we continue to primarily focus on the three measures – school closures, stay-at-home orders, and non-essential business closures – more widely implemented by US states to date. If or as more states mandate severe travel limitations, we will revisit its inclusion into our updated weighting approach.
- Starting with today’s release, we use three different weighting schemes to better approximate variation in potential policy impact across social distancing mandates. Our statistical tests suggest that early actions such as school closures may have a larger effect in some communities than expected; in other locations, fully mandated stay-at-home orders may be required for detectable impact. To better capture this community-by-community difference of mandates on social distancing behaviors, we now use an ensemble model of three different weighting schemes. More details will be included in the technical paper that will be released by Tuesday, April 7, and will be published here.
Model update 3: Uncertainty estimation and out-of-sample predictive validity.
- As of today’s release, we have changed the way we estimate uncertainty in our COVID-19 predictions. We now run our models holding out data for each location and evaluate how the model performs in predicting number of deaths at various points in the future. We then incorporate this information into our estimates of uncertainty for future-date predictions. This method of evaluating uncertainty is more rigorous and reflects model performance.
- More specifically, we evaluate model performance both in terms of how many days into the future we estimate and how many data points have been reported for each state. As expected, projection accuracy improves when there are more data on the number of daily deaths in each state and gets worse the farther into the future projections are made. Model projection accuracy is now included in our prediction uncertainty estimates.
Model update 4: Incorporating adaptive priors for states with low cumulative COVID-19 deaths.
- As mentioned in our April 1 estimation update, model performance can be less consistent from day to day for states with very few total data points or few cumulative COVID-19 deaths.
- Our out-of-sample predictive validity tests have now identified that this is particularly pronounced for states that have fewer than five data points.
- In an effort to better capture these locations’ trends amid limited model inputs, we now use adaptive priors in the statistical model estimation. We wish to highlight that we are no longer using the model specification where the social distancing measure was a covariate on the final death rate (i.e., p-model; see original manuscript’s methods section for details) due to its poor performance in out-of-sample predictive tests.
- The use of adaptive priors and dropping the p-model have led to improvements in model performance and stability in settings with limited data. However, it is important to note that for these states, especially during the first few days of including their new data, our predictions are likely to vary considerably from release to release.
PREDICTING HOSPITAL RESOURCE USE: WHAT HAS CHANGED AND WHY
As informed by our updated death models, predictions from today’s release show lower overall hospital resource needs in the U.S. for the three indicators we measure (see table below).
While the predicted peak date of overall hospital use has remained the same – April 15 – at the national level, we now project that a total of 140,823 hospital beds (estimate range of 73,390 to 284,569) could be needed on that day. In contrast, our April 2 release projected 262,092 total hospital beds (estimate range of 123,885 to 417,672) could be needed to support COVID-19 patients. Today’s update also suggests lower levels of predicted need for ICU beds and invasive ventilators than previous model estimates; however, their prediction differences are far less pronounced than what we now see for total hospital beds.
Hospital use prediction measure | Predictions from our April 5 release (today) | Predictions from our April 2 release | Change of average values since the April 2 release* |
Peak date for hospital resource use | April 15 | April 15 | 0 days |
Total hospital beds need at peak | 140,823 (73,390 to 284,549) | 262,092 (123,885 to 417,672) | ↓ 121,269 |
ICU beds need at peak | 29,210 (16,149 to 55,869) | 39,727 (16,258 to 67,670) | ↓ 10,517 |
Invasive ventilator need at peak | 18,992 (10,533 to 36,318) | 31,782 (13,016 to 54,113) | ↓ 12,791 |
*Change estimates do not include uncertainty; they are only based on the average value. If prediction values’ uncertainty intervals (the numbers reported in parentheses) overlap a lot across different releases, changes in these estimates are not considered substantively different.
Why did these changes occur?
These changes in predicted hospital resource use and related gaps in states where demand might exceed supply are fairly large. In this section, we explore the changes in our analytical framework that resulted in these revised estimates of overall lower hospital resource use due to COVID-19.
First, we have been able to include more up-to-date data for estimating ratios of hospital admission to deaths. These ratios inform model parameters that are used to predict need for hospital beds, ICU beds, and ventilators. In previous releases of our estimates, our ratios were informed by a CDC report with information on early COVID-19 cases in the US – from February 12 to March 16 – and those patients’ outcomes. Based on these data (509 admissions divided by 46 deaths), our overall ratio was 11.1 hospital admissions per COVID-19 death.
Over the last few days, we have been able to incorporate data sources, including data provided by state governments, on a substantially larger sample: 16,352 hospital admissions and 2,908 deaths related to COVID-19. This allowed us to estimate state-specific ratios where data were available on at least 50 deaths from COVID-19, using random-effects meta-analysis.
Our estimates released today use the state-specific ratios noted below and for those states without data, the pooled ratio of 7.1 hospitalizations per death (95% CI 4.0 to 12.7). These lower ratios of admissions to deaths result in predicted peak hospital resource use – total beds, ICU beds, and invasive ventilators – that is lower than previously estimated.
Based on new data, we also re-estimated the length of stay in hospital for COVID-19 patients. The latest data show that, among hospitalized COVID-19 patients who are discharged alive, patients who require ICU support have longer hospital stays than previously predicted, and patients who do not require ICU support have shorter hospital stays. The table below compares key parameters in the hospital model from the previous releases and today’s release. The long hospital stays for patients requiring an ICU stay mean that the reduction in ICU beds and ventilator use due to the lower admission-to-death ratios is less than for total beds.
Hospital length-of-stay parameter | For our April 5 release (today) | First release (March 26) to April 2 release | Change in parameter values since the April 2 release |
Time from hospital admission to death (among COVID-19 patients who did not survive) | 6 days | 8 days for patients ≥ 70 years 10 days for patients < 70 years | ↓ 2 to 4 days |
Time from hospital admission to discharge, with ICU stay | 20 days | 8 days | ↑ 12 days |
ICU length of stay | 13 days | 8 days | ↑ 5 days |
Time from hospital admission to discharge, without ICU stay | 8 days | 15 days* | ↓ 7 days |
*Based on a 12-day length of stay for all COVID-19 patients admitted and discharged alive.