3 - Quality of this prediction

Epidemiological modelling and its use to manage COVID-19

Insights into mechanistic models, by the DYNAMO team

Over the next few weeks, we will present some key elements of epidemiological modelling through short educational articles. These articles will help you to better understand and decipher the assumptions underlying the epidemiological models that are currently widely used, and how these assumptions can impact predictions of the spread of pathogens, particularly SARS-CoV-2. The objective is to discover the advantages and limitations of mechanistic modelling, an approach that is at the core of the DYNAMO team's work. The examples of models will be inspired by models used in crisis, but sometimes simplified to make them accessible.

#3 – What about the predictive quality of models at the beginning of an epidemic?

Before reading this article, we advise you to first read article #1 and article #2.

Now imagine that we are at the beginning of an outbreak of a new disease, for example COVID-19! Here is what was observed in France at the beginning, during the months of February and March 2020, in terms of the number of cumulative deaths:

Nombre de décès cumulées par jour selon 4 jeux de paramètres (zoom sur le début de l'épidémie)

Data observed in France on cumulative mortality due to COVID-19
 between January 1st and March 6, 2020 (points),
and predictions (lines) of the SAIRM model for 4 different parameter sets,
 defined in the table below.

Let us superimpose to the observed data the predictions of the SAIRM model described in the previous articles for four different sets of parameters. Visually, it is difficult to decide which scenario will be the most realistic! None of them is perfect, the model being in essence a simplification of reality. However, this "small" model already has many parameters, which cannot be estimated precisely at this stage because there is "not enough" available data... And yet, at the last point, we are ten days before the start of lockdown!

Parameter

Definition

Set 1

(1-p)

Proportion of symptomatic infected at risk of dying

1%

50%

20%

20%

βA

A-transmission rate (per individual per day)

0.296

0.40

0.15

0.10

βI

I-transmission rate (per individual per day)

0.293

0.20

0.30

0.305

1/γA

Average time in A (days)

7.5

5

14.3

25

1/γI

Average time in I (days)

27.5

16.7

33.3

27.8

α

I mortality rate (per day)

0.0073

0.00015

0.0073

0.015

date_intro

Date of virus introduction

7/01

7/01

10/01

9/01

Parameter values for the 4 scenarios.
See article #2 for parameter definitions.

One might want to use these four predictions interchangeably, or even make the model more complex to make it more realistic, and thus add new parameters whose value is still unknown. The problem is that the four dynamics predicted in the longer term quickly become very different :

morts_SAIRM_selonParams

Predictions of the 4 scenarios with a longer time horizon,
between January 1st and June 18 (170 days later):
even the 2 scenarios that seemed very close (black & green)
have a 20,000 gap in death toll after another 100 days!

Every day counts in this race to make predictions, bringing new knowledge and new observations useful for specifying parameters, but also new cases and new deaths... Of course, once at the end of the epidemic, it will be much easier to correctly calibrate models, even the most complex ones, and to predict the epidemic dynamics on the basis of data that have become historical! 

Article #4 will discuss the model's assumptions regarding the onset of the epidemic.