COVID-19 Sparks Research Into Modelling Future Pandemics

Astrology, dream interpretations, palmistry - humans are incredibly fascinated by the future and go to great lengths to find out what it holds. Naturally, this tendency was manifested during the recent pandemic. Over the past several years, countless strategies were developed to predict how COVID-19 would pan out. When computational and statistical methods gave unreliable predictions amid increasingly complex transmission dynamics, researchers looked to machines for help.

Machine learning has become the tool for pattern recognition in many modelling contexts. As a subset of artificial intelligence (AI), machine learning enables software to learn and become increasingly competent at predicting outcomes without humans needing to set explicit parameters. Deep learning, then, is a further subset of machine learning with a very specific structure that consists of many layers of algorithms known as an "artificial neural network" (ANN). The combination of an input, output and a set of hidden layers work together to optimise parameters, which produce a function that best fits the observed trend. This process is often likened to the neural network of humans and other animals, which similarly has evolved to learn from observing.

In a standard ANN, inputs travel sequentially from the input layer, through the hidden layers and finally to the output layer. In an RNN, however, inputs travel recursively to learn better. In essence, if we visualise the learning method of an ANN as a linear process, then that of an RNN looks more like a recurring cycle. One of the best RNN-based algorithms is the Long Short-Term Memory (LSTM) because it overcomes the major problem of a vanilla RNN - the inability to handle long-term dependencies. As the name suggests, LSTM has both long and short-term memory, yet the two work together to retain patterns and information from very early data points and so make it an excellent candidate for making predictions due to the temporal nature of forecasting.

Earlier in the pandemic, a team of researchers trained, tested and used LSTM models to forecast the number of confirmed COVID-19 cases in Canada [1]. Their LSTM model-1 was trained and tested using a Canadian dataset with a small root-mean-square error (RMSE) of 34.83, outperforming many other forecasting models. RMSE is a common error metric that calculates the average difference between the predicted and observed value. With a promising RMSE, the team then used this newly trained LSTM to question when the outbreak would end in Canada. The model calculated an end date in June 2020, which impressively reflected reality as the number of positive confirmed cases began to fall in May. However, as the nation progressed into a second wave of outbreak in Autumn, the algorithm failed to capture similar fluctuations as it had no concept of how infections come and go in waves during a pandemic. This highlights a fundamental limitation of deep learning: machine learning algorithms are only capable of using historical datasets and patterns to make predictions; hence, unexpected events that may, or may not, occur in the future are impossible to be accounted for.

Although, as the saying goes, ‘all models are wrong, only some are useful’, it would be impetuous to eliminate the possibility of a model to be accurate despite having a high RMSE. Another team of researchers trained several variations of LSTM using an Indian dataset from the first (2020) and second (2021) waves of infections. Shockingly, their study achieved a RMSE of almost 20,000 [2]. Although this implied that their prediction uncertainty was relatively large, the team could still obtain an average prediction for every date from October to November 2021. Overall, their results were largely accurate, especially in predicting the decreasing trend in daily positive cases towards the end of November 2021. Further, it was able to successfully predict that the number of daily confirmed cases during this period would decrease towards 10,000.

Arguably, such studies point towards a future where deep learning software would be a crucial tool for pandemic prevention. Importantly, that it should be used early on in an outbreak where data is scarce is of utmost importance. In remedying the limitations of statistical methods to choose optimal parameters used for fitting limited data, LSTM can also be trained to select hyperparameters by fitting real-time data without assumptions, allowing it to significantly improve prediction accuracy.

Despite its potential to outperform traditional statistical models, LSTM and other machine learning algorithms have to be used with caution. As algorithms get increasingly deep and complex, so do their difficulty of understanding them. In other words, their ‘blackbox’ nature amplifies so much that even the programmers who code them lack thorough understanding of their prediction algorithms. This results in lower interpretability and potential for application. Regardless of how impressive LSTM algorithms seem, they cannot be used in isolation. Traditional epidemiological models still have a place in pandemic prevention precisely because they are simple, transparent and highly flexible to account for cultural, societal and regulatory considerations, for example. After all, this defines the fine line between science and fortune-telling; we aren’t satisfied merely by knowing what the future holds, but why it turns out that way.

References

[1] Chimmula VKR, Zhang L. Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos, Solitons & Fractals. 2020 Jun;135:109864.

[2] Chandra R, Jain A, Singh Chauhan D. Deep learning via LSTM models for COVID-19 infection forecasting in India. Chen CH, editor. PLoS ONE. 2022 Jan 28;17(1):e0262708.