During a Pandemic, Beware the Man of One Model

“I have great respect for the scientific model builders who attempt at quantifying the uncertainty we face and those policy advisors who are willing to accept differences of the outputs and predictions from alternative models. At the same time, I worry when policymakers embrace models without a full understanding of the underlying assumptions or which deliver the findings that they prefer to see,” argues Nobel laureate Lars Peter Hansen.

I am an economist who uses dynamic models and who explores uncertainty impacts in a variety of settings. I use tools from statistics and decision theory to investigate both market behavior and implications for policy.

Like the rest of the population, I now find myself in rather surreal and, at times, dire circumstances. I wish, but do not claim to be an expert in epidemiology.

Like many, I have been trying to give myself at least some basic knowledge of pandemics and models aimed at understanding how this disease can evolve dramatically over a short period of time.

As an outsider, I find the modeling impacts from epidemiology of considerable interest. In the mornings, I search immediately for updated numbers and predictions, hoping that we will soon see an important turnaround in Covid-19 cases, deaths, human suffering, and the subsequent global socioeconomic turmoil.

Why Quantitative Models?

Policymakers look to forecasts or projections about the future evolution of contagion and subsequent fatalities to guide their policy choices. These can be best guesses or warnings about how bad things could become. These considerations factor into their decision making in at least informal ways.

Epidemiologists no doubt have important insights that we all look to digest. Economists and other social scientists are quick to consider ways by which they can draw upon their current stock of knowledge to incorporate endogenous responses of individuals and businesses to various policy alternatives. Quantitative predictions of disease transmission under alternative policies and the resulting social behaviors, however, bring special challenges.

The reason is that models require specific assumptions and ingredients that govern the dynamic evolution and consequences of alternative forms of social and economic interactions. Subjective judgements are unavoidable. There are unknown parameters to calibrate in the face of limited data.

These challenges are pervasive in quantitative modeling that aims to support policy. The unique challenges of the Covid-19 global situation are what draws our attention as we witness and participate in this harrowing episode.

Looking Across Models

Different researchers or research groups build models with different implications. I have great respect for the scientific model builders who make revealing attempts at quantifying the uncertainty we face and those policy advisors who are willing to accept differences of the outputs and predictions from alternative models. At the same time, I worry when policymakers seemingly embrace models without a full understanding of the underlying assumptions or which deliver the findings that they prefer to see.

I find it insightful to think of each quantitative model for this phenomenon and other policy-relevant behaviors as providing a quantitative story. Each model may seek to offer guidance and insight, but alternative models may have different predictions and implications for policy.

By nature, each model is an abstraction and necessarily a simplification, and sometimes the approximation can appear to be bold. There are uncertainties within each model having to do with unknown inputs, and there are differences across models in terms of how they aim to depict behavior. When thinking about using models in a variety of settings, including our current health and economic crisis, I am reminded of Saint Thomas Aquinas’ alleged comment:  “Beware the man of one book.”

We should replace the word “book” with “model” when our understanding of the phenomenon in question has such apparent limits. Looking across the predictions across models is a valuable exercise. Substantive expertise can help in weighing the pros and cons of alternative models, but when there are obvious bounds to our understanding, this seldom rules out all but one model.

Looking to Data

We look at data to help calibrate inputs, but many concerns have been raised about data quality and its reliability. Even at a more rudimentary level, we are unsure of the actual numbers of contaminated people.

Death attributions are challenging because unhealthy people are substantially more vulnerable to the disease. We do not know yet know how strong the immunity of those who have already been affected by the virus and survived is. We look to evidence of earlier experiences, such as from China where the disease and its initial transmissions started. But serious concerns have been raised about the officially quoted numbers there and elsewhere.

If only we could just “let the data speak,” but that is not how most model building proceeds. There has to be some guesswork in terms of determining how best to exploit the evidence we have from previous experiences.

Data limitations make it challenging, even for experts, to assess the merits and limits of alternative models and predictions.

Where Does Economics Come In?

Policy-relevant modeling entailing such phenomena isn’t just about epidemiology. Inside the models are individuals making decisions about social interactions and businesses responding to new economic demands and policy restraints. The people “inside the model” respond to changes in their environment and policies that might be implemented along the way.

The economist in me has been observing a quantitative macroeconomics literature quickly emerge that incorporates simplified epidemiological specifications of disease within a macroeconomic framework in the face of the crisis.

To my colleagues’ credit, they aim to address important policy challenges and to introduce endogenous responses to changing incentives. They explore the health benefits and economic consequences of quarantining a significant portion of the population and the best ways to use testing to improve the social and economic outcomes of the current crisis.

We know from a variety of experiences that incentives can matter when assessing policies. But it is no small feat to incorporate epidemiological forces within dynamic models of the economy in credible ways, even putting aside how best to confront the overriding uncertainty.

My guess or hope is that much of this quantitative modeling literature that is merging in the fields of economics and epidemiology will help us to design policies to better confront future pandemics, as this one is unfolding at a much faster rate than the necessary scientific advances needed to produce new and better-integrated models, inclusive of the social sciences.

In my view, for these efforts to be successful, it will require that uncertainty be incorporated formally into the modeling and not treated as an afterthought.

Uncertainty and Trade-offs

Economists identify and assess tradeoffs pertinent to the conduct of prudent policy,  which even at a qualitative level is an important contribution. Indeed there are extremely tricky economic and social trade-offs that policymakers must cope with, although some have suggested naively that we should put them aside. For instance, we cannot quarantine everyone and leave society without access to food and necessary pharmaceuticals.

Exactly where we draw the line entails a trade-off between protecting people from exposure to the virus and making the accessibility of necessary food and medicine difficult and costly.

When exactly do we choose to remove restrictions on various social and economic activities as we emerge from this pandemic? Such assessments clearly involve weighing costs and benefits of alternative courses of action.

How we use alternative model predictions to guide policy also exposes a trade-off that warrants serious consideration. When policy advisors explore alternative courses of action, they are necessarily unsure of the consequences. Various projections get reported in the press about how the infections and resulting deaths will evolve in the future.

We are keenly interested in when things will turn around. In sifting through projections reported in the media, we encounter a wide range of outcomes. On more careful inspection, an important reason for some of the differences is that they represent different protocols or conventions. Some projections represent “best guesses” and others represent “worst-case” possibilities.

Even the term “worst-case” is a misnomer, as even these forecasts are typically premised on “reasonable” bounds in terms of their model inputs. Both types of projections can be informative as long as it is understood that they serve different but related purposes.

In formal or even informal approaches to addressing urgent social problems, we are confronted with how much weight or attention we should attach to the alternative health trajectories that might play out.

How much attention should be paid to our best guesses of how the disease will evolve under alternative policies relative to the more cautious examination of “worst-case” trajectories whereby the number of infections and deaths are much more severe?

There are “in-between” possibilities as well. Best-guess and worst case-type simulations, and for that matter, even in-between ones expressed using probabilities, are all revealing. I believe it is the role of the media to do a more
balanced job of reporting the options, and it should aim to be more transparent with the public about assumptions made for each simulation.

However, these simulations alone do not inform us of the best course of action. This is true even if we were in a simpler setting in which we could assign probabilities with great confidence. Determining prudent policy choices includes taking a stand on how concerned or averse we should be to uncertainty. This goes beyond merely assigning probabilities to alternative outcomes.

How much attention should we pay to the potentially bad outcomes in contrast to our best guesses of the disease and fatality forecasts, when exploring alternative courses of action? Why do I call this a trade-off? Going with the best guesses may leave us vulnerable to very bad outcomes.

Featuring only so-called “worst-case” analysis in future policy considerations is not some panacea either. Embracing this approach could potentially induce subsequent social losses when unlikely worst-case outcomes do not emerge. It is these types of considerations that I wish were formally integrated into the economic analysis of policy as it applies here and to other policy challenges.

Policy advisors necessarily confront this trade-off when they look at alternative model projections. I am a firm believer that models can provide useful frameworks for prudent policy design provided that they are used sensibly and without unjustified confidence in their predictions. Existing quantitative models are tools that give stories we should take seriously when done by experts who are willing to acknowledge limitations.

This willingness should be a virtue and not a vice. The modest amount of good news is that new information now flows quickly and openly to challenge model predictions and to lead to updates in information and the performance of implemented measures.

There have been some remarkable changes in model predictions of new infections and fatalities in response to the most recent evidence. The emerging body of evidence will no doubt lead to important modeling advances in the future.

I only wish that I, and other academics, could provide even better quantitative ways to guide policy in this challenging time. We are living in what Mark Twain referred to as the “miserable uncertainty” that comes with the bounds to our understanding.

But as scholars with quantitative ambitions are seeking to distill and process what information and insights are now unfolding at a rapid pace, I can only applaud the sensible policymakers as they weigh the alternative possible outcomes in real time.

Thankfully, I work in a state (Illinois), a city (Chicago), and at a university (the University of Chicago) in which our leaders are showing good acumen in these difficult circumstances. Such leaders are placed in the hot seats of having to implement sensible policy over the very short scale during which this pandemic is unfolding and in the face of obvious uncertainty. Unfortunately, political agendas often get in the way of sensible policymaking.

While economists struggle to come up with the best way to model individual altruism, I can only hope that, at least for this episode, altruism is much more prevalent than it is in the models that economists typically use.

Along these lines, I am continually reminded of the socially conscious contributions of the real heroes from our healthcare system whom we are placing on the frontlines of this global crisis, and who are risking their personal health to support that of their own communities.

Lars Peter Hansen is the David Rockefeller Distinguished Service Professor in Economics and Statistics, the Kenneth C. Griffin Department of Economics and the Booth School of Business; Director of BFI’s Macro Finance Research Program. He is a recipient of the 2013 Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel for his early research.

This essay was originally published in the University of Chicago Booth School of Business Becker-Friedman Institute series of working papers on Covid-19 crisis. You can find the rest of the series here. 

ProMarket is dedicated to discussing how competition tends to be subverted by special interests. The posts represent the opinions of their writers, not necessarily those of the University of Chicago, the Booth School of Business, or its faculty. For more information, please visit ProMarket Blog Policy.