Download as pdf or txt
Download as pdf or txt
You are on page 1of 14

medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020.

The copyright holder for this preprint


(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

The Effect of Stay-at-Home Orders on COVID-19 Infections in the United States

James H. Fowler​1,2​, Seth J. Hill​2​, Remy Levin​3​, Nick Obradovich​4

1​
Infectious Diseases and Global Public Health Division, University of California, San Diego
2​
Political Science Department, University of California, San Diego
3​
Economics Department, University of California, San Diego
4​
Center for Humans and Machines, Max Planck Institute for Human Development

Summary
Background ​In March and April 2020, public health authorities in the United States acted to
mitigate transmission of and hospitalizations from the severe acute respiratory syndrome
coronavirus 2 (SARS-CoV-2), which causes coronavirus disease 2019 (COVID-19). These
actions were not coordinated at the national level, which raises the question of what might have
happened if they were. It also creates an opportunity to use spatial and temporal variation to
measure their effect with greater accuracy.

Methods ​We combine publicly available data sources on the timing of stay-at-home orders and
daily confirmed COVID-19 cases at the county level in the United States (​N​ = 132,048). We
then derive from the classic ​SIR​ model a two-way fixed-effects model and apply it to the data
with controls for unmeasured differences between counties and over time. This enables us to
estimate the effect of stay-at-home orders while accounting for local variation in factors like
health systems and demographics, and temporal variation in national mitigation actions, access to
tests, or exposure to media reports that could influence the course of the disease.

Findings ​ Mean county-level daily growth in COVID-19 infections peaked at 17.2% just before
stay-at-home orders were issued. Two way fixed-effects regression estimates suggest that orders
were associated with a 3.8 percentage point (95% CI 0.7 to 8.6) reduction in the growth rate after
one week and an 8.6 percentage point (3.0 to 14.1) reduction after two weeks. By day 22 the
reduction (18.2 percentage points, 12.3 to 24.0) had surpassed the growth at the peak, indicating
that growth had turned negative and the number of new daily infections was beginning to
decline. A hypothetical national stay-at-home order issued on March 13, 2020 when a national
emergency was declared might have reduced cumulative county infections by 62.3%, and might
have helped to reverse exponential growth in the disease by April 5.

Interpretation​ Although stay-at-home orders impose great costs to society, delayed responses
and piecemeal application of these orders generate similar costs without obtaining the full
potential benefits suggested by this analysis. The results here suggest that a coordinated
nationwide stay-at-home order may have reduced by hundreds of thousands the current number

1
NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

of infections and by thousands the total number of deaths from COVID-19. Future efforts in the
United States and elsewhere to control pandemics should coordinate stay-at-home orders at the
national level, especially for diseases for which local spread has already occurred and testing
availability is delayed. Since stay-at-home orders reduce infection growth rates, early
implementation when infection counts are still low would be most beneficial.

Funding​ None.

Introduction
Coronavirus disease 2019 (COVID-19) first appeared as a cluster of pneumonia cases in Wuhan,
China on December 31, 2019​1​ and was declared a global pandemic by the World Health
Organization (WHO) on March 11, 2020.​2​ As of April 12, 2020, the European Centers for
Disease Control reports that worldwide there have been 1,734,913 confirmed cases of
coronavirus disease 2019 (COVID-19), resulting in 108,192 deaths.​3

The United States recently became the country with both the highest number of cases (529,951)​3
and deaths (20,608) due to the disease. As a result, the U.S. government has been widely
criticized for inaction in the early stages of the pandemic.​2​ Although the first confirmed case of
COVID-19 was reported to the Centers for Disease Control on January 21, 2020 and documented
transmission commenced immediately​4​, a national state of emergency was not declared until
nearly two months later on March 13. At that time, no mandatory actions were ordered at the
national level other than international travel restrictions.​5

While the national government has the authority to act, the United States is a federal political
system where public health is normally the purview of the fifty states. Furthermore, each state
often delegates health authority to cities and/or counties, geographic political units nested within
states. As a result, responses to COVID-19 varied across states and counties and led to spatial
and temporal variation in implementation of mitigation procedures. This variation in policy
responses has likely contributed to significant variation in the incidence and growth of infections
across jurisdictions in the United States.​6

A variety of government policies have been proposed and used to mitigate the spread and
consequence of pandemic diseases like COVID-19, ranging from investments in medical testing,
contact tracing, and clinical management, to school closures, banning of mass gatherings,
quarantines, and population stay-at-home orders​7​. China’s extensive interventions appear to have
been successful at limiting the outbreak.​8,9​ These include quarantines both for those diagnosed
and those undiagnosed but who had been in Hubei province during the outbreak​10​, and

2
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

restrictions on travel to and from affected areas.​11​ In contrast, school closures across East Asia
were estimated to be much less effective.​12

With estimates that nearly half of transmissions occur from pre-symptomatic and asymptomatic
individuals, epidemiological simulations suggest that quarantines of symptomatic individuals
alone will be insufficient to halt the pandemic.​13​ This has led to widespread adoption of
population-wide policies to dramatically reduce social contact.

Here, we study the role of stay-at-home orders, perhaps the most common policy intervention in
the United States and Europe. Stay-at-home orders require citizens to shelter in their residence
with very few exceptions, and they have typically been implemented along with school closures,
bans on mass gatherings, and closure of non-essential businesses. These policies are associated
with a significant reduction in observed mobility,​14​ and initial evidence from New York City
suggests that they can be effective in reducing case growth in the United States.​15​ Yet, because
each locality in the U.S. has many factors that contribute to differential rates of transmission,
statistical efforts to control for potential confounds and to identify the precise effects of
stay-at-home orders are critical to understanding whether -- and to what degree -- such policies
are working.

Methods
Data
The time and date of county-level “stay-at-home” or “shelter-in-place” orders for each state and
locality were aggregated and reported on a web page maintained by the ​New York Times​ starting
on March 24, 2020.​16​ As new orders went into effect, this page was updated. We checked it once
daily to update the data through April 11, 2020. In some cases a statewide order was reported
with reference to earlier city-level or county-level orders in the state without specifying where
they occurred. In those cases, we searched local news outlets to find references to official city
and county orders in the state that preceded the statewide order. For each county in each state we
recorded the earliest time and date that a city, county, or statewide order came into effect. Figure
1a shows the distribution of order dates. As of April 11, 2020, 18 states (1,451 counties)
exhibited county-level variation in order dates, 27 states and the District of Columbia had
statewide orders with no local variation (1,307 counties) and 5 states (386 counties) had no order
in place.

County-level data on cumulative COVID-19 confirmed cases were also aggregated daily by the
New York Times​.17​
​ We discarded all observations where cases were not assigned to a specific
county (these account for 1.3% of total cases). We retained observations where cumulative cases
declined from one day to the next due to official revisions to the counts (0.4% of cases). Figure
1b shows that the number of cases grew exponentially in each county over time from March 1,

3
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

2020 to April 11, 2020. It also suggests that efforts to “flatten the curve” initiated in mid-to-late
March may have helped to reduce the rate of exponential growth. Below we describe how we use
data on cumulative cases, which include both currently and previously infectious individuals, to
estimate the growth rate in the total number of currently-active infections.

Figure 1.​ (a) Distribution of stay-at-home orders at the county level by date in the United States. (b) Log
of cumulative confirmed COVID-19 active infections by county and date, gradient-colored by date of first
case (blue = early to light blue = middle to orange = late).

Availability of tests for COVID-19 in the United States has not been uniform over the date range
of the study.​18​ To mitigate the effect of changes in rates of testing on our measure of confirmed
cases, we also collected data on the number of tests administered each day. This information is
not currently available for each county, but it is available for each state by date from the COVID
Tracking Project​19​ for about 80% of our observations. We merged this data with information
about stay-at-home orders and confirmed cases.

Estimation
The spread of disease in a population (​N​) is an exponential process, with each person among
those currently infected (​I​) capable of infecting one or more other individuals who are
susceptible (​S​). After a period of time, infected individuals recover from a disease (​R​) and are no
longer infectious. The classic ​SIR​ model suggests that for a county ​c ​on date ​t ​the daily rate of
new infection is

I ct − I ct−1 = β I ct−1 (S ct /N ct ) − I ct−1 (1)

4
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

where 𝛽 is the infectiousness of the disease (how many other people does an infected person
infect per day) and 𝛾 is the rate of recovery (what fraction of infected individuals cease to be
infectious).

In our data, we observe cumulative infections (​y​ct​) but these include both infected and recovered
individuals ​y​ct​ = I​ct​ + R​ct .​ ​ We do not observe the number of recovered individuals, but we
assume it approximately equals the number of infected individuals ​d​ days prior to the current
period (​R​ct​ = ​y​ct-d​ ​), where ​d​ represents the number of days individuals remain infected. Because a
specific value for ​d​ is unknown for COVID-19, we estimate models with different assumed
values. We also assume that the portion of the population that is susceptible S ct /N ct = 1 , since
the number of observed cumulative cases is typically 1% or less of the population in each county
for this study, suggesting the rest remain susceptible (we elaborate on the limitations of this
assumption in the discussion).

Under these assumptions, we can rewrite the above equation as

[(y ct − y ct−d ) − (y ct−1 − y ct−d−1 )]/(y ct−1 − y ct−d−1 ) = β − . (2)

Notice that the left hand side is simply an expression for rate of growth in cumulative cases (
%Δy ) that adjusts for recovered individuals or, put differently, rate of growth in active
infections. To ensure that this value was not undefined, we added 1 to the denominator for all
observations.

If we assume that the left hand side of equation (2) is measured with error uct , and rates of
increase in active infections ( β − ) are a linear function of fixed factors within each county (
αc ), factors that apply to all counties but vary over time ( αt ), and county-specific stay-at-home
orders ( xct ), this equation can be rewritten as a two-way fixed-effects ordinary least squares
regression model​20

%Δy ct = αc + αt + ∑ τ≠−1 δ τ xctτ + uct . (3)

Because we do not know the temporal dynamics of stay-at-home orders, we measure the effect of
stay-at-home orders non-parametrically as they unfold in the days following the order. We sum (
∑ τ≠−1 ) over all possible observations of number of days τ prior to or after an order (excluding
the reference day τ =− 1 immediately prior to the order). For each τ , we estimate an effect size
δ τ using a set of indicator variables xctτ that equal 1 if the number of days since a stay-at-home
order in county ​c​ on date ​t ​equals τ , and otherwise equal 0. Negative values of τ allow us to

5
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

measure effects on days prior to the order in order to evaluate whether or not differences in case
growth rates might cause changes in the date an order is enacted, rather than the other way
around.

A strength of this model is that county-level fixed effects αc control for all time-invariant
features of each county that might drive rates of case growth in the epidemic.​21​ ​ For example,
each county has its own age profile, socioeconomic status, local health care system, base rate of
population health, and date on which a first case of COVID-19 was observed. Additionally, time
fixed effects αt control for factors that vary over time.​21​ For example, case rates could be
affected by changes in the availability of testing nationally, in social behaviors influenced by
daily events reported in the media, and national-level policies that vary from one day to the next.
Finally, we cluster standard errors ​uct​​ at the state level. This adjusts the estimated standard errors
for unobservable factors correlated between counties within the same state.

We identify the effects of stay-at-home orders δ τ using variation in the timing of implementation
by counties and municipalities. Our two-way fixed effects regression is equivalent to a
difference-in-difference model with variation in treatment timing. Models with variation in
treatment timing are known to exhibit bias if the treatment effect is heterogeneous over time.​22
We test for temporal treatment heterogeneity bias by performing a Bacon decomposition.​23
Results from this analysis indicate that this source of bias is moderate (weight of Later Treatment
vs. Earlier Control = 0.29) and that bias attenuates our results towards 0, in which case our
estimates are a lower bound for the effect of stay-at-home orders on case growth.

Our estimator captures the causal effect of stay-at-home orders on case growth if counties that
implement these orders on a specific date would have had similar changes in case growth to
counties that had not yet implemented these orders had the implementing counties not
implemented the order on that date. This is the standard parallel trends assumption of
difference-in-difference models.​24

Role of the funding source


There was no funding source for this study. The authors had full access to all the data in the
study and had final responsibility for the decision to submit for publication.

Results

Figure 2a shows how the mean county-level daily growth rate in COVID-19 infections has
changed over time. After peaking on March 25, 2020 at 18.0%, it declined quickly to 3.6% as of
April 11. Figure 2b restricts observations to those that had implemented a stay-at-home order and

6
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

shows that peak growth in these counties occurred three days before the order went into effect
(17.2%) and declined to 2.7% three weeks afterward.

Figure 2. ​Mean U.S. daily growth in total active COVID-19 confirmed infections ​(a)​ by date, and ​(b)​ by the
number of days before or after the stay-at-home order. Growth in active infections declines after
stay-at-home orders towards 0, where the number of new active infections is approximately equal to the
number of already-infected who recover. When growth goes below 0, the number of daily infections
begins to decline.

Growth rates begin to decline following the orders. However, a number of factors may confound
this association in the raw data. For example, stay-at-home orders may closely follow earlier
targeted mitigation measures at the national level (such as travel restrictions issued by the State
Department or recommendations by the CDC on mass gatherings). There may also exist spurious
correlation between local factors (such as susceptibility to the disease or the capacity of the
health system) and the timing of stay-at-home orders. To control for these factors, we apply the
fixed-effects model in equation (3) to the data.

Table 1 shows four versions of the model. The main parameter that creates some uncertainty is ​d,​
the number of days an infected person remains contagious. Recent research suggests this period
is about two weeks, so that is our assumption in Model 1. In Model 2 we show results if we set
the value of ​d​ to 7 and in Model 3 we set ​d​ to 21. There is also some concern that measures of
growth in cases of COVID-19 may be affected by the rate in growth in the availability of tests
for the disease, so in Model 4 we include that variable as a control. Parameter estimates for all
models are similar, so we will focus on Model 1 for the remainder of this analysis.

7
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

Model 1 Model 2 Model 3 Model 4


(​d = 14​) (​d​ = 7) (​d​ = 21) (​d​ = 14)
Estimate SE Estimate SE Estimate SE Estimate SE

Days since
order
0 -1.28 1.91 -0.30 2.16 -0.84 1.93 -1.32 1.93
1 -1.91 1.48 -1.33 1.46 -1.70 1.47 -2.04 1.50
2 -2.80 1.30 -2.39 1.38 -2.51 1.31 -2.94 1.32
3 -3.10 1.72 -2.88 1.65 -2.78 1.70 -3.27 1.84
4 -1.55 1.71 -0.50 1.78 -1.24 1.77 -1.75 1.86
5 -3.99 1.42 -3.06 1.59 -3.86 1.45 -4.22 1.57
6 -5.72 1.54 -4.97 1.78 -5.14 1.49 -6.03 1.69
7 -3.79 1.59 -3.14 2.03 -3.88 1.59 -4.12 1.65
8 -4.74 1.47 -3.96 1.63 -4.70 1.49 -5.08 1.60
9 -3.73 2.37 -3.21 2.83 -3.90 2.27 -4.01 2.58
10 -6.79 1.98 -6.54 2.10 -6.54 1.98 -7.21 2.26
11 -5.64 2.40 -5.49 2.61 -5.66 2.26 -6.11 2.65
12 -5.95 2.09 -4.26 2.69 -5.78 2.02 -6.41 2.47
13 -6.53 2.52 -6.11 2.79 -5.91 2.30 -7.09 2.80
14 -8.55 2.81 -8.84 3.60 -8.58 2.65 -9.23 3.21
15 -8.67 3.14 -8.27 3.76 -8.07 3.02 -9.42 3.53
16 -10.34 3.48 -10.78 4.10 -9.37 3.25 -11.21 3.83
17 -10.38 3.19 -11.23 3.85 -10.03 3.05 -11.37 3.62
18 -8.94 4.29 -6.32 4.73 -8.77 4.09 -9.95 4.63
19 -12.83 5.56 -16.50 7.23 -12.70 5.02 -13.79 5.91
20 -11.34 5.20 -13.96 5.23 -11.64 5.03 -12.34 5.65
21 -16.18 3.02 -15.89 3.59 -15.30 2.95 -16.74 3.45
22 -18.16 3.00 -19.79 3.38 -19.29 2.91 -18.78 3.44
23 -25.53 5.20 -28.71 10.84 -24.64 4.79 -25.64 5.30
24 -25.39 3.16 -30.07 3.58 -25.46 3.08 -25.36 3.59
Daily growth
0.00 0.01
in tests (%)
N 110,817 110,725 110,838 87,943
Adjusted ​R​2 0.050 0.031 0.055 0.041

Table 1. ​Coefficient estimates from regressions of daily growth rate of COVID-19 infections on variables
indicating the number of days before and since a stay-at-home order went into effect. The variable ​d​ for
each model indicates the assumption made about the number of days cases remain infectious. All models
include fixed effects for county and date and standard errors clustered on state. Model 4 includes as a
control a daily state-level measure of the growth rate in the number of tests administered for COVID-19.
All models include coefficients for days prior to the order that suggest differences in the case growth rate
do not predict the timing of stay-in-place orders (Models 1-3 shown in Figure 3).

8
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

The numbers in Table 1 can be interpreted as percentage point changes in the rate of growth of
COVID-19 infections associated with the number of days since a stay-at-home order has gone
into effect. Negative numbers indicate a slowing of rate of growth though it is important to note
that a smaller rate of growth still means an increasing number of total cases. For example, by
Day 2, counties with stay-at-home orders achieve their first statistically meaningful reduction in
the case growth rate (2.8 percentage points, 95% CI 0.3 to 5.3). After a week the reduction in the
rate is 3.8 percentage points (0.7 to 6.9). At two weeks the reduction in the rate is 8.6 percentage
points (3.1 to 14.0). And by Day 22, the expected reduction in the infection growth rate (18.2
percentage points, CI 12.3 to 24.0) has surpassed the average magnitude of growth rate at its
peak (17.2%). When the growth rate turns negative, the number of new daily infections will start
to decline and the epidemic will eventually come to a halt.

Figure 3 shows these estimates along with the estimates for each day ​prior​ to the day a
stay-at-home order goes into effect. Each panel of the figure displays the results given different
assumptions about the number of days cases remain infectious. Unlike the raw data shown in
Figure 2b, the estimates here are adjusted for unobserved factors that vary over time and between
counties that can influence the course of the disease. Notice that the estimates in Figure 3 before
the order goes into effect stay very close to zero. This suggests that differences in case growth
are not influencing the timing of stay-at-home orders, helping to rule out the possibility that the
later associations we see are driven by reverse-causality or differential trends.

To better understand the scale of these estimates, consider that these growth rates are ​cumulative.​
Lower growth in infections today means fewer infections tomorrow, so the full effect on future
infections is the ​product​ of these improvements. The coefficients suggest that counties with
stay-at-home orders have 20.8% fewer infections by day 7 and 48.6% fewer by day 14.

It is a complex task to model what might have happened if the United States Federal government
had coordinated a nationwide stay-in-place order when it declared a national state of emergency
on March 13, 2020. We have already seen county-level results that suggest the rate of growth in
infections turns negative by day 22, suggesting a nationwide policy might have done the same by
April 5.

Moreover, consider this: the median date a stay-in-place order was issued for counties in the
United States was a full 17 days later on March 30. Our coefficient estimates suggest that acting
17 days earlier in each county would have reduced new infections by 62.3%. It is thus highly
likely that a nationwide order would have reduced the number of infections by hundreds of
thousands. And with case fatality rates currently ranging from 1% to 4% in the United States, it
is possible that such an order could have also prevented thousands of deaths.

9
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

Figure 3. ​Estimated total effect of stay-at-home orders on the daily growth rate of COVID-19 confirmed
infections in U.S. counties that implemented these orders, by the number of full days since the orders
were issued. Estimates are derived from a fixed-effects regression of the daily growth rate of cases on
days since initiation of the stay-at-home order as a categorical variable, where the reference category is
the day prior to an order (Equation 3). Each panel of the figure displays a different model from Table 1,
representing a different assumption about the number of days ​d​ cases remain active ​(a) ​d ​= 7 (Model 2);
(b)​ ​d​ = 14 (Model 1); ​(c)​ ​d​ = 21 (Model 3). The models control for all county level and date fixed effects
and for correlated observations with cluster-robust standard errors at the state level.

Discussion

The results here suggest that the United States federal government may have erred in not acting
to coordinate stay-at-home orders at an early stage in the outbreak of COVID-19. We find that
these stay-at-home orders appear to be effective at the county level in limiting spread, and
provide some hope that the physical distancing measures now widely implemented are working
to flatten the curve.

With that said, we note numerous limitations in our analysis. Stay-at-home policies are
ultimately assigned endogenously so we cannot say for certain that the associations we have
measured are the result of a causal effect. Our tests of reverse causality suggest that stay-at-home
orders influence case growth and not the other way around, but there is no way around the fact
that these are observational data from which causal estimates are notoriously difficult to obtain.

10
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

Our dependent variable, growth in infections, is based on incomplete data. It is well-known that
rates of testing in the United States were extremely low in the early part of the pandemic,​2​,​18​ so
measures of cumulative cases over time probably increased faster than the disease itself due to
previously undetected infections. We attempt to control for this issue with county and time fixed
effects and a measure of the growth in testing at the state level, but we are unlikely to have
entirely adjusted for daily local variation in access to tests.

Our model of the disease is derived from the classic ​SIR​ model to allow for an empirical strategy
that estimates causal effects of stay-at-home interventions. Using this model, however, requires
us to make strong assumptions. For example, we must assume the number of days that infected
individuals are contagious even though the scientific community is currently unsure about the
precise distribution of this parameter. We account for this uncertainty by ensuring our results are
robust to different assumed values, but it is still possible that the true value falls outside the range
we show here. We also assume that the proportion of the population susceptible to the disease is
constant, at 1, over space and time. If this assumption is strongly violated it might influence our
estimates since they are linearly related to a parameter that is multiplied by that value. At this
stage in the disease we believe high susceptibility is a reasonable assumption, but low rates of
testing coupled with pre-symptomatic and asymptomatic transmission suggest it is possible that
the proportion susceptible is lower than is currently indicated by the data.

Our independent variable, stay-at-home order status, measures a policy intervention that was
often implemented simultaneously or within days of several other local interventions, such as
bans on mass gatherings and closures of schools, non-essential businesses, and/or public areas.
Given the uncertainty about how many days infected individuals are contagious both before and
after the onset of symptoms, efforts to generate a sharp estimate of the effects of policies that
were implemented within days of each other are difficult. Moreover, our analysis suggests these
other local interventions may also have an effect on infection growth. On average, the peak of
infections happens three days ​prior​ to the stay-at-home order. In addition, we see significant
reductions in the growth rate of infections at just two days ​after​ the order. This is in spite of the
fact that case identification during the early part of our observations was based on tests that often
took a week to be resolved.​2​,​18

With our current empirical approach we cannot perfectly separate the effects of these other local
interventions from that of stay-at-home orders. This means that our estimates should properly be
interpreted as the effect of stay-at-home orders bundled with the effects of these other local
interventions. As such, our model compares a “do everything” approach to a counterfactual mix
of “do something” and “do nothing” approaches at the local level, which is the status quo that
prevailed in the United States until mid-March. An interesting question which we leave for
future work is which local interventions in the policy mix helped the most.

11
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

One final limitation is that we assume the effects of stay-at-home orders between localities are
independent, but it is likely that significant spillovers exist. Consider the effect of the epidemic
in New York City on neighboring counties in New Jersey, Connecticut, and as far away as
Rhode Island. Or the effect of Mardi Gras in Louisiana or Spring Break in Florida on a variety
of locales throughout the United States. To the extent these spillovers are positive -- as seems
reasonable to assume -- we have likely ​underestimated​ the effect of a hypothetical coordinated
effort at the national level.

Would a hypothetical reduction in infections from an early nationwide stay-at-home order have
limited fatalities? We do not know. Perhaps such an order would simply have delayed the timing
of the pandemic. We do know, however, that slowing the initial rate of growth in infections helps
hospital systems to better figure out how to provide supportive care for COVID-19, enables the
implementation of better testing and tracing procedures, and provides time for clinical trials to
produce results regarding immunization and treatment. Perhaps most importantly, spreading out
infections over more time can help prevent the number of total cases requiring hospitalization
from spiking above existing hospital capacity, which is relatively fixed in the short-run. Thus, it
seems reasonable to assume that death rates seen in early cases might be higher than if earlier
stay-at-home orders had moved these infections later in time. If this is the case, then even a
moderate delay in infections could produce a significant decrease in total fatalities from the
disease.

It is important to note that although we are currently observing decreases in the ​rate of growth​ of
daily COVID-19 infections in the United States, it remains positive so that total cases continue to
increase exponentially. Only when the rate of growth turns negative will we know whether or
not we slowed the disease in time to keep it from overrunning our health system capacity. There
is much still to be done, and we are hopeful that the work here will help our fellow scientists,
policymakers, and the public-at-large to plan for the next steps in managing this disease.

Contributors
All authors contributed to collection of data, design and execution of analysis, and drafting, review, revision, and
approval of the final manuscript.

Declaration of Interests
We declare no competing interests.

Acknowledgments
We thank Robert Bond, Chistopher Dawes, Micah Gell-Redman, Lauren Gilbert, Tim Johnson, Arman Khachiyan,
Sam Krumholz, Brad Leveck, Peter Loewen, Lucas de Abreu Maia, Robyn Migliorini, Niccolò Pescetelli, Daniel
Rubenson, Wayne Sandholtz, and Clara Suong for helpful comments.

12
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

References

1 Zhou P, Yang X-L, Wang X-G, ​et al.​ A pneumonia outbreak associated with a new coronavirus of probable bat
origin. ​Nature​ 2020; ​579​: 270–3.

2 WHO Timeline - COVID-19. World Health Organization.


https://www.who.int/news-room/detail/08-04-2020-who-timeline---covid-19​ (accessed April 12, 2020).

3 Coronavirus disease (COVID-2019) situation reports #83. World Health Organization.


https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports/​ (accessed April 12,
2020).

4 Ghinai I, McPherson TD, Hunter JC, ​et al.​ First known person-to-person transmission of severe acute
respiratory syndrome coronavirus 2 (SARS-CoV-2) in the USA. ​Lancet​ 2020; ​395​: 1137–44.

5 Chowell G, Mizumoto K. The COVID-19 pandemic in the USA: what might we expect? ​Lancet​ 2020; ​395​:
1093–4.

6 Team CC-19 R, CDC COVID-19 Response Team, Bialek S, ​et al.​ Geographic Differences in COVID-19
Cases, Deaths, and Incidence — United States, February 12–April 7, 2020. MMWR. Morbidity and Mortality
Weekly Report. 2020; ​69​. DOI:​10.15585/mmwr.mm6915e4​.

7 Ferguson NM, Cummings DAT, Fraser C, Cajka JC, Cooley PC, Burke DS. Strategies for mitigating an
influenza pandemic. Nature. 2006; ​442​: 448–52.

8 Kraemer MUG, Yang C-H, Gutierrez B, ​et al.​ The effect of human mobility and control measures on the
COVID-19 epidemic in China. ​Science​ 2020; published online March 25. DOI:​10.1126/science.abb4218​.

9 Tian H, Liu Y, Li Y, ​et al.​ An investigation of transmission control measures during the first 50 days of the
COVID-19 epidemic in China. ​Science​ 2020; published online March 31. DOI:​10.1126/science.abb6105​.

10 Maier BF, Brockmann D. Effective containment explains subexponential growth in recent confirmed
COVID-19 cases in China. Science. 2020; : eabb4557.

11 Chinazzi M, Davis JT, Ajelli M, ​et al.​ The effect of travel restrictions on the spread of the 2019 novel
coronavirus (COVID-19) outbreak. ​Science​ 2020; published online March 6. DOI:​10.1126/science.aba9757​.

12 Viner RM, Russell SJ, Croker H, ​et al.​ School closure and management practices during coronavirus outbreaks
including COVID-19: a rapid systematic review. ​The Lancet Child & Adolescent Health​ 2020; published online
April 6. DOI:​10.1016/S2352-4642(20)30095-X​.

13 Ferretti L, Wymant C, Kendall M, ​et al.​ Quantifying SARS-CoV-2 transmission suggests epidemic control
with digital contact tracing. Science. 2020; : eabb6936.

14 Engle, Samuel and Stromme, John and Zhou, Anson. Staying at Home: Mobility Effects of Covid-19. SSRN.
2020; published online April 2.

15 Harris JE. The Coronavirus Epidemic Curve is Already Flattening in New York City. 2020; published online
April. DOI:​10.3386/w26917​.

16 Mervosh S, Lu D, Swales V. See which states and cities have told residents to stay at home. ​NY Times ​ 2020.

17 covid-19-data. Github ​https://github.com/nytimes/covid-19-data​ (accessed April 12, 2020).

13
medRxiv preprint doi: https://doi.org/10.1101/2020.04.13.20063628; this version posted April 17, 2020. The copyright holder for this preprint
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
It is made available under a CC-BY 4.0 International license .

18 Dyer O. Covid-19: US testing ramps up as early response draws harsh criticism. ​BMJ​ 2020; ​368​: m1167.

19 Most Recent Data. The COVID Tracking Project. ​https://covidtracking.com/data​ (accessed April 12, 2020).

20 Reeves A, Gourtsoyannis Y, Basu S, McCoy D, McKee M, Stuckler D. Financing universal health


coverage—effects of alternative tax structures on public health systems: cross-national modelling in 89
low-income and middle-income countries. ​Lancet​ 2015; ​386​: 274–80.

21 Baud D, Qi X, Nielsen-Saines K, Musso D, Pomar L, Favre G. Real estimates of mortality following


COVID-19 infection. ​Lancet Infect Dis​ 2020; published online March 12.
DOI:​10.1016/S1473-3099(20)30195-X​.

22 Goodman-Bacon A. Difference-in-Differences with Variation in Treatment Timing. 2018; published online


Sept. DOI:​10.3386/w25018​.

23 T. Goldring. 2018. ddtiming: Stata module to perform a Bacon decomposition of difference-in-differences


estimation. 2019 ​https://tgoldring.com/projects/ddtiming​.

24 Angrist JD, Pischke J-S. Mostly harmless econometrics: An empiricist’s companion. Princeton University
Press (Princeton, NJ), 2008.

Research in Context

Evidence before this study


Past research has shown that stay-in-place orders can be effective for mitigating influenza
pandemics and recent research suggests stay-in-place orders have helped halt the spread of
COVID-19 in China.

Added value of this study


To our knowledge, this is the first nationwide study of the effectiveness of COVID-19 mitigation
efforts in the United States. The uncoordinated nature of the response to the disease has created
variation between localities and over time that we can exploit to more precisely estimate the
effect these policies have on growth in infections. And we can use these estimates to quantify
the effect that early coordinated action by the United States federal government might have had.
The results show that the suite of physical distancing interventions that were implemented with
stay-in-place orders probably greatly reduced the rate of growth in COVID-19 infections.

Implications of all the available evidence


Even as we submit this paper for publication, policymakers are debating whether to keep
stay-in-place orders intact, and this study contributes to the conversation with a scientific
estimate of their effect so far specifically for the COVID-19 epidemic in the United States. The
evidence here suggests that the effectiveness of stay-in-place orders shown in studies of
pandemic flu apply more broadly to other diseases as well.

14

You might also like