School Rankings: Framing (Slightly) Differently

You might have thought it would be settled by now, but the school reopening debate is far from over. In a sense, we are in no different a place than we were in September. Schools are open in some places and not in others, and the correlations are odd. If anything we see more open schools in places with higher COVID-19 rates (see a tracker here). Some districts which planned to open (i.e. Boston) have pushed their openings back, but others have opened more despite increases in rates (i.e. places in Pennsylvania).

There is broader agreement — although by no means universal — that schools themselves are not locations with a lot of COVID-19 spread. But this doesn’t mean there is no possibility of spread, and there is little agreement on what is an acceptable level of risk.

This has become even more complicated as community rates have gone up. Many people — even some who have advocated for school openings — have suggested we may need to take a pause between Thanksgiving and Christmas. It may not be the fault of schools, but community rates may make school impossible. As I’ve been reading and thinking about this, I see three main threads of public discussion.

  1. There is much focus on the question of what is the community rate “threshold” for closing schools. Is it 3%, as DeBlasio has said in NY? Is it 20%, as some Iowa districts had been told back in August? Should this depend on testing positivity or case rates or a combination of the two? Is there any community threshold at which schools should be closed?

  2. There is a rallying cry among those who want schools open. “Schools before Bars” (or indoor dining). Frustration has bubbled up, especially in (relatively) low prevalence areas when restaurants and gyms are open but not schools.

  3. Where schools are open, especially in high prevalence areas, school leaders are frustrated at logistical challenges. Too many teachers and students out in quarantine — often based on cases acquired elsewhere. Even when there has been little spread within schools themselves, these logical details have meant some move to remote learning. Children’s Hospital of Philadelphia has walked back some of their in-person learning recommendations, based largely on issues associated with community spread.

I think all three of these points are swirling around the same basic issue, which requires us to think about how we rank schools. This framing — more below — helped me think through (2) and (3) above, and maybe make some progress on (1).

I will say, as a side not: independent of the issue of schooling between Thanksgiving and Christmas, these issues will not be gone in January. Widespread vaccines are, at best, a late spring option. But this holiday break and new semester may afford us a (last) opportunity to change our thinking.

Risk/Benefits Matrix, Redux

Many months ago, I talked about the idea of a risk-benefits matrix for deciding which activities you should engage in. (You can see the post here). The idea in that post was to highlight that every activity has a COVID-19 risk, but also an associated benefit. By placing them on a matrix with one axis for risk and one for benefit, we can think about which activities make sense to do. This applies to individual choices, but also to policy choices. Although that post was back in July, I was already talking about schools there and arguing that they had high value.

(As a side note, one of my amazing former colleagues at Booth, Eric Budish, has an academic paper out on this same idea, but much more complete. Check it out here. One of his big points, which I will echo here, is that there are some precautions we can take which allow us to do more of everything. SPECIFICALLY WHAT HE MEANS IS WEARING MASKS, THANK YOU FOR WEARING YOUR MASK EVERYONE.)

I want to suggest, now, that we might even further simplify this matrix into…a line. I want to narrow in here on policy choices. Let’s imagine we’re the Governor of a state, and facing decisions about what activities to allow. In most states and cities policy-makers have some control over parks, schools, dining and bars, gyms and store capacity. Policy-makers can make statements about expected gathering sizes although it’s hard to control what people do.

Each of these activities entail some risk, and some benefits. The key decision for the policy-maker is to combine these risks and benefits, and generate some kind of ranking. I have an example below, with a few policy decisions (note I’ve left off schools here).

You open the things furthest to the left first. State parks — low risk (outdoors safer), high benefit (people need something to do) — should be among the first things open. Outdoor dining before de-densified indoor dining before dense indoor dining. Smaller gatherings before larger gatherings, and so on.

We can argue (and will!) about where things fall on this line. Should indoor dining come before, or after, gyms? Indoor dining is probably less safe (harder to wear a mask) but the benefits are higher. We may need to argue about what we value in terms of keeping people employed and sustaining mental health versus illness risk.

But once we’ve agreed on the ranking, all that’s left is to draw a cutoff. To decide, based on some notion of community rates, what is open and what is not. I’ve illustrated one example below. A key point is that even with the same ranking, different policy-makers will draw their lines differently. Some states in the US have been more conservative than others around COVID. A difference in these cutoffs is one way to visualize, say, the difference between South Dakota and Maine. They may agree on rankings and disagree on where to draw the colored lines.

By ranking places in this way we can also recognize, even more strongly, the value of restrictions at the top. Part of the reason for putting some things “first” on this list is the recognition that they have high value for risk. Limiting the things at the top of the line — say, concerts or dense indoor dining — is important because it lets us do the things at the bottom. If we allow the high-risk, lower-benefit items at the top of the list, we run the risk of increasing COVID-19 rates so much we have to limit the items at the bottom.

What about Schools?

The data is showing schools themselves are low risk; not everyone agrees with this, I understand, but it is the premise for what I will say below. Beyond this, I think we mostly do agree that in person learning has large benefits. For the youngest learners, the risks are lowest and benefits highest. If you asked me, I’d put schools in the ranking as below (note I’ve separated out age group). (I might put high schools above outdoor dining; it depends a bit on the dining restrictions and the presence of sports in high schools).

In fact, though, this isn’t where schools are. I think we can organize more or less everything we see by moving schools in the ranking; see below for what I think our actual implied ranking is.

It’s easy to see how this helps us understand what we see in areas where schools have remained closed while other things are open. If you put schools all the way to the right in the ranking, of course it makes sense to open other things first.

What may be less obvious is that this also organizes the experience of areas which did open schools. In most cases it isn’t, really, that they prioritized schools. They just opened everything. Open schools are a bi-product of open everything. The fact is that neither decision prioritizes schools, at all.

This mistake, or failure, in ranking schools has led to both sets of frustrations I outlined at the top of this piece. Ranking schools “wrong” in lower prevalence areas led them not to open while other things did. Ranking them “wrong” in higher prevalence areas has led us to having too limited efforts to control community spread in order to help schools.

If we actually made schools a priority, we might better recognize that restrictions on other activities are necessary to keep schools open. As others have recognized, this is precisely what Europe has done as they have moved back toward lockdown. Grocery stores, hospitals, essential services and school remain open.

What about Resources?

Prioritizing schools would also mean providing more resources to open safely. The main way we can help schools is by controlling community spread. But making teachers and others feel safe at school is also key. This means requiring masks, yes, but also providing good PPE and, ideally, improving ventilation. It means better testing and screening — both symptomatic and asymptomatic — and money for substitutes. Financial woes have always been a part of the landscape of public schools in America, partly because this isn’t the first time we failed to prioritize them. This has made it worse.

So, What’s Your Point?

First: The school reopen debate is extremely contentious. The “More Reopen!” people (sometimes) argue that places which opened in spite of high rates are doing the right thing. The “Less Reopen!” people (sometime) argue that the places that haven’t opened in spite of low rates are doing the right thing.

What this framework makes clear, to me at least, is that probably very few, if any, places in the US did the “right thing”. Everyone made the same mistake, it just manifested differently in terms of the outcomes. If we can recognize that, I wonder if we can all organize to make the same point that schools should get higher priority. I think we’d be a more powerful voice yelling this together than wasting our breath yelling at each other.

Second: I will renew my call here for transparent stating of priorities by policy-makers. When I wrote about the risk/benefit matrix, I suggested Governors should put a version of this up at a press conference to talk about how they were weighing what to do. But a matrix is maybe too complicated. A line is even easier. Let’s have some transparent, public, accounting of how our priorities stack up.

Weigh in!

Keep the thoughts coming. I cannot write back to everyone but I do read all of your emails, I promise.

Write to Me!

Where to Find Me

COVID School Dashboard

My Website


Expecting Better