Numlock Awards: Why Best Picture has so many upsets

An exclusive interview with Rob Richie about the impact ranked-choice voting has on the Oscars.

Numlock Awards is your one-stop awards season newsletter, and it’s back! Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Walter.

Something really new and exciting for today! We’re in the part of the season when all eyes are on Best Picture. While most of the Oscars are decided by just a simple first-past-the-post vote, Best Picture is decided by ranked choice. I, like a lot of you, have been really fascinated by this category over the past several years, enjoying the perceived upsets and just attempting to make sense of a category that resists sense.

A key reason for this is that a few years ago, Best Picture switched to this new voting system which just muddies the predictive waters, while all the other categories use first-past-the-post or plurality voting.

I wanted to find an expert to bounce ideas around with about the precise impact this has been having on the Oscar winners. I reached out to the organization that’s been pushing for ranked-choice voting across the country since the ’90s to get into the nitty gritty of why Best Picture — which alone uses ranked-choice to pick the winners — has specifically been so volatile, while the other categories have been easier to forecast.

This week, I’ve got a great conversation with Rob Richie, the executive director of the organization FairVote. This conversation has been condensed and edited for clarity. It gets a little nerdy at times, but read to the end for a really fascinating look within the Academy’s thinking on their voting systems.

---

Walt Hickey: Can you just talk a little bit about, for folks who might be unfamiliar with ranked-choice voting, the reasons why people pursue it? What is it, and what are the advantages of it?

Rob Richie: On the most fundamental level, a ranked-choice ballot is offering the voter an opportunity to indicate backup preferences — ranked choices — rather than just a single choice. That doesn't matter if you only have two choices, because if you only have a single choice and there’re two options, it doesn't matter because one's going to get more votes than the other. But as soon as you introduce a third choice or more, if you're only limited to a single choice, you're leaving your opinion off the table for the other options. You can get an unrepresentative outcome, when you get into these sort of strategic voting things like, “Well, do I vote for the one I most like? But I think that guy's going to lose. I better vote for this other one, it's sort of a lesser of two evils, but better than the third.”

Ranked-choice voting is designed to both allow voters to be more expressive, in a sort of simple way, and more powerful, by giving them this backup to their first choice, with a second choice and a third choice if the first couple of choices can't help their candidate win. So, just giving your vote a backup gives you more power.

On the outcome side, you're trying to get outcomes that are more representative, so that more voters contribute to the outcome. If you're electing one person, rather than someone being able to win a four-candidate race with 29 percent or something, you've got to reach out to more people and, ultimately, win with more than half, and that's the goal of the system. You add up all the first choices, and see if you have a first-round winner with 50 percent or more. If not, the candidate that is in last place is the loser, first loser. They're out, their ballots go to their second choice ranked on those ballots. You rinse and repeat until a candidate surpasses the victory threshold of 50 percent.

What it means functionally, say in a New York City primary, is the winner is not only more reliably representative, but they also will have had incentives to campaign in such a way that they engage with and touch more voters. Often, particularly in primaries or a nonpartisan city race, the winner not only gets over 50 percent, but actually gets ranked first, second or third by 65 to 70 percent of voters, so that you're having that deeper connection between voter and candidate.

I think it's really interesting, particularly within the context of the Academy Awards, just because the goal of it is to make it so that at minimum, at least half of the Academy has a fingerprint on the winner in some way or another. Right?

Right, and that's based on, we don't really know, I don't think they ever released the ballot data. Right?

Nope.

There's one thing that's a complicating feature is some other awards — like the Emmys used to do this — they used to have a ranked-choice ballot, it was like a point system. In a point system, if you indicate a second or third choice, it could hurt your first choice. You're giving your first choice 10 points, and your second choice gets eight points or something. Then actually, by giving your second choice eight points, that can cause your first choice to lose. There's some people who from that experience think, “Oh, I should only rank my top choice,” or something like that.

That's not true of the Oscar system, where your backup choices only count if your first choice is out.

You have to get more than 50 percent in that instant runoff of head-to-head. You can't win by being, say, everyone's second choice. So, you do have to be a film that some people think is the best, and probably a pretty decent share. It is a system that both balances your need to have enough of the passion vote that you're in the mix of the top two or three, but of those ones, you're the one that's the most inclusive choice, the one that people are also willing to rank second and third, and so on.

I really love this part, and I want to explore that last sentence that you said. A lot of the time when watching the Academy Awards, I will feel very, very good that, if you give me three guesses, I'll get the winner for Best Picture like 100 percent of the time, but picking the one winner is always hard. You always kind of have a sense of, “It's going to be between these three.” Just because, again, you see these revealed preferences over the course of the precursor campaign. But what I'm curious about is just how these ranked-choice races tend to come down? Which is, as long as you're in the top three when it comes to preferences, after that, it just kind of shakes out into the overall majority.

It’s a little more unpredictable in a nonpartisan city race or a primary. So, say, if you’re in New York City, there's Scott Stringer, Andrew Yang, Eric Adams and Maya Wiley. There’s these top tier four or five candidates.

The way voters are, certainly in these nonpartisan races, is there's almost always a real coherence to what the individual is doing. If you ask an individual what they’re doing, they have an internally coherent rationale for how they’re ranking, but it’s not often grouped simplistically. It’s not like everyone that ranks a certain candidate first is going to rank the same candidate second. Which means, from a candidate perspective, if you think that candidate is going to go out, the one who has, say, 10 percent of first choices, there are reasons to compete for that vote because those voters aren't just all going to go to the same person, usually.

You translate that to the Academy, it's like, what is the pattern of when a certain movie is going to lose in the algorithm, what's the rationale for that individual voter? It will be different, right? It'll be like, "Well, I really like this kind of film, or I like these kinds of actors, or this kind of costume design,” or something like that.

There's been these critiques of the ranked-choice voting winners, as somehow being...I don't know, less daring or something. It doesn't feel like that's been true to me — it seems like there have been some pretty interesting ones. But also that there's been a lot of overlap with Best Director. Because obviously people are looking for different things sometimes. But is there a different pattern since the ranked-choice voting era of Best Director versus Best Picture?

No, you're really onto something, because Best Director used to be the gold standard in predicting Best Picture, and there's been a deviation recently. It's lost a little bit of its shine. The DGA has just been getting worse, losing its ability to correctly predict the Best Picture winner.

There's a world in which that's just because the DGA hasn't been as aggressive as the Academy has in expanding membership, but there's also a world in which, realistically, the way that we select the Best Director, both for the DGA and for Best Director at the Oscars, is different than how we select Best Picture. So, that's a really interesting point that you've highlighted.

It will be interesting to see if there are different kinds of movies that win in those — the Best Director movie versus the Best Picture movie. And does that say something about the system, or what the system rewards?

Yeah, to give you a little ammunition there, I think we can look at some of the bigger splits recently. So, you saw Damien Chazelle win for La La Land, but then Moonlight won Best Picture. You also saw when Green Book won, it beat out Roma, but Alfonso Cuarón did win for Best Director. You're seeing more and more split decisions, but I don't think that you're seeing a pattern in Best Picture when it breaks with Best Director. You’re not seeing only popcorn movies win, or only critical darlings — it’s quite mixed.

I know some point to The King's Speech, but The King's Speech also won Best Director. It's like, “Well, I guess there was support for that kind of movie among the Academy voters.”

The ranked-choice algorithm that's used for Best Picture, the instant runoff concept, is used quite widely. Like in New York City, all these other cities, a couple of states now in presidential races, all the Canadian political parties.

What it does — I think that why it's chosen — is it's a bit of a balancing act. In a plurality system, it's the only thing that matters: your first choice, that's it. And in a ranked-choice, in an instant runoff system, it's pretty darn important. You do want to be in those top three choices, almost certainly, to have any chance to win. It's a compromise between another kind of ranked-choice, where it doesn't matter at all what your first choice is. So, it's sort of somewhere in the middle.

Applied to Oscar world, given how a lot of people have thought about Oscar voters traditionally, they wouldn't want the majority winner to always have to win the Oscar.

The big question that we're leading around to is in a plurality system, the person with the highest floor of support can oftentimes succeed. We kind of saw that, not to dredge anything up, but in 2016 with Trump. He didn't really have a particularly high ceiling of support in those primaries, but he had a very reliable floor, and because of the plurality vote he was able to steamroll.

With ranked-choice, it seems like it's less about how high your floor is, and instead it's more about how high your ceiling is. Where, if you don't have north of a majority, or north of a rival contender, you're going to have a hard time building the coalition that you need.

Yeah, and I think that's a great way to say it, with one caveat being that you need enough of a floor to advance out of the first round. You do have to have some core support to be in the mix.

Then after that you better watch out if you have a low ceiling. That's a good way to think about why ranked-choice voting can be interesting, in candidate elections, because you do get these candidates sometimes. Arguably, Trump in the primaries would have been a good example where he did have a ceiling, and he would have had to change that to win in ranked choice.

We actually did a ranked-choice poll during the 2016 primaries, we did it with the College of William and Mary at the time of the Iowa caucuses. And Trump was the plurality first choice and the plurality last choice, among 11 candidates.

He was very polarizing and he did lose the instant runoff at that time to Ted Cruz, who was at his peak of popularity.

That was sort of a pattern we saw, in ranked-choice type polls, that he was generally the first-choice candidate, but he also had a ceiling. That kind of candidate doesn't do as well with a ranked-choice system unless they change their behavior and learn how to raise their ceiling.

Expanding that back to the Oscars, most of, if not all of the precursor awards, tend to be plurality vote systems, first-past-the-post systems. I'm thinking that they're very good for discerning the floor, but they're not as good at discerning the ceiling.

Yes, absolutely.

Roma, it was a Netflix movie, and who knows if the Academy voters at that time just had a ceiling for Netflix movies, or something like that, like, “We're not ready to give up on the cinema yet.” And so that alone might have been the difference for the plurality versus the ranked-choice outcome, which is sort of simplistically looking at it.

That's a movie that does have things that give it a ceiling. Like, obviously, great film, but again, its distribution was Netflix. It is in a different language, which may have presented difficulties for some voters. There is a world in which that's a movie that has the floor that gets it to the top tier of contenders, but it has the ceiling. I think you can also argue La La Land, which is a musical about Los Angeles in an increasingly global Academy, or 1917, which is a war movie, may not be everybody's cup of tea. These are movies that demonstrated having a very robust floor, but lacked the potential ceiling to close the deal, so to speak, under ranked-choice.

Yeah, yeah. Exactly. It's interesting to think about. Then you could get a sort of predictive model, and that's what you're doing. It sounds like you have a pretty sophisticated model for doing your predictions.

It's fine. I'm very interested in understanding the world, rather than simply trying to model it, but that's one reason that I was really keen to get you on the phone. But sorry, go on.

It's both looking at what's happened in the other award shows, that's going to be a huge predicting thing. And then it's this other one, where you almost have to begin understanding the electorate. That kind of Academy voter electorate, and what are their ceilings? What's going to define them? How is that changing with this infusion of new participants, which is pretty fascinating.

It's a fascinating question, I'm obsessed with it. I'm very interested in voting systems, and I think that it's so cool that the Academy actually makes a go of it and does one of the better ones. It's definitely a really good chance to understand what the benefits are of some of these alternative voting systems.

It's interesting that they chose the ranked-choice system, the single transferable vote, way back in the ’30s for nominations. It really was that insight that they wanted the nominees to be reflective of the Academy voters, so that people felt like they had a stake in the Oscars, which for me is a nice lesson in, “Wouldn't it be nice if we felt that way about Congress?” where you could say, “I have a stake. I have, actually, a voice.” Wouldn't that be nice?

They did that early. You think of some of the other award shows, they're having to have people come in and change the nominations. We have these sort of super people, whatever, this non-democratic process of overriding the nominations because they don't use a fairer approach that just crowdsources, collectively, a set of nominees. There's actually an interesting push to bring that proportional form of ranked-choice voting to the Grammys. It is a lively conversation, within that world.

Can you talk a little bit about when you talked to the Academy about what their goals were when they decided to roll this out?

This is back in the ’90s, and I was just getting going at FairVote, and found out about this use of the proportional form of ranked-choice voting, single transferable vote for nominations. This is in pre-internet land, so I wrote a letter to the Academy, and I got a really nice letter back from the then-Executive Director. It wasn't directly this — I picked up on what I felt his intent was — but basically he was appreciative that I was appreciative of their use of this fair nomination system. He had a line about, “this gives more people a stake in Oscar night.” Then he said something to the effect of, “Well, there's sort of a debate about what we want for winning Oscars. And we certainly want to,” — and this is many years ago, so I'm not quoting it exactly right, but — “we want to have some excitement on Oscar Night,” or something like that.

The way I read it, I was like, “Oh, right. You actually want upsets.” They actually want a plurality outcome, where not what everyone thinks should win, will win. You want to draw people to Oscar night, and then you want to have like, “Wow, how did that actor win?” So, that plurality voting was actually almost a virtue in that it can be surprising sometimes, which I thought was interesting.

Then when they went to 10 movies, they expanded the number of nominations. They didn't want to a real outlier to win with 14 percent of the vote or something.

I think the expanded field was what allowed them to use a fairer system for Best Picture. It just created enough conjecture and mulling. Kind of that same thing you were saying, “Well, I can get it down to three, but I just don't know for sure which of those three.” That's exactly what they want, right? They want you to not know.

So, even though ranked-choice voting is a reliable, fairer system, the fact that it's a complex enough field that you don't know for sure speaks to their interests of having surprises while they can still have a fairer outcome than some really bad movie winning, or something like that.

It's worked in that sense, in that even the conversation we're having is sort of grounded in, “Well, it's hard to know.” They actually want to have suspense, and not have like, “Well, everyone knows it's going to be Nomadland, so forget it.” Or something like that. And probably it will be Nomadland, but it might be Minari. And that's where people can have these conversations, that ranked-choice voting invites speculation. In a way, if I'm an Oscar organizer, I actually like that.

Can you just tell me, as we wrap it up, a little bit about your organization? Where can folks find your work and the work that you all do?

Yes. So, we're FairVote, FairVote.org, and have carried the lantern for these fairer voting systems since the ’90s. What's been really exciting for us is, in the last five years in particular, it's just taken off. I wouldn't say we're at a tipping point, but we're at a point where, say, New York City is using ranked-choice voting, and we expect maybe a double digit number of Utah cities to end up using ranked-choice voting this fall, and Virginia Republicans are using it to nominate their candidates for governor, and Democrats in four states did it for presidential primaries.

There's now a really fun, lively conversation in Congress, that is an increasing focus for us as an organization, on what we ultimately can do in Congress. Because we do feel that a breakdown of political norms and polarization is creating a real conversation about how to change political incentives through new, better voting systems.

We think within the next 10 years, we all might follow the Academy, in some of its choices.