Numlock Awards: When can we get cocky about predicting Best Picture again?

Numlock Awards is your one-stop awards season newsletter. Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Walter.

For a while I’ve been banging the drum about how the significant changes the Academy that votes on the Oscars has undergone has seriously impacted the ability of Oscar forecasting.

The gist is pretty simple. The Academy added a ton of new members over the past several years compared to their previous rate of invitation.

This had the effect of drastically increasing the size of the Academy.

And, more significant for us, I argued that a lot of our Academy prediction know-how — whether anecdotal, traditional, algorithmic or otherwise — should probably be completely overhauled until we know what the hell is going on with the preference of the people who now account for a majority of the Academy.

The analogy being basically if I combined the state of Pennsylvania and the state of Ohio into Pennsylhio or Ohsylvania, we’d have a bit of trouble predicting Senate races there for a few cycles.

I wanted to basically explore the exit strategy of this holding pattern. Namely, when do we get to feel comfortable in our understanding of what the Academy preferences look like, and how do they align with the preferences of the precursor awards?

There are some complications to this — as the Academy changes, so too do all of the other organizations with award shows, and many toward the same purposes of better representing a broader spectrum of the international film scene. This is one reason that even though it’s been a challenging few years in the Oscar forecast niche, it hasn’t been a total fiasco and in general we know that one of the two Best Picture nominees that won the most important precursors is gonna win it. We’ve been pretty solid at predicting acting prizes I think in part because BAFTA is growing alongside the Academy.

But mainly, I wanted to just figure out a number for each of the past cycles. What percentage of the Academy has been in the group for over five cycles? This number is useful for me — precursor performance over the past five years is half a given award show’s weighting in my model — but also serves as a good rule of thumb for how long a cohort takes to really become enmeshed in the establishment of the Academy.

I also wanted to carry it forward a bit. In each of the past two years, just under 400 people were invited to join the Academy, and the average number of active voters increased by about 100 people. (The balance is due to deaths, retirements, and declined invitations.) I kept that expectation going for the next few years, as my gut is the Academy probably intends to end up at around 10,000 members in the long term.

The result was this:

Incidentally this chart’s title could also be, “hey Walter in a given year how awesome did you feel about having to predict the Oscars?”

It shows two things of consequence.

One, the worst of the uncertainty is likely over. The Academy’s expansion, which largely satisfied the goals AMPAS set out to meet at its inception, was effective and now we can expect some stability in membership. That means it’s going to get easier over the long term to project the preferences of the organization. We can expect our models to get better, our weighting to get more accurate, and things to improve as time goes on.

Second, we’re going to improve, but it’s not going to get back to the way it used to be, at least not for a long time. More of the Academy is going to be newbies than in the pre-2010s, if they continue inviting at the pace they are. This will change if they really crank it back down to be a replacement-only invitation style, but by now the Academy understands that’s what got them into this mess to begin with and I’d be surprised if the era of arbitrary caps at 150 invitations a year ever returns.

Basically, I’m here to report some good news. We’re gonna get better at this. It’s still going to be very competitive. It’s never going to be easy again, but I think that’s a great thing. If it was easy, it would be boring, and who wants to watch a boring Oscars anyway?

I’ve been working on some really fun stuff coming that will be coming down the pipeline for the rest of this Oscar season! Be sure to tell friends if you enjoy this newsletter. We’re a small operation and any word of mouth is very cool.