Numlock Awards: Post-Oscars Mailbag & Wrap Up

Thanks for reading Numlock Awards this Oscar season! We hope you’ll join us again next year, when Bong Joon-ho’s translator triumphantly wins Best Director.

The Oscars are officially over, and we’re more than a year out from the 93rd Academy Awards. This was a super fun season — the Parasite sweep was unexpected, delightful & ground-breaking — and we hope you enjoyed the ride as much as we did. Now, for our final mailbag post.

Share

What did you think about the ceremony? What about the model?

Walt: Alright nobody actually asked this I just figured it was kind of implied.

Pretty happy with this year. As I’ve been saying all season, Oscar prediction is in a state of great upheaval. What the Academy has undergone over the past several years is equivalent to the entire population of Ohio moving to Illinois, and the best thing to do at this time is merely to monitor the pulse of the organization and be ready for when things calm down. We’re getting there, and my view is that if you’re going to be wrong, you better at least be learning stuff, and in that view this year was pretty good as far as figuring out the extent to which ranked-choice voting may be impacted by nominee count and whether the recent wins at the PGA are as critical as they appear to be

Oscars were pretty good, I’d be surprised if they go hostless a third time. I wrote a salty post about ratings on Sunday, and I feel like the only reason there was a second year without a host was overfitting on the mild success of last year’s ceremony. My proposal: get Adam Sandler to do it. He wants an Oscar quite badly, so he’ll put on a good show rather than the more cynical or contemptuous hosts we’ve seen before. He’s also internationally beloved — worth noting his movies do insanely well — and if there’s someone who would appeal to the kind of viewer who’s tuning out of the Oscars, it’s a guy like Sandler.

Michael: I concur — provided frequent Sandlerverse collaborator Jennifer Aniston cohosts.

With Parasite winning for both International Feature and Best Picture, what really differentiates the two categories? Do you think this line will be blurred in the future.

Do you think award categories will ever be consolidated? Like sound mixing and editing merging to be Best Sound. Or Male and Female actor merging to be Best Actor. - Rahat Bathija

Michael: Fantastic question!

First, I think International Feature needs a lot of reworking — the rules, as I went over in the last mailbag, are archaic and odd. International features with predominantly English dialogue cannot compete; a country’s film-selecting commission — often part of a foreign government’s ministry of culture — can only choose one film to submit to the Oscars on behalf of the entire country, leading to a lot of internal politicking. (Take China, for example, which has placed artists and filmmakers under house arrest because of their art. Some films will just never be submitted for reasons of censorship.)

That being said, I think Best International Feature still is a pretty valuable award since there is very little overlap between it and Best Picture. Best International Feature can drive people to films like Asghar Farhadi’s A Separation (one of the best movies you’ll ever see, and it’s streaming on Netflix!) and Denmark’s A Royal Affair, which introduced audiences to Alicia Vikander, who’d go on to win her own Oscar just a few years later. My suggestion would be to change the rules. First, let international features predominantly in English compete since there are dozens of countries where English is an official language. Second, remove the middle man (often the ministry of culture) and let directors and studios submit multiple films (so long as they meet some sort of calendar-year criteria). Third, give the Oscar to the director, not the country the film is “from.”

As to your other question about award consolidation, there is some movement to degender the acting categories. Asia Kate Dillon, a non-binary actor best known for the show Billions, has been pushing the Emmys to do away with categories separated by gender. The Grammys and MTV’s film, television, and music video awards have all ditched gender distinctions. But the Oscars, the BAFTAs, the SAG Awards, the Tonys, and the Emmys still split awards between female and male. It’s not a change that will happen over night — the Oscars are notoriously slow to change — but the debate is just getting started.

Walt: Piggybacking on this: the post I didn’t get the chance to do this cycle but will roll out early next cycle is a deeper dive into the new membership, specifically that quickly growing Members At Large cohort. The key finding for the purposes of this question is that the fastest-growing segment of Members At Large have been people credited as stunt performers and coordinators. Word has it they’re nearing or at the 100-member threshold at which they’re under consideration to become a branch in their own right, for which there’s precedent: the casting directors were in a similar position a few years back and now they have their own branch. Now granted, there isn’t yet an Oscar for casting, so the presence of a stunt branch — which isn’t a certainty, but I’d bet money on happening soon — does not immediately produce an Academy Award for Best Stunt Choreography. But Brad Pitt’s shout-out of stunt performers in his acceptance speech did feel something like momentum, so something may be on the horizon. 

As this is a mailbag, and must have a question, I suppose my question is this: what should I do now that award season is over? - Jayden Ross Daniels

Walt: Award season? Over? Don’t be ridiculous, we’re well into award season 2021 already

Anyway, things to look out for: the Academy announcing their new class in late spring or early summer. I am fascinated to find out how many people they invite. We’re hitting the point where I’d anticipate a slowdown in invitations, if only because it’s genuinely unsustainable at the current clip of invitations. I’m thrilled to find out how many people get the nod. 

I angle to avoid editorializing but basically, the way I see the Academy’s issues is not an issue inherently with diversity but one stemming from the longstanding arbitrary caps on the count of membership coupled with the lifetime membership. The lack of diversity was a symptom of that broader issue: if you’re only replacing members who retire or die, you’re only letting in a fraction of the qualified people in, and that fraction will probably look like the people already in the Academy. 

The average age of the Academy will rise, and the relevance will decline as its voter base resembled the absolute best in cinema of the ’70s and ’80s rather than the modern era. Had the Academy been willing to simply invite anyone and everyone who deserved it, rather than capping the number of invitation arbitrarily at “number of decedents,” they could  have avoided the membership problems they only ventured to repair in the past decades. 

It’s possible it’s too late to repair the damage, but the best-case scenario is they slow down the invitations, and they stabilize at a level above replacement. If they stabilize at a level at replacement, we’re going to have this exact same problem in 2040, and that’s not a sustainable path. Also, they’d potentially determine more specific eligibility requirements than the touch-and-go requirements they have now. I’d personally advocate for active membership requirements — meaning only people currently working in the movie industry get to vote. (The Academy tried to implement a form of this rule, but backed off.) But that’s perilous and arguably amplifies the ageism inherent in the industry, though I can see some ways around it.

Anyway, AMPAS, don’t hesitate to shoot me an email and I’d be delighted to help out. 

Thanks so much for reading! This is always really, really fun for us and I appreciate you all reading. We’ll see you again in the fall.

If you don’t subscribe to Numlock, you should subscribe to Numlock, it’s my daily newsletter about numbers in the news. 

Numlock Awards: Oscar Ratings

Numlock Awards is your one-stop awards season newsletter, and it’s back! Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Walter.

Tomorrow, we’re going to do the final mailbag of the season and wrap it all up! I’m thrilled with how things went down this year — Best Picture chaos is the only thing we know for certain, at least until the Academy membership stabilizes and we get repeatable data — and I’m delighted at how well the stats have borne out in the acting prizes over the transitional period.

But I wanted to get in one last post that I feel like I’m always too burned out to write but finally got around to this year: Oscar ratings!

Every year, the Oscar ratings are bad. With the occasional, slight outlier, the Oscars have been on a downward march for decades at this point, and this point is loudly emphasized by ABC — which pays to broadcast the ceremony — as a justification to stick their grubby fingers into a perfectly serviceable event.

Maybe they want to cram in a “best popular film” category, maybe they want to exile the smaller prizes like cinematography to a kiddie table ceremony, maybe they want to make Dwayne Johnson hop the stage in a flaming motorcycle while Tom Bergeron of Dancing With The Stars emcees. The mind of network executives is inscrutable.

This is hilarious, because if anything ABC is the thing dragging the Oscar ratings down, not the actual ceremony itself.

Here’s a chart:

That’s the ratings of the Oscar ceremony (blue line), plotted alongside the rating of the top-rated television program on television in that season (red line), and then the rating of the top-rated show on the network that aired the Oscars (yellow line) which has been ABC except for a few NBC airings in the early ’70s.

You’ll observe here that on average, the Oscars have been declining at less of a pace than the networks overall and ABC in particular.

(Download the data here)

Here’s another chart:

(There’s some missing data in the 1980s, hence the gap.) ABC is still pulling in a ton of money for ads, and I don’t have to adjust for inflation to assure you that ABC is doing fine year over year with these numbers.

So, what’s the problem?

The frequent scapegoat is the smaller movies that dominate the awards ceremony — films that aren’t seen by all that many people and as a result have less of a draw for audiences unsure about whether to tune in.

This is crap. I pulled the average domestic box office of the Best Picture nominees since 2010 and checked to see if there was literally any relationship between the Oscars ratings and the box office. (Data from The Numbers)

When I ran the regression, the r-squared — which is a stat that tells us what percent of the variation in the ratings is explained by the variation in the box office — was less than 1 percent. So no, nominating Jumanji is not going to do the job.

But would you like to know what did actually get an interesting r-squared?

When I did the regression looking at the top-rated show of the past 46 television seasons against the ratings of the Oscar ceremony that aired within it, that explained 20 percent of the variation.

And when I did another regression looking at the top-rated show of the past 46 seasons on ABC, and then compared it to the ratings of the Oscar ceremony that aired within it, that explained 27 percent of the variation.

If ABC wants bigger numbers for the Oscars, maybe instead of trying to mess with the Oscars it should try to get better ratings. If they’re genuinely worried that not enough people are seeing the Oscars, the right thing to do isn’t to mess with what works, but rather to give the show to CBS or something.

Remember, you can email us your questions for tomorrow’s mailbag at awards@numlock.news!

Numlock Awards: 2020 Oscar Predictions

Numlock Awards is your one-stop awards season newsletter, and it’s back! Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Walter.

We’re doing a mailbag next weekend, get those questions to awards@numlock.news. If your new, sign up to get this, it’s a low-key award season popup newsletter that will make you smarter about the Oscars.

Let’s get right to this, the Oscars are hours away. If you’re new, read all about exactly how this works here, how the Academy has changed and even feel free to grab the data for yourself here.

Down underneath the predictions is a longer post about what that experimental model I made last year says.

Share

Acting Awards

This year, the acting prizes seem about as much sealed deals as they could be, with all four front-runners locking down BAFTA, SAG, and the major critical prizes. It’s still entirely possible one of these has an upset — the Oscars are great and the last thing this project ever wants to do is imply this event is easy to predict — but as far as front-runners go, you couldn’t ask for four more solid ones.

Best Supporting Actress

Best Supporting Actor

Best Actress

Best Actor

Best Director

This early on looked to be a seriously competitive year, with Mendes and Tarantino splitting the historically predictive local wins, and Bong taking most of the local critics’ prizes when ignoring historical predictiveness. But Mendes has slowly swept:

Share Numlock Awards Supplement

Best Picture

The big one. This one’s a big lead for 1917, that much cannot be ignored.

Someone asked in the mailbag what, if anything, I’m looking for this year in terms of the predictive power of precursors, and honestly this couldn’t be a bigger distillation. The Producers Guild is flexing, lately, and I want to know if they’re the real deal or a flash in the pan.

I wrote a whole post on why the Producers Guild is of interest to us, why it might be better than other organizations at picking winners, and I think it’ll be the bellwether for how well the Academy is actually shifting and why.

Is it because the Academy didn’t just let in young people and international producers of film, but rather because they allowed in an abundance of a specific type of person, that is to say “suits” rather than performers or technicians, the type of person whose preferences are best captured by the PGA rather than any of the bona fide guilds? We won’t know for a while, but a 1917 win or loss would be more evidence one way or another.

If you want a counterweight to this, my friend James England produces an exceptional model that uses the same techniques as his college football ranking model to figure out Best Picture front-runners. It’s a really neat one. Like us, he publishes all his data: openness is really important, if you use a black box and pretending it’s quantitative you’re serving absolutely no one but yourself. I come back to it every year because it’s a completely different approach, so it captures different things, and it’s been right when we’re wrong. I think he gets at an under-analyzed way of estimating vote and TL;DR this year he likes Parasite.

And that about wraps it u—

Wait, didn’t you build a program last year to simulate the Academy and actually turn these weights into probabilities?

I did! Here’s what it found this year. Again, take it with a grain of salt (I still consider it fairly experimental) but I’m happy with it: last year, it gave Roma a 60 percent chance, and winner Green Book a 23 percent chance, which I felt pretty good about, a 1-in-4 chance that was higher than a lot of the other forecasts I’d seen.

There will always be variation on this stuff, and mainly I want to use this model to better understand how the Academy comes to its choice (see: Why Best Picture front-runners aren’t locks anymore and Oscar Front-runners and ranked choice voting) rather than pretend I know what’s going to happen.

Here’s how this works. If this is boring, skip to the bullets down below. But this is how the sausage gets made whenever you see predictive models, from weather to politics to sports, so if you’re interested read on.

We want to know how 8,400 people voting on Best Picture might go under the ranked-choice voting system. To do this, we can run a program that simulates 8,400 ballots, with the inputs being our understanding of how much weight the Academy as a whole would lend each candidate. If was doing this for a presidential election, I would have 49% of the weight for the Democrat, 49% for the Republican, 1% for the Libertarian and 1% for the Green. We could then simulate a person’s decision by drawing a random number and then figuring out where they’d land. Instead, we do that for 9 Best Picture nominees, and we do it several times per person (because they submit a ranking, not a single vote) and we do it 8,400 times to simulate the whole Academy.

First a simulation takes the weights that we know — the overall scores for the movies — and then it kind of scuffs them up a bit by adding or subtracting a randomly generated number from each. Remember, we don’t actually know the “true” preferences of the Academy, so this process of randomly adding or subtracting value over 10,000 simulations helps us see how we could be wrong. By directly inducing randomness, we’ll get a better idea of the overall chances movies have.

Next, it simulates a ballot for each of 8,400 voters.

The way we do this is basically like a raffle. Every movie gets the same number of tickets to start, and then we add more tickets in depending on how its scores came out after the “scuffing.” For each voter, we draw a ticket from the raffle, and that movie is put first on their ranking, then we draw another, and so on. This means that the ballots can look really random — voters are weird and not automatons — but the rankings are nevertheless informed by the overall preference of the Academy.

Then, I figure out what won in that simulation.

Then, we repeat that whole process for 9,999 more simulations. We do a new “scuffing” each time, 8,400 new “raffles” each time, the whole shebang. 84,000,000 simulated ballots across 10,000 simulations. It takes like an hour and it makes my 2013 MacBook Air distressingly warm.

After doing that 10,000 times, I count up how many times each movie won. If a movie won 500 simulations out of 10,000 total, I’d say it has a 5 percent chance. Anytime you’ve seen a forecast, it usually involved something like this happening: you feed a program some observational data, the program does a couple thousand simulations, each one randomly messed with in a different way. Then it says how many times it went each way. That’s all there is to it.

Also, I kept track of how often movies were ranked #1. Across the 84,000,000 simulated ballots, 1917 was top ranked on 12.2 percent of ballots, Parasite on 11.6 percent, Once Upon a Time In Hollywood on 11.3 percent, and so on, down to Ford v Ferrari and Little Women with 10.7 percent. You will observe that’s not a huge difference. But it compounds. Here’s the percentage of the 10,000 simulations each movie won:

  • 1917 — won 7,222 of 10,000 tries, implying 72% chance of victory

  • Parasite — 1,296 of 10,000 tries, implying 13% chance

  • Once Upon a Time In Hollywood — 552 of 10,000 tries, implying 6% chance

  • The Irishman — 301 of 10,000 tries, implying 3% chance

  • Jojo Rabbit — 200 of 10,000 tries, implying 2% chance

  • Joker — 127 of 10,000 tries, implying 1% chance

  • Marriage Story — 116 of 10,000 tries, implying 1% chance

  • Ford v Ferarri — 98 of 10,000 tries, implying 1% chance

  • Little Women — 88 of 10,000 tries, implying 1% chance

There you have it. 1917 appears to be a stronger contender than Roma was, making the possibility of an upset slightly more remote. But it is possible, so I will be watching.

Numlock Awards: Oscar Ballot

Numlock Awards is your one-stop awards season newsletter, and it’s back! Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Michael.

It’s Oscars weekend! We’ve discussed the highs and lows of awards season, and now we’re down to the wire. Below you’ll find my ballot for every Oscar category. I’ll throw in some commentary at the end, but just so it’s clean when you’re copying this over to your own ballot, here it is:

Best Picture: 1917

Best Director: Sam Mendes, 1917

Best Actress: Renée Zellweger, Judy

Best Actor: Joaquin Phoenix, Joker

Best Supporting Actress: Laura Dern, Marriage Story

Best Supporting Actor: Brad Pitt, Once Upon a Time in Hollywood

Best Original Screenplay: Parasite

Best Adapted Screenplay: Little Women

Best Animated Feature Film: Toy Story 4

Best International Feature Film: Parasite

Best Documentary Feature: Honeyland

Best Documentary Short Subject: Learning to Skateboard in a Warzone

Best Live Action Short Film: Nefta Football Club

Best Animated Short Film: Hair Love

Best Original Score: Joker

Best Original Song: “(I’m Gonna) Love Me Again” from Rocketman

Best Sound Editing: 1917

Best Sound Mixing: 1917

Best Production Design: Parasite

Best Cinematography: 1917

Best Makeup and Hairstyling: Bombshell

Best Costume Design: Once Upon a Time in Hollywood

Best Film Editing: Parasite

Best Visual Effects: The Lion King

As you can see, I’m predicting that 1917 is going to walk away the big winner of the night. For Best Picture, I think the people who don’t rank 1917 as their first choice will still have it relatively high up on their list — it doesn’t seem divisive or niche in the way that Three Billboards or Roma did, respectively. It feels more like The Shape of Water: no major detractors, broad appeal for both the storytelling and the technical craft, a relatively scandal-free awards run. Since Best Picture is ranked-choice voting, it helps to be amenable to a large swath of people, and 1917 racked up 10 nominations.

As for the other categories, I think there are a lot of races that are still up in the air. For Best Production Design, I decided to go for Parasite, which won the guild award for Best Contemporary Film… but, at the same time, Once Upon a Time in Hollywood won the guild award for Best Period Film. I’m betting that the iconic house used in Parasite and built exclusively for the film — will sway voters, but there are a lot of folks predicting otherwise.

All this is to say: as fun as it is to win your Oscar ballot, it’s even more fun to lose, since that means the awards were full of surprises. If we have to go hostless again for the ceremony, we at least deserve some unexpected winners. I’m personally rooting for a write-in win for J. Lo for Hustlers. If it can happen for Hal Mohr, it can happen for her.

Remember: we’re doing a post-Oscars mailbag. Email your questions to awards@numlock.news, and thanks for reading all season! Enjoy the Oscars.

Numlock Awards: Why Best Picture front-runners aren’t locks anymore

Numlock Awards is your one-stop awards season newsletter, and it’s back! Every week, join Walt Hickey and Michael Domanico as they break down the math behind the Oscars and the best narratives going into film’s biggest night. Today’s edition comes from Walter.

Gonna keep it quick today!

This past weekend, we got the final bits of data we need going into Oscar night. The BAFTA went to 1917, capping off an award-season tear where the film was honored by the Producers and Directors Guilds. These three prizes are among the most predictive — the lack of a nomination for an acting prize at the Screen Actors Guild will likely give the film’s backers pause — and we have a fairly indisputable frontrunner heading into Oscar night.

Indisputable, however, does not mean insurmountable.

Though they’re considerably less predictive than the three prizes 1917 locked down, it needs to be said that 1917 didn’t pick up anything at the Writers Guild, the American Cinema Editors, or the Critics’ Choice awards. That the actors didn’t even nominate it — instead awarding Parasite, now the strongest of the underdogs — means that 1917 may have the lead, but it’s not unanimous by any stretch.

As I highlighted last week, the larger the field the lower the advantage granted to the perceived front-runner. A reason for this is that lots of the other precursor prizes use a first-past-the-post voting system, not a ranked-choice voting system. In first-past-the-post systems, something can win a prize with a plurality of support, while in the ranked-choice system something can only win with a majority of support.

A film with 35 percent support detested by the other 65 percent can win in a first-past-the-post system but will lose in the ranked choice system.

The reason that frontrunners do worse with more nominees is that our understanding of the current state of the race is far fuzzier; even with a small field, we only have approximations because precursor awards don’t exactly release vote counts. And the more nominees in the mix, the higher the probability the outcome of precursor awards is being affected by vote splitting, which goes further to undermine our understanding of the race.

Basically, it’s not that more nominees mean less advantage for the movie that’s ahead. Better stated, more nominees mean that we have less certainty that the movie that looks like it’s ahead actually is.

Let’s do a thought experiment. Let’s say we had an Oscar race with just two nominees. A first-past-the-post election and a ranked-choice election would have the same outcome every single time. Let’s say I have three nominees: the chances get a little worse for the perceived front-runner, but a lead is a lead. Once you increment the number of nominees, the certainty the entity you think is the frontrunner has a lead shrinks. We don’t know that a lead by winning a bunch of first-past-the-post races remains a lead after being run through ranked-choice voting.

All this is to say that even though every single one of our races this year has a distinct front-runner, I’m much less certain about Best Picture, which does ranked choice, than say Best Director, which does first-past-the-post.

This weekend: final predictions, more stuff about The Academy, and the big night!

We’ll be doing the mailbag again for the weekend after the Oscars to cap off the season, so get those questions in to awards@numlock.news.

Loading more posts…