A Substitute for War

Basketball philosophy

American Playoff Systems: Closure or Chaos?

with 6 comments

Created by Jason R Remy (Jayron32)

Image via Wikipedia

Last week we saw the two #1 seeds in NFL lose in their first (and thus only) playoff game of the season.  Just a few weeks ago I wrote a post talking about just how good one of them, the New England Patriots, was compared with history.  Am I shocked they lost?  Not really given what I know about the NFL playoffs – it’s not uncommon for favorites to lose in the playoffs.

This event though has made a few people start thinking about the system we have and its pros and cons.  Joe Posnanski over at SI writes a great thought provoking article on the matter, its implications to the BCS, and on what playoffs mean in general.  In one line he asks:  “Is a playoff really MORE FAIR? What does fair even mean?”

Fairness in Competition = Removing Bias while Minimizing Randomness

A few  months back I wrote an analysis on that very subject in response to the idea of expanding the baseball playoffs.  To summarize in response to Joe’s question:  What is fair?  Ideal fairness means that we get rid of the unevenness of the regular season schedule without adding too much randomness.  If you’ve got a variety of divisions or conferences that hardly play against each other, the idea that you can have a single champion without a playoff tournament of some sort is absurd – but of course playoffs in some sense always mean throwing out a larger sample size for a smaller one, which never entirely good.

In the end, the existence of the playoffs is inherently pragmatic and capitalistic so we cannot expect that the one and only goal of the playoffs is to crown the most deserving team.  At the same time, add too much randomness and people will lose interest – though granted from what I see, other people’s tolerance of randomness is much higher than mind, and I’ll speak to that a bit later.

Analyzing Luck in the Regular Season and in the Playoffs

I did an analysis of the 4 major American professional leagues (NFL, NBA, MLB, NHL) going back to 1995.  Why ’95?  That was the first year of wild cards in baseball, which was the last major change in the 4 sports.  I’m not going to claim this study couldn’t be improved, but I think you’ll find that the major trends are pretty obvious.

I asked the following questions:

-What’s a typical regular season winning percentage for the top 2 teams in the league?

-What’s the likelihood that one of the two best wins the title?

-What are the winning percentages of those teams in each round of the playoffs?

I’m interested primarily in identifying which sports have the most and least lucky champions.  I’ll admit that being the best in the regular season doesn’t necessarily mean you are the most deserving of a championship, but I don’t think there’s any doubt that there will be a huge correlation – and again, I think the trends are pretty obvious when you look at them.

2 notes:

-In situations where teams tied for the best or 2nd best record, I went based on the team that did best in the playoffs.  Not a perfect system, but we’re doing the quick and dirty here.

-In hockey, the regular season winning percentage is from ’05-06 onward because that’s the point where ties were eliminated, and it’s not as obvious how best to factor in ties in this situation.  Note also, that because of hockey’s unusual situation where they award team points based on ties and overtime losses, I didn’t use the same teams to judge winning percentage expectations and playoff performance.  To me the former is used to judge the sports tendency toward dominance over luck in the game in general, while the latter is used to judge how regular season success of particular teams translates into championship.  Do let me know if you have question.

Okay, here is a table showing regular season dominance by win percentage and post-season success based on regular season success:

So, generally speaking, a given game, football has the least luck involved, baseball has the most luck involved.  Come playoff time, where football is the only sport that plays a single elimination tournament, that adds to the luck involved, and gives the crown of “least lucky champion” to basketball.   However, the NFL champion in general is still far more predictable from regular season success than the MLB or NHL champions.

Now, generally speaking, the most pragmatic of factors involved in whether to have a single elimination tournament or a series is the money that can be made.  More games means more opportunities to earn revenue, but less games means more excitement.  Every sport has a different sweet spot, but I think it’s safe to say that the NFL is doing more than fine in this regard.  So the question is really:  Does the sport get away with it without reducing the Super Bowl champion to a total crapshoot?

Sure looks like it.  If you’d personally like to see even less luck involved, I hear ya, but when you’re not too far behind basketball, which gets criticized for being so-predictable-that-they’re boring, hard to complain too much.

On the other hand, baseball and hockey’s randomness, while others don’t seem to mind, bothers me, and it’s part of the reason I’m not as keen a follower of those sports.  To be perfectly honest, I don’t actually think others are fine with randomness so much as they are under the delusion that there’s more meaning to the unpredictability than there actually is.  People ascribe much to improving as the season goes along and the ability to be clutch.  When Barry Bonds has a bad series and his team gets eliminated, they don’t take that to mean that there’s something amiss with the playoff system, no, Bonds must have CHOKED!  Or maybe it’s karma, eh?  Human beings are pattern recognition machines, who find meaning even where not actually exists, and thus Bud Selig can actually expect that if and when he adds another round of playoffs, and makes the playoffs even less meaningful, there will still be a contingent making morality plays out of the whole thing.

No Reason to Panic in the NFL

Now, Posnanski also talked about the tendency of teams with byes in the playoffs losing after that bye.  I’ve got a graph here giving the winning percentage of Top 2 teams in each round of each sport’s playoff.  Again I’m going back to 1995.  Posnanski talks about how more recent years have seen more upsets after the bye week, but when you see the graph it becomes clear we’re dealing with precariously small sample size.  Also, from a perspective of what the ideal playoff system is, if we can’t conclude the system was flawed in the 90s, it’s real hard for me to say that it’s fatally flawed now.

Can you make anything out of the graph?  I know, it’s tough.  Suffice to say:  Precariously small sample size means there’s a lot of noise in the analysis, and football actually look like the most normal among the various sports.

So I’m pretty satisfied with the NFL’s system, even as I’m quite sure that if the Jets and Patriots played 100 more times, the Patriots win more than the Jets.

6 Responses

Subscribe to comments with RSS.

  1. Good work

    One thing I’d say is the greater the impact of luck in the playoffs, the closer the quality of the teams. For example the 92 W Giants beating the 97 W Phillies this year seemed like a huge upset. In reality 5 wins is a tiny gap for a 162 game season. It’s the equivalent of 2.5 Ws for the NBA. But for us a 60 W team vs a 56-57 team is seen as even. The more I look at it, the Giants winning the World Series doesn’t come off as badly for the sport as it felt like back then. The Giants gap to the top the league was only a 2-3 game equivalent in the NBA. But the reason the Giants were so close to the Phillies despite a seemingly big talent gap is that baseball just has a ton of luck in it, regular season and playoffs

    It’s a good subject though, there’s no question luck has a factor in who wins and doesn’t.


    January 19, 2011 at 4:10 pm

    • Well, it’s true that a 5 win gap isn’t that big in a 162 game season, so a related upset isn’t that big of an upset.

      My real focus though is that in a sport where contenders often can’t reliably win 6 games in 10 against an average team (as was the case with both the Phillies and Giants), any reduction in sample size down to 7 game series induces a large amount of randomness.

      Matt Johnson

      January 19, 2011 at 4:22 pm

  2. A couple interepretations on the bottom graph:
    1-the team with the better record in the NFL gets HF advantage for 100% of the “series” which is 1 game. For the other leagues, it is a multi-game series, which minimizes HF/HCA.

    2-HF/HCA is much more important in football and basketball. Both sports require verbal communication among teammates and both also have discrete offensive and defensive possessions. The crowd can disrupt the opposing offense more easily. Hockey requires communication, but possession and the ability to disrupt the away team is more ambiguous. Baseball is too task oriented to offer much (if any) HF advantage.

    3-Baseball and hockey appear to be more random because the postseason departs more from the basic nature of the regular season. Pitching rotations shorten and goalies don’t take days off.

    I know this is more detail than you probably wanted to dive into, but it would be interesting to see expected vs. real results using regular season SRS and incorporating HF/HCA. For example: if we knew that in basketball, a team in the regular season with an SRS advantage of 1 playing at home wins 80% of the time, while they win on the road 50% of the time, you can more distinctly look at upset odds by round. That’s a pretty big project though.


    January 26, 2011 at 10:24 am

  3. Nat I appreciate your thoughts. As you say, you’re making suggestions for drilling down into more detail, quite understandably. Before going further though, to me the big trends seem clear. Football and basketball are far more dominate-able from a season-wide perspective than baseball or hockey, so the real question was whether football’s single game elimination playoffs up the randomness so much so that the apparent randomness starts to approach that of baseball and hockey. And the answer is that it doesn’t look like the effects are really THAT huge.

    Getting into your points:

    1 – You’re mistaken. The team with the better record does not always get the home field. In fact it’s pretty common for the best wild card team to have a better record than the worst division winner. So I wouldn’t draw any conclusions based on this item, though you’re quite correct that a complete analysis would take this into account.

    2 – Importance of HFA/HCA. Check out this related link:


    While that says that basketball and football have bigger home advantages than baseball and hockey, it’s really basketball where the advantage is huge.

    I’ll have to look into this in more detail, but in terms of the basic conclusions: Basketball is already easily the most dominate-able of the 7-game series sports in the regular season, despite the fact that home field advantage there makes such domination more difficult. I don’t have much doubt that whatever the effect of HCA, the NBA playoffs are much less chaotic than MLB or NHL primarily because of that ability for the superior team to dominate.

    3-The post-season change in format for baseball and hockey is an excellent point that I thought about getting into in my article but it seemed it would just convolute matters. As I said, given the regular season randomness of these sports, evidence of post-season randomness is simply to be expected, so I don’t feel a great need to really answer every angle here. It’s the football side of things that I was most curious about.

    I do however find outright changes come playoff time completely fascinating. I remember watching the Arizona Diamondbacks win the World Series and thinking that this team was essentially unbeatable in the playoffs because of Schilling and Johnson. In terms of “playoff theory”, it’s a whole different topic for discussion what changes are a good thing, and what changes aren’t.

    Your specific suggestions for teasing this apart further are good, and I’d be interested to do it or see it done.

    Matt Johnson

    January 26, 2011 at 11:06 am

    • Yeah, you’re correct on the HF/HCA with the divison champs vs.wild card not always allowing the best team to host.

      I suppose my only complaint about the NFL playoffs is that the bye, which is designed to reward teams for regular season performance, actually has a tendency to hurt the bye team. Rhythm is so important and we often see teams start off slow after an extra week off in the playoffs and in the regular season. Case in point, my hometown Colts. People are quick to say they’ve choked a lot in the postseason, but when you look at SRS and factor in +3 for the home team, the results tell a different story. Over the last ten years, they’ve played 16 postseason games. SRS w/ a HF allowance has correctly predicted only 9 of those games. The Colts are 3-4 in the other 7, so they’re pretty much right at expectations.

      The intersting part is when those 7 upsets occurred. In the division round, when one of the two teams is coming off a game while the other sat around at home, an upset occurred 4 out of 6 games. In wildcard, conference championship and SB games, when both teams are equally rested, the upset rate is 3 out of 10.

      Ignoring SRS for a moment and looking at what I knew about the injury situations of both teams heading into the games, matchups, etc: All 3 of the non-division upsets weren’t really upsets (SB win vs. Bears, Conf title over the Pats, and maybe the 2008 WC loss to Chargers, who always give them fits). In games where the bye doesn’t enter into the equation, I could have predicted the outcome 9 or 10 times out of 10. Doing the same thing at the division level, I would have been 3 of 6 rather than the 2 of 6 SRS predicts.

      Just kind of going through my general memory of the div games of the past few years, the upsets tend to hit here more (not necessarily at the top 2 level, but in overall, top to bottom). I’d much rather see them just do away with the bye entirely and go with 4 wildcards per conference, even if it means a lot of .500 teams getting killed in the first round.


      January 27, 2011 at 11:07 pm

  4. […] previously analyzed the playoff systems of the 4 major professional sports leagues in the US, looking at fairness, which I’ve defined […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: