SRS Standard Errors, the Probability of Being the Best Team, and a Layup
Posted by Neil Paine on January 20, 2011
I finally got around to calculating the standard errors for our team Simple Ratings today:
| Team | Estimate | Std. Error |
|---|---|---|
| SAS | 7.97 | 2.62 |
| MIA | 6.90 | 2.60 |
| BOS | 6.67 | 2.63 |
| LAL | 5.78 | 2.59 |
| CHI | 4.81 | 2.61 |
| ORL | 4.61 | 2.61 |
| DEN | 3.48 | 2.63 |
| DAL | 3.30 | 2.62 |
| NOH | 2.40 | 2.60 |
| OKC | 2.05 | 2.61 |
| ATL | 1.75 | 2.60 |
| UTA | 1.73 | 2.61 |
| HOU | 0.86 | 2.60 |
| POR | 0.52 | 2.60 |
| MEM | 0.49 | 2.61 |
| NYK | 0.09 | 2.62 |
| MIL | -0.57 | 2.65 |
| PHI | -0.79 | 2.63 |
| IND | -0.87 | 2.65 |
| LAC | -1.51 | 2.63 |
| PHO | -1.91 | 2.64 |
| GSW | -2.92 | 2.62 |
| CHA | -3.74 | 2.64 |
| DET | -3.94 | 2.61 |
| TOR | -4.23 | 2.62 |
| MIN | -5.33 | 2.60 |
| WAS | -5.82 | 2.64 |
| SAC | -6.12 | 2.64 |
| NJN | -6.22 | 2.61 |
| CLE | -10.88 | 2.62 |
Then I set up a little Monte Carlo sim to estimate what is the probability of each team being the NBA's best (aka the team with the greatest "true" SRS skill). After 10,000 simulations using the estimates and standard errors above, here were the results:
| Team | Best Team | p(Best) |
|---|---|---|
| SAS | 3532 | 35.3% |
| MIA | 2045 | 20.5% |
| BOS | 1773 | 17.7% |
| LAL | 984 | 9.8% |
| CHI | 576 | 5.8% |
| ORL | 476 | 4.8% |
| DEN | 211 | 2.1% |
| DAL | 152 | 1.5% |
| NOH | 69 | 0.7% |
| OKC | 56 | 0.6% |
| UTA | 42 | 0.4% |
| ATL | 33 | 0.3% |
| HOU | 17 | 0.2% |
| POR | 11 | 0.1% |
| MIL | 6 | 0.1% |
| NYK | 6 | 0.1% |
| MEM | 5 | 0.1% |
| IND | 2 | 0.0% |
| PHO | 2 | 0.0% |
| LAC | 1 | 0.0% |
| PHI | 1 | 0.0% |
| CHA | 0 | 0.0% |
| CLE | 0 | 0.0% |
| DET | 0 | 0.0% |
| GSW | 0 | 0.0% |
| MIN | 0 | 0.0% |
| NJN | 0 | 0.0% |
| SAC | 0 | 0.0% |
| TOR | 0 | 0.0% |
| WAS | 0 | 0.0% |
According to this, we can be about 94% sure that the best team is either San Antonio, Miami, Boston, the Lakers, Chicago, or Orlando.
One interesting idea for a playoff system would be to eliminate all teams we were 95% sure weren't the best team and set the odds of winning the playoff to mirror the uncertainty we had regarding who was the best -- i.e., rig it so San Antonio had a substantially larger chance of winning the tournament than Chicago, etc. The NBA already does this to a degree via seedings and home-court, but you could even go as far as giving teams automatic 1-0 leads in a series to get the probabilities right.
For a great article that goes further with that idea, check out this Sky Andrecheck piece on the MLB playoffs, and the philosophy of why playoffs are necessary at all:
January 25th, 2011 at 6:00 pm
This is really cool. Have you done this it all for previous seasons. I wonder what the highest level of certainty has been at the end of the the playoffs?
January 26th, 2011 at 10:36 am
I only did it for this year, but then I plugged in 2007 (playoffs included) just as a test. Here were the errors:
And here were the likelihoods of being the best:
January 26th, 2011 at 1:15 pm
Very cool. I bet 2006 would not be very decisive at all. 1996, probably pretty clear cut.
January 26th, 2011 at 1:15 pm
Also would be interesting to see pre and post playoffs for different years.
January 26th, 2011 at 1:19 pm
45% for Spurs. 6% chance for other Conference Finalists! And the best part is, that it really does make sense.
January 26th, 2011 at 2:23 pm
Here's 1996, post-playoffs:
I would imagine that's about as much certainty as you're going to get in a given season.
January 26th, 2011 at 3:36 pm
Interesting - would've guessed they'd crack 90%, but from the first post you'd made I was a little surprised at how big the standard errors were so I shouldn't be surprised here. Thanks for doing that. Very fun to see.
January 26th, 2011 at 4:20 pm
If you used this methodology for players (using adjusted plus minus), the very best would probably be lucky to hit 10%.