Layups: Ken Pomeroy on Single-Game Plus-Minus
Posted by Neil Paine on February 10, 2011
Ken Pomeroy (of the outstanding college hoops stat site Kenpom.com) ran an interesting simulation last month with regard to the randomness inherent in single-game plus-minus scores:
A treatise on plus/minus - the kenpom.com blog
According to Ken's simulation, a player with precisely average "true +/- skill" can show up with wildly variant observed +/- values over the course of a game, or even 20 games.
Just for fun, I re-ran this experiment for ten thousand games, tracking the observed +/- impact of the player through various checkpoints. Here were the results:
| #Sims | On | Off | MOV | per40 |
|---|---|---|---|---|
| 1 | 5.00 | -17.00 | -12.00 | 44.00 |
| 10 | 0.70 | -2.60 | -1.90 | 6.60 |
| 100 | 0.49 | -0.85 | -0.36 | 2.68 |
| 500 | -0.39 | -1.02 | -1.41 | 1.26 |
| 1000 | -0.16 | -0.46 | -0.61 | 0.60 |
| 5000 | -0.06 | 0.10 | 0.04 | -0.31 |
| 10000 | -0.13 | 0.12 | -0.01 | -0.51 |
Even after 10,000 games, a massive sample that would never be possible to achieve in real life, our perfectly average "player" appears to be a half-point per 40 min worse than average by raw on/off-court plus minus. As Ken says, "respect randomness"!
February 10th, 2011 at 10:38 am
What distribution does +/- per minute/per game/per year follow empirically?
February 10th, 2011 at 2:31 pm
I agree with the point on variation: single-game data is highly likely to experience variation over the course of any basketball season. Here's what Ken is missing, however: the +/- data represents substitution, not random players. While not being able to show true value as well, single-game plus-minus surely shows us (at least to some degree), how the team fared with specific lineups in play.
I'm not making an extrapolative statement here: I don't think that small game samples can tell us about a player in the long run. However, I think that single games speak for themselves, just as any other data point does.
February 11th, 2011 at 11:15 am
#1 - It's definitely normal at the NBA level, so I think it's safe to assume that's true for college as well.
#2 - Good point about the on/off stints not truly being random in real life. There is some information being conveyed even in a single-game +/- sample, unlike in the simulation. I think Ken's main point was just that the signal-to-noise ratio is really low for +/- at a game level, probably much lower than it is for the box score stats.