Which Rate Stats Correlate More With a Player’s Role Than His Skill?
Posted by Neil Paine on July 9, 2011
Here's a quick-n-dirty study I ran this morning... The idea is this:
Some stats seem to be more correlated with a player's role than his actual skill. Take a player out of the role, plug another similar player in, the new player produces just like the old one (and the old one can't "take the stats with him" to his new destination).
How can we quantify this, though? Well, let's identify players whose circumstances changed. I took every team since 1978 and assigned its players to 10 "roles" -- primary PG, backup PG, primary SG, etc. -- based on my detailed position file and where the players ranked on the team in terms of minutes played. I then isolated every player in that sample who:
- Played at least 500 minutes in back-to-back seasons
- Was between age 24 and 34 in back-to-back seasons (to filter out potential aging effects)
- Moved to a new "role"
- Was replacing a player who played >= 500 MP in the role and was between age 24 and 34 the previous season
This leaves us with 1,866 player-seasons to look at. For each of those, I need to know which predicts the player's performance in Year Y better -- his own stats from Year Y-1, or the Year Y-1 stats of the player whose role he took?
| Category | Player Y-1 | Role Y-1 |
|---|---|---|
| Usage% | 0.824 | 0.032 |
| PER | 0.712 | 0.137 |
| Stl% | 0.793 | 0.303 |
| Tov% | 0.726 | 0.257 |
| Blk% | 0.912 | 0.457 |
| Stop% | 0.849 | 0.415 |
| Stops/48 | 0.840 | 0.435 |
| OWS/48 | 0.556 | 0.173 |
| Floor% | 0.659 | 0.339 |
| ORtg | 0.519 | 0.208 |
| WS/48 | 0.557 | 0.264 |
| OReb% | 0.933 | 0.648 |
| DReb% | 0.918 | 0.674 |
| Tot Reb% | 0.950 | 0.718 |
| Ast% | 0.906 | 0.681 |
| DWS/48 | 0.562 | 0.486 |
| DRtg | 0.586 | 0.523 |
The second column is the correlation coefficient between the player's rate in Year Y and his rate in Year Y-1. The third column is the correlation between the player's rate in Year Y and the rate the team got from his role in Year Y-1. Categories are roughly in order from least to most role-dependent.
As you can see, component stats like Usage%, Steal%, Turnover%, and Block% are fairly stable year-to-year and are explained far more by the player's previous performance than the role he plays, suggesting that those metrics capture underlying skill in addition to the player's role. At the other end of the spectrum, we see that defensive box-score estimators like Defensive Rating/Def. Win Shares per 48 minutes are heavily influenced by the team and role, which makes sense given that there is a large team component in DRtg.
Also of note: Although it's very stable year-to-year, a great deal of Assist% is role-dependent, as is also the case with the rebound percentages. This seems to confirm Phil Birnbaum's research that much of rebound rate (particularly defensive rebound%) is more a product of team roles and positional designations than actual skill. And the various offensive efficiency metrics (ORtg, Floor%) fall somewhere in between, capturing more skill than Ast% or the rebound percentages, but less than Usage%, Steal%, Turnover%, and Block%.
July 10th, 2011 at 8:46 pm
I don't have anything to add. Good line of thinking.
July 11th, 2011 at 3:11 pm
Really like this topic. It brought to my mind Jimmer Fredette. One could argue that his role with BYU this year (to shoot any shot he wanted) contributed to the success he had. No doubt he's an awesome player but he had free reign at BYU. It makes you wonder if how many other players from the ACC, Big East, Big Ten, etc. could have averaged 28 pts a game if their coach let them shoot it whenever they wanted to from wherever on the court they wanted to.
July 11th, 2011 at 4:34 pm
Neil, I don't have your data, so I say this somewhat tentatively, but I don't find the conclusion you are reaching from the data to be the only or necessary one, and for that reason, I'm not sure I am convinced yet.
Take rebounding or assists, for example. Couldn't this data indicate that coaches/GMs are relatively good at assigning (or in this case replacing) roles based on player skill? If a team loses Deron Williams, for example, they don't replace him with Paul Milsap. Instead, Willams is replaced by Devin Harris, and although Devin is not identical to Deron, their skills -- and thus the role assigned to them -- are similar.
http://www.basketball-reference.com/play-index/pcm_finder.cgi?request=1&sum=0&p1=willide01&y1=2011&p2=harride01&y2=2011
As a Pistons fan, I can think of another example - Stuckey and Bynum. If Stuckey doesn't return to Detroit, he could be replaced in the starting lineup by Bynum. Given their historical production, you'd see a very high correlation for USG rate, AST%, BLK%, etc. To say that this has more to do with their role than skills, however, seems very strange to anyone who has watched the players play, specifically because their skill sets are very similar. And as a direct results of those skill sets, they play (mostly) the same position.
http://www.basketball-reference.com/play-index/pcm_finder.cgi?request=1&sum=0&p1=stuckro01&y1=2011&p2=bynumwi01&y2=2011
Obviously, I'm sort of cherry picking a pretty easy example here (and I can think of plenty more off the top of my head), but I'm curious how often this would be true of the player seasons you've identified.
July 11th, 2011 at 4:36 pm
Or put another way, if I'm reading you correctly, you're arguing that certain types of production are a function of role, not of player skill.
I'm posing the critical question: isn't role a function of player skill?
July 11th, 2011 at 6:07 pm
No, I think you're right, a player's position is definitely a function of which basketball skills he possesses.
I think this study just shows that the baseline skill level within a position varies depending on the statistical category. It suggests that we should be wary of overvaluing the ability to put up big assist or rebound percentages, because the typical player at that position would be able to approximate those percentages if put in the same situation.
However, something like usage, turnover avoidance, or the two boxscore defensive metrics (steals & blocks) are harder to replace -- it's not a given that the typical player at a position would be able to duplicate those rates if put in the same situation.
July 12th, 2011 at 4:17 pm
Neil, would your sample size allow you to run this separately for "big" vs. "small" players? For assists, for example, I think you mainly want to know the impact of a role change for players who leave or become point guards. Including the big men here will ensure a very high correlation in both columns (as we expect their ast% to remain low no matter what). Similarly, you might get more informative results on rebounds when looking only at forwards and centers.
July 14th, 2011 at 11:24 am
I think you need to look specifically at the outliers to get a good read on this. Most NBA players in similar roles are extremely similar. Most guards get a few rebounds a game, and it doesn't matter who they replace. But look for something interesting - Lebron left Cleveland, did the small forwards there have more assists that you would have thought because that team was set up for assists from that position? A historical look at Dennis Rodman would be great - before and after he was with a team, what did his position's rebounding look like? Did he find great fits and exploit them more than anyone else, or did he make his own role by being so good at rebounding?
July 16th, 2011 at 8:24 am
Theres not much to be gleaned statistically from lebron leaving.
1. New coach different scheme on both sides of the court.
2. projected starters only played 6 games together
When lebron left Cleveland so did his role. you would be more apt to compare the 2011 miami team to the 2010 Cleveland team since he pretty much filled the same role.
bottom line is there are too many variables and overall changes to use lebron example to refute Neil's conclusions
July 21st, 2011 at 6:57 pm
One factor that often goes overlooked when considering individual stats is the skills of the player's teammates. Some players put up good per-minute offensive stats thanks to having a superstar teammate drawing defensive attention, or a teammate who is a defensive stopper that allows them to channel their energy into the offensive end.
Other guys get lower rebound numbers than their true skill level might predict, because the play with teammates who are adept at tracking down the ball. I'm thinking particularly of Rick Adelman's Portland teams, when Buck Williams, Jerome Kersey, and Clyde Drexler were all excellent rebounders at their respective positions.