Yesterday I made some changes to the Simple Projection System (SPS). The basic structure remains the same (it is supposed to be simple, after all) with the following modifications:

• The weights for the previous three seasons have changed from 5-4-3 to 6-3-1.
• The regression to the mean component now uses 1000 minutes of league average performance rather than 5000 minutes.
• The age adjustment is now (28 - age) * 0.004 for players younger than 28 and (28 - age) * 0.002 for players older than 28.

Why the changes? To be honest, the weights, etc. I was using last season were based on hunches, and recently I had time to take a closer look at the system. The modifications above not only improve the overall accuracy of the SPS, but they also give better projections for players at the extremes. For example, using the old system Dwight Howard was projected to grab 12.2 rebounds per 36 minutes last season, but under the new system his projected value was 13.0 (his actual value last season was 13.9).

How well does this system work? To compare the projected statistics to the actual statistics I used the NBA's efficiency statistic:

(Yes, I understand that this formula is flawed, but for these purposes it will work fine.)

I looked at all players from 1982-83 through 2008-09 who played at least 500 minutes in the target season and had a projected stat line. First, here are some summary statistics:

            Act     Proj
N          6887     6887
Mean      21.93    22.76
Std Dev    5.12     4.51

A scatter plot of the actual values versus the projected values suggested a linear relationship between the two variables (as one would expect), so a correlation coefficient was computed:

Corr      0.839

Next I fit a linear regression model using actual efficiency as the response variable and projected efficiency as the explanatory variable (I omitted the intercept). Two items are of particular interest here: (1) the parameter estimate for projected efficiency (it should be close to 1) and (2) the residual standard error:

          Estimate Std. Error t value Pr(>|t|)    
projected 0.963146   0.001448   665.4   <2e-16 ***

Residual standard error: 2.788 on 6886 degrees of freedom

The parameter estimate for projected efficiency is relatively close to, but statistically different from, 1. The residual standard error gives us an idea of the size of a "typical" error, and considering that this system is very basic I think the error is more than acceptable. If we look at the root mean squared error as opposed to the residual standard error we get the following:

RMSE      2.916

All in all, not bad.

I also thought it would be interesting to take a look at the system's biggest hits and misses over the years. Here are the ten biggest over-projections:

+-------------------+------+------+------+------+-------+
| Player            | Year | Age  | Act  | Proj | Diff  |
+-------------------+------+------+------+------+-------+
| Otis Smith        | 1992 |   28 | 13.5 | 26.2 | -12.7 | 
| Pervis Ellison    | 1994 |   26 | 19.4 | 31.4 | -12.0 | 
| Ken Bannister     | 1990 |   29 | 14.2 | 25.7 | -11.5 | 
| Jason Hart        | 2006 |   27 | 11.6 | 23.0 | -11.4 | 
| Kenny Satterfield | 2003 |   21 |  8.2 | 19.1 | -10.9 | 
| Jay Vincent       | 1989 |   29 | 13.4 | 24.3 | -10.9 | 
| Vin Baker         | 1999 |   27 | 16.4 | 27.2 | -10.8 | 
| Antonio Burks     | 2006 |   25 |  8.8 | 19.6 | -10.8 | 
| Chris Crawford    | 1999 |   23 | 13.4 | 23.9 | -10.5 | 
| Sidney Lowe       | 1990 |   30 | 14.6 | 25.1 | -10.5 | 
+-------------------+------+------+------+------+-------+

Some of these cases involve productive players whose careers were derailed by injuries. For example, from 1988-89 to 1990-91, Otis Smith averaged 19.1 points per 36 minutes and shot 45.9% from the floor. However, in 1991-92 those averages plummeted to 12.7 and 36.5%, respectively, no doubt due in large part to the serious knee injury that forced him to prematurely end his NBA playing career at the end of the season.

On the other hand, here are the ten biggest under-projections:

+------------------+------+------+------+------+------+
| Player           | Year | Age  | Act  | Proj | Diff |
+------------------+------+------+------+------+------+
| Michael Adams    | 1991 |   28 | 34.6 | 24.2 | 10.4 | 
| Andrew Bynum     | 2008 |   20 | 36.4 | 26.4 | 10.0 | 
| Boris Diaw       | 2006 |   23 | 27.2 | 17.3 |  9.9 | 
| Nazr Mohammed    | 2001 |   23 | 28.3 | 19.1 |  9.2 | 
| Amare Stoudemire | 2005 |   22 | 35.6 | 26.6 |  9.0 | 
| Kevin Willis     | 1992 |   29 | 34.6 | 25.7 |  8.9 | 
| Chris Andersen   | 2009 |   30 | 29.2 | 20.5 |  8.7 | 
| John Lucas       | 1984 |   30 | 30.1 | 21.8 |  8.3 | 
| Richard Anderson | 1984 |   23 | 28.8 | 20.8 |  8.0 | 
| Jose Calderon    | 2007 |   25 | 26.1 | 18.1 |  8.0 | 
+------------------+------+------+------+------+------+

There are many interesting cases here: very young players breaking out (e.g., Andrew Bynum), players returning from long suspensions (e.g., Chris Anderson), and veteran players ending up in favorable systems (e.g., Michael Adams).

Finally, there were 106 cases where a player's actual efficiency and projected efficiency were the same (both rounded to one decimal place). Here are those cases for the last five seasons:

+---------------------+------+------+------+------+------+
| Player              | Year | Age  | Act  | Proj | Diff |
+---------------------+------+------+------+------+------+
| Steve Blake         | 2008 |   27 | 16.9 | 16.9 |  0.0 | 
| Kwame Brown         | 2009 |   26 | 20.4 | 20.4 |  0.0 | 
| Kobe Bryant         | 2009 |   30 | 32.2 | 32.2 |  0.0 | 
| Erick Dampier       | 2007 |   31 | 23.8 | 23.8 |  0.0 | 
| Ike Diogu           | 2007 |   23 | 24.9 | 24.9 |  0.0 | 
| Reggie Evans        | 2006 |   25 | 22.2 | 22.2 |  0.0 | 
| Matt Harpring       | 2008 |   31 | 21.4 | 21.4 |  0.0 | 
| Brendan Haywood     | 2007 |   27 | 22.5 | 22.5 |  0.0 | 
| Luther Head         | 2008 |   25 | 18.3 | 18.3 |  0.0 | 
| Dwight Howard       | 2006 |   20 | 27.8 | 27.8 |  0.0 | 
| Josh Howard         | 2008 |   27 | 25.0 | 25.0 |  0.0 | 
| Jason Kidd          | 2005 |   31 | 27.3 | 27.3 |  0.0 | 
| Brevin Knight       | 2006 |   30 | 24.0 | 24.0 |  0.0 | 
| Donyell Marshall    | 2007 |   33 | 22.6 | 22.6 |  0.0 | 
| Andre Miller        | 2005 |   28 | 23.8 | 23.8 |  0.0 | 
| Andre Miller        | 2006 |   29 | 23.5 | 23.5 |  0.0 | 
| Nazr Mohammed       | 2006 |   28 | 24.6 | 24.6 |  0.0 | 
| Zoran Planinic      | 2005 |   22 | 18.5 | 18.5 |  0.0 | 
| Vladimir Radmanovic | 2008 |   27 | 19.9 | 19.9 |  0.0 | 
| Corliss Williamson  | 2007 |   33 | 19.4 | 19.4 |  0.0 | 
+---------------------+------+------+------+------+------+

For the most part these are veteran players with a well-established level of performance. Note that Andre Miller appears on this list in back-to-back seasons.

For more details on the SPS please see this article, and please check out our projections for 2009-10.

This entry was posted on Wednesday, October 21st, 2009 at 8:22 am and is filed under Announcements, Projections. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.