Last week, I set out to write a fawning post on Lamar Odom and how vastly under-appreciated he seems to be, particularly among GMs. So, like any good researcher, I set out to search for evidence that would support the conclusion I had already reached. I figured Hollinger's Player Efficiency Ratings, a frequent stop for bearers of unconventional wisdom, would be a good place to start. Upon finding that during the 06-07 season Lamar sported a surprisingly average PER of 16.1, I realized I would have to revise my angle. My first thought was that this was further evidence that despite his obvious virtues, maybe Lamar really is just a bad fit on the Lakers alongside Kobe. But after doing a little more digging and discovering that Lamar's PER was worse than the likes of Brent Barry, Earl Boykins, Bernard Robinson, etc., I came to the conclusion that the problem doesn't lie with Lamar or the Lakers' system, but with Hollinger's system and the thinking that goes behind new stats.
While spending a half-hour trying to decipher what caused his PER to drop 1.2 over the past two years, I started to realize one of my problems with stats of its ilk. Despite the limitations of traditional stats, we understand their flaws and can reasonably discuss them while keeping their limitations in mind. We know that the quality of a player's teammates will affect his assists, that a team's pace will skew its numbers, that a player's height should be considered when looking at rebounds (thanks FD), etc. Because we understand the limitations of these numbers, we can use them reasonably when discussing the impact of various players. With PER, on the other hand, nobody (or very few of us, I should say) understands it well enough to even know its problems. Maybe that's our problem, and the stat's been around long enough that the onus is on us to figure out its strengths and weaknesses, but, seriously, look at this monstrosity:
uPER = (1/MP)*
[ 3P
+ (2/3)*AST
+ (2 - factor*(tmAST/tmFG))*FG
+ (FT*0.5*(1 + (1 - (tmAST/tmFG)) + (2/3)*(tmAST/tmFG)))
- VOP*TO
- VOP*DRBP*(FGA - FG)
- VOP*0.44*(0.44 + (0.56*DRBP))*(FTA - FT)
+ VOP*(1 - DRBP)*(TRB - ORB)
+ VOP*DRBP*ORB
+ VOP*STL
+ VOP*DRBP*BLK
- PF*((lgFT/lgPF) - 0.44*(lgFTA/lgPF)*VOP) ]
Keep in mind that's just for the unadjusted PER. That equation still has to be adjusted for pace, and then normalized around 15. As an econ major who's had to take more than his fair share of stats classes, I'm still not capable of breaking down that equation in any meaningful way.
Now, this isn't intended to be a simple "PER sucks because it's complex" rant. My main problem with PER -- and a lot of the modern era stats -- is that by attempting to reveal truths by combining numbers, they often obscure most of the story. For example, I've never been a fan of relying on numbers that have been adjusted for minutes played. Rather than telling me that Ike Diogu averaged 22 points per 40 minutes during his time at Golden State this year, I'd much rather know that he averaged 7.2 points per game while averaging only 13.1 minutes. By combining those numbers, you lose a part of the story. Trying to extrapolate what someone does in limited time by assuming he could continue that production if just given a chance is terribly faulty logic. Unless you're David Lee, most players don't get buried on the depth chart without a reason. It's exactly that kind of reasoning that leads Hollinger to make ridiculous statements like Indiana got the best of the Harrington/Jackson trade.
That per-40-minute nonsense also makes it so Dajuan Wagner's PER was 17.2, Julius Hodge's was 16.0, and Pape Sow's was 16.4. In other words, without knowing a whole lot more about a player's statline, just glancing at their PER will often be completely useless. But isn't the whole point of the number to boil a player's statistical contributions down to one easy-to-reference number? By consolidating stats into more complex measures, you gain the ability to compare across players, but you lose explanatory power. Obviously there's a balance to be struck with this trade-off: for example, if given the choice between knowing a player's field goal percentage or his field goals made and field goals attempted, I'd take the former because it allows for clearer comparisons across players. PER simply takes that kind of thinking to the extreme. What is sacrificed by this metric is rarely worth what is gained.
As usual, I'd like to end this rant with a few disclaimers. I'm not saying I can't ever appreciate what Hollinger and APBRmetrics accomplish. For instance, in a recent post for TrueHoop, Kelly Dwyer used Penny Hardaway's 99-00 PER to explain that he had a quality year; in that case, writing ten stats would have been overkill. When trying to evaluate a player, 82games or Basketball Reference is almost always my first stop. Taking into account the extra value of free throws and 3-pointers when calculating a player's eFG% is a nifty trick. You very well might catch me referring to someone's PER at some point in the near future and will want to cry hypocrisy. But in the end, if you're going to take the time to analyze a player in depth, you might as well look at his entire statline to get the complete picture.
15 comments:
hey guys. check this out.
http://www.youtube.com/watch?v=ci3j363HWQM
enjoy.
Great post. Made even better by the Built to Spill reference in the title.
Carter, this is an interesting post--I'm all for questioning the merits of PER, which can definitely oversell some players. I once insisted to my friends that the Wizards needed to get local boy Michael Sweetney from the Knicks--his per-minute numbers were so great, he was bound to blow up.
(Well, several dozen pounds later, he did, sort of...)
Regardless, while Sweeney's weight problems made it obvious why he wasn't earning the minutes PER suggested he should, I'm sure there are less obvious, similarly strong reasons why some guys don't get time.
But in making your anti-PER case, I think you're overlooking how all statistics lie; just as too many NBA fans and followers aren't savvy enough to look beyond Ike Diogu's 7.2 ppg to determine his worth, relying only on PER is also foolish. Even Hollinger says so.
Otherwise, it's too easy to mislead the reader...which your examples more or less do.
Yes, Pape Sow, Julius Hodge (in his four games with the Nuggets), and Dajuan Wagner all had above-average PERs...but each played between one and seven games, and no more than a few minutes at a time. (And Bernard Robinson's PER for the year was a more logical 7.6, when you factor in the real minutes he played for the Bobcats, not just the 3.7 mpg in a few games with the Nets).
Each recorded too small a sample size for their PER to be relevant; if Hollinger was writing about these players, he'd immediately note that their PER came during garbage time of a few games. Instead, he'd point to someone like a Paul Millsap, who put up a sterling rookie PER playing a more representative 20 mpg, as a player who should be in line for more minutes.
Since PER is a newer stat, consider a long-trusted metric in another sport with a similar per-unit production figure: Baseball. Perhaps a young pitcher has a lifetime ERA of 0.00...because he pitched for one-third of an inning and got one guy to fly out. Even though a 0.00 ERA is the greatest ERA possible, no one will coronate that player as the greatest pitcher ever, save as a joke. That said, smart baseball fans know that ERA matters a ton more than wins, and if you were forced to choose one non-sabermetric stat to judge performance, that would be it.
PER is much the same way. OK, Brent Barry and Earl Boykins were slightly more productive in '06-'07--but that doesn't mean the Lakers should rush out and trade Odom for them. If anything, PER reiterates Odom's value because it's easier to find a bench guard with an above-average PER than a starting PF. But, did Barry have a bounce-back year? Yeah, and noting his change in PER between '05-'06 and '06-'07 is a lot easier than going through and listing changes across five or six different statistical categories.
As someone who occasionally blogs about basketball, though, I know I'm biased--I'd much prefer having one stat as shorthand. And until I find a better one (or you can convince me otherwise)...well, let's just say it's per-fectly acceptable.
Doctor Dribbles: Carter's job made him go in today, so I'll answer this one.
"Despite the limitations of traditional stats, we understand their flaws and can reasonably discuss them while keeping their limitations in mind. ... Because we understand the limitations of these numbers, we can use them reasonably when discussing the impact of various players."
The post admits that all statistics lie. The reason that's okay in certain cases is that we know their limitations and express them when we discuss the stats. I think what you did to explain the PERs of Sow, Hodge, et al. is actually a good example of what we're talking about. When Carter used those examples, he did so to show that just having PER available doesn't tell the whole story. The fact that you point to minutes to explain the stat is our point--you can only explain PER by the stats that go into it. Every time I check PER, I look at a player's common stat line. What, then, is the point of aggregating all of them if I'm just going to explain them with the common stats?
It would make sense to do so if I knew exactly what a PER unit signified. What does saying that someone is 1 PER better than another player mean? This, in the end, is my main problem with the stat. Okay, someone scored a 16 and another guy scored a 15. Is there any difference there? If so, what is the difference? Maybe one's a better rebounder, and another's a better shot blocker. I explained the stats using the common numbers--how is that any different than what I'd do otherwise?
Your point about using PER to compare the amount of quality players at a position is a good one, I think. I don't have much of a response to it. But if that's the case, why is Boykins better than Odom--shouldn't there be something there to account for the amount of quality players at a position? If we're using it to compare players, why not judge a position's worth, too?
As for your Barry improvement point, I'm not sure it matters that he went up however many PER points. I could glean the same information from just glancing at his stats--he did better. I'm interested in why it went up, which is what the common stats tell us.
Again, appreciate that you fielded my comment (and responded to my own blog post). Since this debate now seems to be concentrated at Ballhype, with four or five other blogs writing on it too, I'll just excerpt my response to Ty and Carter from We Rite Goode.
(Honestly, following the different threads of this conversation has to be harder than breaking down the PER formula.)
If your "main problem" is that the gradient of PER scores is confusing, well, I can't help you there. Hollinger has long publicized his relative ranking scores (which I linked to in my post). And there's always context, even with the stats we know and love. Going across sports, what does it "mean" if a baseball player has 90 RBIs or 105 RBIs, or if a QB throws for 3,500 or 4,000 yards? For these raw stats, a lot depends on the system and the players around you, which is what PER helps equalize.
And, to a significant extent, the degree of performance excellence is always subjective. (If the Wizards had Reggie Miller and his 18.4 career PER, I'd be thrilled; Hollinger, meanwhile, thinks he comes up short of real greatness). But what does it mean for a starting pitcher to have a career ERA of 2.75 or 3.15? Or a football QB to have a rating of 95 or 101? You can trust that a certain stat is "the 10th highest all-time" or the "lowest in history," but really, all stats just fuel a larger conversation about any player.
Ty, you point out that I used minutes (and even games) as a qualifier. Well, that's about the only qualifier that's necessary--if anything, it's often redundant. If I wanted to say that Gilbert Arenas had a higher PER than Allen Iverson last year, any decent NBA fan knows that both played most of the season; ditto with mid-level folks like Andres Nocioni or Raja Bell. When you start mucking around on the margins, that's when folks get confused. Do you honestly know, off the top of your head, who among Roger Mason, Kevin Ollie, Shawne Williams, and Othella Harrington played legitimate minutes?
I don't know if PER needs to adjust by position; if anything, we can discriminate by looking position-by-position at PER, as Hollinger's prospectus always used to do.
Why dismiss whether it's important if Barry's PER points went up? Maybe I'm just curious how Barry's career has ebbed and flowed over the years, and if I need one stat as a gauge, it's PER. If I was interested in why Barry's performance changed, of course I'd look deeper...because PER never claims to tell you the why in the first place.
The intriguingly named Hot Shit College Student says it best, on his blog: "What I still don't get, is why you're hellbent on dismissing PER because you've found that it's often misused. Why not have a beef with the people who are wrong?"
Wagner played one game last year. You can't possibly be serious.
Hardly the best anti-PER player to pick out.
Interesting post. There are other statistics you can try which arguably work better than PER. Dave Berri, co-author of the Wages of Wins, rated Odom at .210, with .100 being average, in 05-06, and .161 last year. See:
http://www.wagesofwins.com/Lakers0507.html
http://www.basketball-reference.com/fc/stats_search.cgi?req=1&sum=0&type=total&min=48&from_year=2007&to_year=2007&lg1=NBA&lg2=BAA&lg3=ABA&franch=&from_season=1&to_season=-1&from_draft=1947&to_draft=2006&draft_round=&draft_pick=&draft_franch=&from_age=0&to_age=99&min_height=0&max_height=99&active=&hof=&pos=&c1stat=MP&c1comp=gt&c1val=0&c2stat=&c2comp=gt&c2val=0&c3stat=&c3comp=gt&c3val=0&c4stat=&c4comp=gt&c4val=0&sortby=VAR&layout=full
Use VAR
Value Above Replacement
the formula is (PER - 9) / 15 * MP. Replacement-level PER is estimated to be about 9, so this measure computes the player's value above the league replacement level taking playing time into consideration.
that truncated link in different format
http://tinyurl.com/289j4k
maybe lamar is actually just average...
To everyone who's commented here: Thanks for your input -- even if we disagree with you, it's all much appreciated. I'm not going to write specific rebuttals here. I think I've said everything I can about this issue on Ballhype, so I will point you there if you haven't already checked out the great discussion on Tom Ziller's two articles.
Thanks again to everyone who's provided any input.
I think PER is a nightmare -- and I like Hollinger a lot.
Here's the thing. The guy, Doctor Dribbles, asks rhetorically "Going across sports, what does it 'mean' if a baseball player has 90 RBIs or 105 RBIs, or if a QB throws for 3,500 or 4,000 yards? ... But what does it mean for a starting pitcher to have a career ERA of 2.75 or 3.15?"
Ah-hah! See, I can answer these. In order, those stats mean
... the player's actions at-bat (hits, walks, sacrifices, etc., but not GIDP) caused runners to score 90 or 105 times ...
the QB's completed throws to his teammates resulted in 3,500 or 4,000 yards ...
and, against that pitcher, over the course of his career, the opponents scored an average of 2.75 or 3.15 runs (without aid of a fielding error) every nine innings.
What does it mean to say Odom has a 16.01 PER?
Dunno.
THAT's why PER is so difficult for a guy like me. And I'm not really a dilettante. I played in (NAIA) college, I'm a pretty good coach, and I've followed all levels of basketball since I was 5.
But I'm only a B+ basketball stats fan. (You posters on this site are all A-level. Your knowledge of the statistics and your ability to write about them is so impressive to me -- I feel like my dad when someone's trying to explain an iPod to him.) I'm a B- baseball stats fan, C+ football stats fan, which is why I don't know exactly what it means for a QB to have a 95 or 101 rating -- the QB rating system involves too many variables to be meaningful to me.
Something tells me that PER is like QB rating even for guys who are big stats guys. It's just too amorphous.
For me, though, PER is worse for one big reason. If you look at QB rating, it tends to match up pretty well with what you see on the field.
Brady/Manning/Montana/Elway = high rating.
Wade Wilson/Tim Couch/Rick Mirer = low rating.
PER doesn't have that verisimilitude. Yes, Garnett and LeBron and Duncan have good stats. But nobody can tell me that Josh Smith or Nene are better than Andre Miller or Tayshaun Prince. When I see something that dissonant with reality, which is also based on numbers that cannot be explained readily, my animal instinct is to reject it.
Nene is, beyond a shadow of a doubt, better than Andre MIller.
Some of you have made PER more complex than it is. It's an efficiency estimate that allows you to easily compare the relative worth players in a neutral environment. Any player above 15.0 is above-average, anybody above 20.0 is all-star caliber, anybody above 25.0 is truly great player.
PER has plenty of limitations. At least in my opinion it tends to over value players that take alot of shots because PER gives + credit for every shot taken. PER also tends to over value undesized power forwards that rebound well. But, it's biggest limitation is that's statistically messy. It's a good offensive rating with steals, blocks, and defensive rebounds thrown in complicating the meaning of PER. Hollinger acknowledges this has talked about eventually creating seperate offensive and defensive PERs.
Personally I prefer Dean Oliver's work, but that requires more knowledge and interpretation to understand what the player ratings mean. Hollinger's PER is more suited to ESPN's mass audience. If it's used in context as estimate to easily compare players in a neutral setting then there's nothing wrong with it.
As far as Lamar Odom goes. I think PER is simply reflecting reality. Lamar Odom is very talented, but that talent hasn't led to significant production. PER actually teats Odom better than most player evlauation methods becuase it's kinder on his low shooting %s and defensive liabilities. Since your not adverse to statistics check out his b-r.com page. Odom's shot selection is terrible. In only two seasons has he shot the 3 well enough to justify taking them in the first place. He's not a particuarly great FT shooter. His turnover rates aren't ecatly great. High enough that you wouldn't want him handling the ball alot, which limits the use of his ability as a passer. He finished with .508 PW%. Which is Dean Oliver's statistical way of saying that Odom played like an average player last year. I know Odom is talented, but he's rarely been able to convert that talent into production. And that's what statistics like PER measure production. And alot of that his own fault in terms of shot selection and turnovers. But attacking PER for Odom playing average basketball is misguided. Blame Odom for playing better basketball and not realizing his potential. Blame the coaches if you want for not putting him positions to succeed.
Post a Comment