The Last, Greatest Post About Austin Jackson & BABIP: Part 4 – Variance

First of all, here are their career numbers:  bear in mind that due to talent and longevity all of these guys are at least in the running for future admission to the Hall of Fame (where thus far only Carew presently resides):

For reference, so far in 2011 the league as a whole is averaging a BB% of .0845, a K% of .1823, an ISO of .138 a batting average of .252, a BABIP of .297 and OBP of .320 and a slugging percentage of .390. Bear in mind that in spite of the drop in scoring following the end of the juiced era, the 1960s and 1970s (and to a lesser extent 1980s) saw less offense than today.

Now take a look at the players’ standard deviations and standard errors:

Which stats seem the least reliable?

If we compare standard errors for BABIP and Isolated Power, walk rates and strikeout rates – all of which are supposedly much more consistent and predictable attributes than BABIP – it is BABIP that looks like the one to bet on. If we start from the knowledge, after a long career, that a player actually does have (or does not have) BABIP skills we don’t see guys that rely on high BABIP numbers to be productive being any flakier or less consistent than guys that rely on a batting eye or biceps – if anying they may be more consistent and reliable. There is nothing wrong, inherently, with trying to build a team around balls in play skills. Nor is there anything wrong, inherently, with putting a guy in the leadoff slot who needs that high BABIP to set the table. The problem is in recognizing, without the benefit of hindsight, whether those skills exist at all.

Exceptional power hitters can have career ISO numbers at or close to double the league average, the same goes for walk rates (take Barry Bonds for example). An exceptional hitter may strike out only half as much as the league average. No batter, no matter how good, will have a career BABIP of .594 – as I mentioned before Cobb has the highest career BABIP at .378. For actually assessing whether the skill exists, we need something that looks not only at year-to-year variability of a metric but also relates the player’s career number to the league average.

So, it’s time for a new term: the ‘confidence interval’.  Most statistical analysis is based on making certain assumptions about probabilities and then plugging in estimates in place of true values.  Using, for example, Jeter’s career BABIP number and his year-on-year standard deviation as a measure of variability, we can estimate a range (assuming a ‘normal distribution’ for annual stats around that career average) in which we are fairly certain that Jeter’s ‘true BABIP’ will fall.   For Jeter specifically, we can be 95% certain that Jeter’s true BABIP falls somewhere in between .350 and .363.  .350 is clearly much greater than the league average of .297 so after 16 years in the big leagues we can be pretty darn sure that Jeter actually has BABIP skills (and loads of them).  The higher the standard deviation for a statistic is, especially relative to the size of the variable, the wider the confidence interval will be and the more likely that we will be unsure (in a statistical sense) whether the true value is greater than the league average at all.  The confidence interval also gets narrower the longer the player’s career has been, after only 5 seasons (if Jeter had exhibited the same career BABIP and standard deviation) we could only say that his true BABIP was between .368 and .345.  That would be much wider if we had only, say, two years by which to judge a player.

Of course, the closer a player’s career BABIP number is to league average, and players like Jeter are unusually far from the mean, the harder it is to say anything at all about their skill.  If a player with two years of service time had a .350 BABIP in his rookie year and then followed it with a .270 in year two – his career BABIP would be .310, which shows a ‘skill’ along the lines of a Magglio Ordonez, in a statistical sense we could only say that his ‘true’ BABIP was somewhere between .275 and .345 – which could either mean HoF-caliber BABIP skills or a relative lack of BABIP skills that could ultimately cost him a roster spot.  Most of the time, hitters and pitchers who have shown some BABIP skills in their first few years (like Porcello or Avila) fit into this category.  The variability in their numbers vastly outweighs the possible difference between them and the average pitcher or batter, by the time Porcello is 35 we may be able to see conclusive proof (in a statistical sense) that he has the skill to keep a BABIP 10 or 15 points below average – but by then it will be a little late for the organization to make smart roster decisions.

Statheads should bear in mind, however, that this is also true for other attributes:  a player who hits a lot of home runs one year and experience a ‘power outage’ the next is a common phenomenon, and we are equally unable (in a statistical sense) to tell which year’s production is closer to the player’s true talent.  It’s just there is more differentiation across players in ‘power’ than there is in ‘BABIP’.

Now:  Back to Austin Jackson, who is, afterall the reason this has been researched, written and posted:  Jackson does have a limited track record, but he does not fit into that ‘sometimes quite a bit over, sometimes a little under’ class of player for which we have difficulty identifying true abilities.  Last year Jackson was waaaaay over league average, this year he is still waaay over (with two fewer a’s) – even given the tiny sample size, we can be 95% confident that Jackson’s true BABIP lies between .356 and .405.  So please, don’t expect Jackson to simply ‘regress to the mean’.  Jackson isn’t an average player, where BABIP is concerned.  Nor is BABIP uniquely unreliable as a measure of skill.  The true question is this:  If Jackson’s true BABIP is at the lower end of that range (which would be my guess), does he have enough talent in other areas to be a great player?