Will The Real Best Bullpen Please Stand Up?

At first glance, it looks obvious that the Oakland Athletics should have a big advantage over Detroit in the late innings due to the strength of their ‘pen.

Oct 6, 2012; Detroit, MI, USA; Detroit Tigers relief pitcher Joaquin Benoit throws to first base in the eighth inning during game one of the 2012 ALDS against the Oakland Athletics at Comerica Park. The Tigers won 3-1. Mandatory Credit: Rick Osentoski-US PRESSWIRE

I love that photo. Looks like Benoit is just serving one up. It’s not even a pitch he’s throwing, but still… looks just like the kind of BP toss that’s asking to get sent over the fence. Obviously Benoit’s pitches are better than that, usually a lot better than that. But he has had some rough times this season.

The Tigers bullpen is 10th in the American League in bullpen ERA (3.79) and 10th in the American League in (the theoretically non-luck-influenced) xFIP compiling a rather pedestrian 25-23 record over the course of the season. Pitching is a strength for Detroit, but it is pitching of the starting variety. The bullpen IS disproportionately important in the postseason due simply to the way games are managed and this looks like a cause for concern.

The A’s bullpen is 2nd in the American League in bullpen ERA (2.94) with a 30-14 bullpen record – trailing only Baltimore in bullpen wins and bullpen winning percentage. In terms of Win-Probability-Added the A’s bullpen added 6.56 wins while the Tigers ‘pen added only 2.00. But… is the Oakland bullpen actually as good as advertised? If we look at xFIP, the A’s relievers are 13th out of 14 AL teams, trailing only the Minnesota Twins (and we know the kind of bullpen rabble they throw out there). They’re also 12th out of 14 AL teams in SIERA (Skill-Interactive Earned Run Average, which predicts ERA based on K%, BB% and GB%).

So what’s the deal here? Why does the A’s bullpen look great by all those facts-on-the-ground measures of actual performance and so bad by the de-lucked xFIP & SIERA measures? Is it, well, “luck”? Maybe, maybe not. As a unit the A’s bullpen doesn’t do a spectacular job in terms of strikeouts (8th in the AL) or walks (last in the AL). As a unit they seem to get the job done as very effective flyball pitchers – they have the highest FB% in the AL but also the lowest line-drive rate. On top of that, their Home Run to Fly Ball ratio is the lowest in the AL at 8%. Fly balls that do not leave the park are fairly unlikely to fall for hits and line drives are extremely likely to fall for hits, so it should come as no great surprise that the Oakland A’s have (by far) the lowest bullpen BABIP in the league at .250.

I did a research piece the offseason before last looking at the relationship between line drive rates and GB/FB – if you want to read it, here’s the link – the gist of which was that extreme ground ball pitchers and extreme fly ball pitchers both do better at limiting line drives and hits on balls in play than does the “average” pitcher. There’s plenty to suggest that those A’s relievers are succeeding in a legitimate and understandable fashion. The Tigers bullpen, for what it’s worth, is also made up primarily of fly-ball pitchers… the Tigers ‘pen has the lowest GB% in the American League and second highest FB%. But… unlike the A’s they haven’t done a particularly good job of limiting line drives or hits on balls in play. Now a part of that BABIP gap might be the big difference in defense at the outfield corners but I’d chalk a lot of it up to the balls getting stung into the gaps. They have done a similarly good job of limiting home runs (with a HR/FB% of 9.1%, 3rd in the AL). With the exception of Joaquin Benoit…

Ah… Benoit. Fly ball pitchers – when they’re at their best – can be extremely effective. But they’re always playing with fire too, aren’t they? This year he has still been pretty effective in most ways: low BABIP, great K/BB, stranding a lot of runners… 18.2% of the fly balls that he has allowed have gone yard. That’s a lot. Fly ball pitchers are going to live and die themselves on whether the ball dies on the warning track. For the A’s, it pretty much had all season long.

And as far as hits on balls in play are concerned – how sustainable is the A’s .250 bullpen BABIP? They should have a lowish BABIP, as a team that allows a lot of fly balls and not a lot of line drives (and if that LD% can’t be sustained that would be effectiveness that could not be sustained rather than luck that could not be sustained). If we use Jeff Zimmerman‘s Rotographs xBABIP formula we would predict a BABIP for the A’s ‘pen of .281 and for the Tigers ‘pen .297 (almost exactly the same as their real .298). That makes it look like the A’s bullpen has been both good and lucky. That’s not the whole story: xBABIP systematically overestimates BABIP for bullpens this year by an average of 17 points. Part of that is the lower league-wide BABIP in 2012 and part of that is the simple fact that more effective pitchers DO have lower BABIPs even for a given xBABIP. Relievers in general are more effective than starters, hence relievers in general have lower BABIPs than starters. Correcting for this “structural overestimation” we’d say that the A’s ‘pen has been only a little bit lucky on balls in play and the Tigers bullpen has been just as unlucky.

Or maybe it isn’t luck at all. By UZR/150 the A’s outfield is above average defensively – coming in at +2.6. The Tigers outfield, on the other hand, has been the worst in the league by UZR/150 at -8.3.