Join the SIERA Club

We at MCB – or at least the more sabermetrically-inclined of us – talk a lot about FIP and xFIP (even xBABIP) when describing pitching performance. A metric that we don’t talk about much at all – but probably should – is Skill-Interactive ERA (or SIERA). Of all the “advanced metrics” for pitchers used to give a “true” picture of ERA, it’s SIERA rather than xFIP that is considered to be the best at actually describing how good a pitcher was and the best at predicting what his real-world ERA is likely to be in the coming season (though all of them are better predictors of 2014 ERA than 2013 ERA – which is why they’re actually worth something to begin with).

All of these advanced metrics try to do basically the same thing: predict what ERA “ought” to look like based on a pitcher’s peripherals (like his strikeout rate). What is supposed to make SIERA different and better is that A: they use all the relevant “skill-driven” peripherals and B: the relationship between those peripherals and ERA is carefully modeled and nonlinear. Sabermetricians are always interested in building a better mousetrap and for the moment this seems to be the best mousetrap on the market. You’ll find links below to the exhaustive coverage on Fangraphs of exactly how SIERA is generated, why it works that way and how you should apply it.
SIERA: FanGraphs Sabermetrics Library

Sep 20, 2013; Detroit, MI, USA; Detroit Tigers relief pitcher Luke Putkonen (36) pitches in the ninth inning against the Chicago White Sox at Comerica Park. Detroit won 12-5. Mandatory Credit: Rick Osentoski-USA TODAY Sports

The reason to do this is because while ERA is a near-perfect description of a pitcher’s contributions (it doesn’t get much more fundamental than whether or not the other team scored) there is a lot of luck and noise in there that is due to things other than how well the pitcher threw the ball. A two-out walk, followed by a home run followed by a strikeout is two runs. A two-out home run followed by a walk followed by a strikeout is only one. This noise makes ERA pretty poor for evaluating actual raw talent and for predicting future contributions – and this is especially true for guys that don’t pitch all that much and guys for whom rules for assigning runs to different pitchers add another layer of noise. I’m referring, of course, to relievers.

Here’s a table of the Tigers relievers in 2013, their actual ERAs and their SIERAs – excluding the handful of relief appearances from Doug Fister and Rick Porcello

Name

ERA

SIERA

Joaquin Benoit

2.01

2.86

Drew Smyly

2.37

2.65

Luke Putkonen

3.03

3.13

Jose Veras

3.2

4.18

Bruce Rondon

3.45

3.16

Jose Ortega

3.86

4.17

Evan Reed

4.24

3.72

Al Alburquerque

4.59

3.39

Darin Downs

4.84

3.06

Phil Coke

5.4

4.49

Jose Valverde

5.59

3.32

Jeremy Bonderman

6.48

4.27

Jose Alvarez

7.59

4.82

Octavio Dotel

13.5

5.26

Brayan Villarreal

20.77

7.04

Some of these guys, of course, didn’t see much action but we’ll still include them to paint a more complete picture.  For the most part, what you notice is that ERA and SIERA match up fairly well.  That’s not always the case, but mostly so.  Remember that the whole point of these metrics is to differentiate between pitching well and having a low ERA.  If a guy has a high-ish SIERA but a low ERA – we would figure that he sort of got lucky with that ERA.  Drew Smyly and Joaquin Benoit may have been a little bit lucky, but in terms of talent and performance they were still tops on the team.   Phil Coke – on the other hand – has a bad ERA and a bad SIERA, even if his “luck” was a little bit bad.

To draw special attention to some of the other guys on the list…  Luke Putkonen’s very good ERA roughly matched a very good SIERA.  He actually threw the ball well – you can trust that ERA.  Perhaps we should be less worried about a more prominent role for Putkonen next year.  The same is true for Bruce Rondon – he did throw the ball well, which was reflected in the low ERA.  It wasn’t just a matter of hit sequencing and getting the run charged to some other guy.  Jose Veras, on the other hand, did not pitch particularly well as a Tiger (he was much better in the first half as an Astro) despite the 3.20 ERA.  Maybe that played a role in Dombrowski’s decision to decline Veras’ option and go with other righties.

Al Alburquerque was better than he looked (or rather better than his ERA looked) in 2013 – but then he was also worse than his ERA in 2012 and 2011 so these things do have a habit of balancing out.  The Tigers’ 3rd best reliever, by SIERA, in 2013 was the much-maligned Darin Downs at 3.06 – far, far below his actual ERA.  I get that Downs doesn’t throw particularly hard and I get that Downs was awful against righties and I get that GMs and managers do care about actual runs allowed as much or more than “peripherals” and “advanced metrics” but I cannot fathom to this day why Dave Dombrowski elected to go with Phil Coke (with his high walk rate, low strikeout rate and inability to get righties out) for $2 million rather than Darin Downs (with his low walk rate, high strikeout rate and inability to get righties out) for $500K.

Overall, the Tigers bullpen was 6th in the AL in SIERA – and while that would leave some room for improvement it’s nowhere near as bad as the state of the pen has been made out to be.  Tigers relievers were unlucky in 2013 with hit sequencing, fly ball distance, ground balls just outside the reach of a belly-flopping Prince Fielder, etc…  and that gap between actual ERA and what their ERA “should have been” was the second largest in the American League (only the miserable Astros were unluckier).  It would look like – since 2013 SIERA is supposed to be a much better predictor of 2014 ERA than 2013 ERA would be – that staying the course would have meant a pretty solid bullpen.

However, Dave Dombrowski (or maybe Mike Ilitch) clearly did not see it that way and the Tigers bullpen is being stripped down and rebuilt.  Since Jose Veras wasn’t great as a Tiger (though he was Benoit-good as an Astro) and maybe the Tigers don’t lose much of anything by going with Putkonen, Alburquerque and Rondon instead of him.   Replacing Benoit with Joe Nathan is basically a wash – and though Nathan was the best available reliever (in terms of SIERA) he was barely better than Benoit.  DD made the bizarre decision not to cut ties with Phil Coke – probably the only member of the 2013 (other than Jose Valverde) that fans would have loved to see part of a stripping down/rebuilding of the bullpen is still here.  Drew Smyly was the Tigers best reliever in 2013 by SIERA and he won’t be a part of the 2014 ‘pen.  Darin Downs won’t either.  Ian Krol had a 3.74 SIERA last season for the Nats and we’re left hoping for some kind of a rebound in Coke’s raw ability (not Coke’s luck) and development from Krol just to see mediocrity from the left side of the bullpen.

In short – I believe that doing nothing (aside from holding onto their own players and cutting Phil Coke) would have made for a fine bullpen in 2014.  The changes that Dave Dombrowski has made thus far have made that goal harder to achieve and less likely overall.  Those SIERAs for the non-Coke guys we expect to see in next year’s bullpen don’t look awful BUT they don’t look “elite” either.  Nathan’s 2.82 was 36th in baseball among relievers with 40 or more innings pitched and none of the rest come close to that.  Rondon’s 3.16 would make him 65th, though he didn’t quite hit that 40 IP cutoff.  Most of our attention goes to worrying about the back end, worrying that Coke can’t hack it and that that Putkonen and Alburquerque can’t be counted upon but I’d say it’s pretty difficult to imagine an elite bullpen if your closer is the 36th best reliever in the league and your setup man is 65th…