Stathead Rebuttal: Wins and WPA

John Parent put up a piece a couple of days ago lauding Verlander for his wins and Jose Valverde for his saves and expressed dismay at not receiving the expected sabermetric lecture in response on why wins and saves are so old-fashioned and don’t matter at all. So, here it is. If you’ll follow through the jump:

Wins, Losses and Saves are not totally irrelevant – they are a crude (but easy to determine) measure of how the pitcher is actually contributing to his team’s chances of success – not how ‘good’ he is, but what he has actually accomplished. A guy that is causing his team to win more is clearly having a good year, right? They capture something that most individual performance metrics don’t: the impact of timing and consistency. The problem is that being crude measures they are contaminated by the impact of what the rest of the team contributes or fails to contribute. We want measures that reward a guy for kicking it up a notch in a 1-0 duel but don’t overly punish him for just trying to get it over the plate with a ten run lead. ERA, WAR, etc… those don’t do the trick and they don’t adequately explain just how important that pitcher has been to the team this year.

But, it’s 2011 and we have better measures than wins and saves – we have stats based on ‘win expectancy’. Those use a sort of probability matrix to assess how likely it is that the team is going to win when a pitcher enters the game versus when the pitcher leaves the game, usually attempting in some way to limit the relevance of the team’s offense. If the Tigers have an 88% chance of winning the game when Jose Valverde enters the game, and have a 94% chance of winning the game after he strikes out the leadoff batter – that adds 0.06 to his ‘win-expectancy’ count. This continues to add up for every play of the game that he is part of. Same goes for Justin Verlander Nonetheless, a 50% chance at the start of the game, a 51.5% chance after getting the leadoff man out, a 49.5% chance after walking the second batter and a 52% chance after inducing a double play give a cumulative 0.02 added for that inning. There is one big caveat: the things that happen in between innings will affect the win probability when he goes back out to the mound – he won’t get credit (unless I am grossly mistaken) if the Tigers score 8 in the bottom of the first, but giving up a home run in the second will cause a much smaller loss of WPA for Verlander if they had, because the situation isn’t as important – and pitching a near-perfect game won’t add as much either. There are a lot of ways to do these calculations, mostly involving tweaks affecting the importance of offensive support, I’m not going to look at all of them – just fangraphs Win Probability Added or WPA, in my mind the simplest option available.

So, with his perfect record in save opportunities where does Valverde stand in WPA?

Well, not bad at all: he’s 6th in the major leagues with 2.77 wins added. That means that with an ‘average’ reliever instead of Papa Grande, the Tigers would probably be in a dead heat with the Tribe right now. One note, don’t compare this to something like WAR – WPA is above average, not above replacement level. Valverde has been great in the clutch, pitching significantly better when it matters more, but relative to some other top relievers he hasn’t had quite the same ‘leverage’ in his appearances. Who’s been better than Valverde? Jonathan Papelbon, Joel Hanrahan in Pittsburgh and John Axford in Milwaukee, Phillies fill-in closer Antonio Bastardo… but the list is topped by a couple of non-closers (obviously not getting a lot of saves) who have been having phenomenal years. The Braves Jonny Venters has been virtually perfect this season, despite only 4 saves. His ERA sits at 1.20, he’s striking out about 10 per 9 and walking about 4. He’s getting ground balls over 75% (!) of the time, and when it is hit in the air it’s resulted in a homer only once. His WPA sits at 4.52, and he’s definitely been better than Jose Valverde with his 34 saves. The same can be said of Tyler Clippard of the Nationals with 4.56 WPA. Clippard’s season has been downright strange – he has not allowed a single baserunner to score all year and yet has a 1.68 ERA. How? He’s a pure flyball pitcher and gives up a ton of home runs, nonetheless – 11 Ks and 3 walks per 9 combined with a BABIP under .200 mean there haven’t been many guys to drive in. Clippard has zero saves this year, and his team is pretty bad, but he’s meant more to them than Valverde has. Of course, I don’t mean to diminish Valverde’s contribution this year… 2.77 WPA is pretty impressive. Consider what some of the other relief aces in the AL Central have given there teams this year: -1.91 WPA for Matt Thornton of the Sox (worst in the league), -1.13 for Joaqim Soria in KC and -0.74 for Matt Capps of the Twinkies. That’s right… trade Valverde for Soria and we’re worse than Cleveland.

The same stat and the same calculation works for starters too, so how has Justin Verlander fared?

4th in the majors and 3rd in the AL with 3.80 WPA. The guys ahead of him? Cole Hamels with 4.03, Josh Beckett with 4.13 and Jered Weaver with 4.75. 4th is good, of course, but given how impressive he has been to watch a Tigers fan would probably expect to see him at the very top. So why isn’t he there? Well, he has been terrible in the clutch – statistically pitching far worse when it matters most. If a guy is no better or worse when it counts, he’ll have a clutch stat of zero, anything positive is ‘clutch’ and anything negative is the opposite. Anything greater than one (like Valverde has) is a guy who has been extremely good in the clutch. Anything below negative one is equally extreme, but bad. Verlander’s clutch stat this year is a negative 1.36, a terrible number 5th worst in the bigs among starters. Everyone else in the bottom 8 has a negative WPA on the season, including the Rays’ David Price since it matters disproportionately what you do in clutch situations. The fact that Verlander still has a 3.80 WPA is a testament to how fantastically well he has pitched overall this year – but says very little about him ‘knowing how to win’, if anything the fact that he does not know how to win has tempered what should be a season for the ages: Verlander has more strikeouts per 9 than Weaver, fewer walks per 9 and a lower BABIP, but still a higher ERA and a lower WPA.

Josh Beckett doesn’t have a lot of wins this year, only 9, and he’s pitched a full 50 innings less than Verlander, but with an ERA of 2.17 he would be considered real competition for him in the Cy Young race this year if WPA was high on voters minds. Jered Weaver isn’t so far behind Verlander in wins and innings, and far ahead in ERA as well as WPA. The key for those two is not simply a positive clutch number but a much higher percentage of runners stranded than Verlander. That does matter, and that is the reason why Verlander’s ERA and WPA are excellent but something less than astounding. I love me some Justin Verlander, but if I had to pick an AL Cy Young winner today, it would go to Jered Weaver instead.

EDIT: Since I wrote this, bad starts by Jered Weaver and Josh Beckett have lowered their WPAs to 4.44 and 3.76 respectively, so Verlander is now second in the AL and not quite so far behind Weaver as he had been. Valverde’s 35th straight save has raised his WPA to 2.97.

Next Tigers Game View full schedule »
Wednesday, Sep 33 Sep7:05at Cleveland IndiansBuy Tickets

Tags: Jered Weaver Jose Valverde Josh Beckett Justin Verlander

  • jestem0

    Not to be too picky here, but was this really posted AFTER Weaver just gave up 8 earned runs over 4.2 innings today? A little up-to-date research from a self-proclaimed Stathead would be nice to see. Weaver’s 2.13 ERA is hardly “far ahead” of Verlander’s 2.35. One of the biggest issues many fans have with Sabremetric stats is that they can sometimes fail the sensibility test which just undermines the whole exercise. I would wager there is not one single individual in the Major Leagues today that would say Verlander’s ability of “knowing how to win” is at the bottom of all starting pitchers. Not being able to withstand such sensibilities should temper the reliability of a statistic that is only as good as the algorithm.

  • channummcb

    Well, the piece was written before that game (and Valverde’s 35th) but set to post overnight. That one game has brought Weaver and Verlander closer together, so the ‘far’ is now obsolete, but Weaver still comes out ahead – in certain metrics. Verlander ‘not knowing how to win’ is simply a jab at John’s earlier piece and not intended to be taken literally.

  • jestem0

    Of course, I figured as much. I didn’t really take any of that literally. I was just taking my own jabs because the way the WPA stat reads Weaver only really has a substantial lead on Verlander in the Clutch metric. I enjoy Sabremetrics as much as traditional stats, but i don’t always agree in how they are perceived.

    Verlander well outpaces Weaver in WPA/LI. Perhaps this is an issue that Clutch has too much weight in the WPA algorithm. Just becuase WPA currently exists does not mean it is an entirely accurate measure. Which ultimately can relegate it to the same category of the associative-stats of Wins and ERA.

  • BobbyDallas

    You might also want to look at the Context Neutral Wins or WPA/LI in your analysis. In that case Verlander is tops in the league. Basically it takes takes the Win Expectancy for a game situation, which you described really well above, and divides it by the Leverage, or “Suspense”, of that situation. If I am reading it right, it seems that Weaver gets himself into trickier or more suspenseful situations, game average leverage (pLI) of 0.99, while Verlander does less so with a 0.95 pLI.


  • sportz

    Sabrmetrics at its finest, great article Chris.

    It does speak to the flaws in the balancing of the formlas, that clutch pitching as a metric measurement, seems to skew the value so significantly..and diminishes the real quantitaive measurement.

    which is:

    If you dont let runners don’t pitch in high leverage situations..who has pitched more low leverage innings because of his own pure dominance than Justin Verlander.

  • channummcb

    @jestem0 I used WPA specifically because it’s the simplest measurement to calculate and the simplest to explain. Granted it isn’t perfect, and when you come right down to it all of the other ways win expectancy stats are calculated are attempts to fix the imperfections so it more accurately reflects what we would think of as the player’s skill. But WPA isn’t supposed to measure skill (we use other stats for that), it’s supposed to quantify the player’s accumulated contributions to his team in the same way ‘wins’ or ‘runs batted in’ do but to do so as completely as possible.

  • channummcb

    @BobbyDallas Yes and no. Since WPA is calculated play-by-play, like leverage, a lot of a starting pitcher’s suspenseful situations are his own doing.

    Verlander isn’t letting a lot of people get on base, even relative to other aces, which would give him a lower pLI all by itself. So on that count, Yes. But Weaver should get a negative to his WPA when he walks the leadoff man, then larger positives when he then retires the side. If Verlander goes 1-2-3 the cumulative result of the scoreless inning should be exactly the same in WPA, but not WPA/LI because Weaver would have a different LI. Verlander pitched better in an objective sense, but the end result for his team was exactly the same. For this purpose and this purpose alone I’d stick with WPA. The other issue, for me, is that trying to correct for LI means that it shouldn’t matter whether the game is close or not – and the importance of being at your best in close games is a big reason to bother with these statistics at all instead of just looking at WAR and WAR alone.

  • channummcb

    @MarkGorosh True, but it is a fact that he hasn’t pitched as well after having let runners on than some of his Cy Young competition this year.

  • jestem0

    @channummcb@MarkGorosh Precisely. Sabremetrics at its finest. A pitcher who leads the AL in innings pitched, strikeouts, wins, WHIP, AVG against and is 2nd in ERA is “proven” to be the inferior pitcher.