Title: Sabermetric Mining: Relief Pitchers Part 2 (LI & WPA)
Date: September 14, 2012
Original Source: Full Spectrum Baseball
Synopsis: The latest Sabermetric Mining piece dives further into Leverage Index and also includes Win Probability Added, giving readers four different stats by which to measure relief pitcher performance. It also looks at the correlation between LI and WPA stats and future save opportunities, while identifying possible 2013 closer candidates.
In last week’s Leverage Index piece, I discussed potentially using Leverage Index and its derivative stats (gmLI, inLI, pLI, and exLI) to identify potential future closers. It seems, based on responses from Gary in the comments and from Tom Tango himself that I failed to fully explain and made some incomplete assumptions. With that said, I thought it was deserving of an encore article explaining the stats and their potential use further, especially since at this point in the season there are not many Sabermetric Mining topics that could help you make a last-minute push for a title anyway.
It was great to get a response from Tango, who happens to be the man responsible for this beautiful creation, a series of charts showing the Leverage Index of every game, inning, score, and baserunner situation imaginable. I referenced this chart in response to Gary’s question about how LI evaluates leverage, so allow me to explain in greater detail now.
Review Of Leverage Index
Leverage Index is an index of how important a game situation is. Based on the game situation (inning, score, outs, etc), it assigns a grade to how important the situation is when a pitcher pitches, across the totality of their appearances. The higher the Leverage Index, the more the game is “on the line,” when that pitcher is on the mound. A pLI of 1 is a neutral situation, while roughly 10% of situations have an LI of more than 2 and 60% have an LI of less than 1, per Fangraphs.
How Leverage Index Is Calculated
Leverage Index takes into account the inning, score, number of outs, and baserunners to give a score to each and every baseball situation.
Inning – The later in the game, the more leveraged the situation in general, especially in close games. This makes intuitive sense, as the bottom of the 9th is essentially the “last” opportunity (ignoring extra innings) for the outcome of the game to change, whereas in the 8th inning a team that fails to score still has the 9th inning to give it another shot. In blow outs, the leverage actually decreases as the game gets later, since a comeback is less likely for the trailing team. As an example, a tie game with nobody out and the bases empty has a Leverage Index of 1.2 in the bottom of the 5th but a Leverage Index of 2.3 in the bottom of the 9th.
Outs – The logic here is the same as with the inning, as an out essentially advances the “inning” situation by a fractional inning. However, it also decreases the chances of any runs being scored in that inning, since the batting team has fewer opportunities to score a runner who reaches base. Therefore, the overall impact of an out on Leverage Index is multifactorial and can’t just be assumed. As an example, a tie game in the bottom of the 4th with the bases empty has a Leverage Index of 1.1 with nobody out and a Leverage Index of 0.5 with two out. However, a tie game in the bottom of the 4th with runners on the corners has a Leverage Index of 1.6 with nobody out and a Leverage Index of 2.1 with 1 out.
Score – Again following logic and intuition, a closer game has higher leveraged situations than a blow-out. Since Leverage Index measures situational importance, a 1-run game is more leveraged than a 5-run game, since any single at bat could have a large impact on the expected outcome. As an example, a 1-run game in favor of the away team with the bases empty and nobody out in the top of the 7th has a Leverage Index of 1.7, but if that team’s lead were instead 3 runs, the Leverage Index would be just 1.0.
Baserunners – Obviously, baserunners increase the leverage of a situation by increasing the potential for runs to score. As an extreme example, a tie game in the bottom of the 9th with nobody on and two outs has a Leverage Index of 1.5, but if a runner reaches first it increases to 2.4, and if that runner instead had reached third it would balloon to 4.7.
Leverage Index and Relievers
Tango’s main concern with my original article was the assumptions I made with Leverage Index and relievers, as well as the way I described how to use the four different measures for reliever analysis. His piece said the following:
If a reliever enters a game with an LI of 2.00, but his PA-by-PA LI for the game is 2.50, it usually means that he got himself into more jams. It could ALSO mean that he came into a high LI situation in the 8th, then his team made it an even closer game when they came to bat so that when he re-enters the game in the 9th, he was faced with a very high LI scenario. Which is why inLI helps (LI when he enters each inning). If he exits with a low LI it could mean that he put out the fire, or it could be that the fire burned down the whole house that there was no leverage left.
To paraphrase, I incorrectly suggested that you could assume things about a pitcher’s performance based on his gmLI versus his exLI, or just his pLI overall. It’s far more useful and relevant to look at a pitcher’s game by game performance, though this can be tedious and time-consuming. Instead, we can still use Leverage Index in reliever analysis, but we have to utilize it in a slightly different manner than I suggested last week.
Manager Trust – I had outlined how Leverage Index stats could be an indicator of manager trust in a reliever, and I stand by that. With that said, we should not care about pLI (total Leverage Index), as poor performance can inflate this, as can a team’s batting performance. Instead, if we just look at gmLI, which is the average Leverage Index of the game when a reliever is first deployed, we can get a sense of how much the manager trusts a reliever in big situations. Your league leaders in gmLI tend to be closers because of the late-inning leverage increase mentioned earlier, but you also see set-up men like Vinnie Pestano and Antonio Bastardo among the leaders as well.
Win Probability Added (WPA)
As Tom mentioned in his response, Win Probability Added (as WPA, or WPA/LI to control for leverage) is a better method of analyzing reliever performance. Win Probability Added is the difference between a team’s Win Expectancy before and after a play. LI and WPA are closely related, as LI is a measure of the potential for change in Win Expectancy in a situation. That is, a higher LI indicates a greater potential for Win Expectancy to change.
Based on that relationship, there is more opportunity for Win Probability Added to be accumulated or lost in high leverage situations. We can thus use WPA to analyze the totality of contributions by a relief pitcher to a team’s win expectancy. This is a better measure of reliever success than the difference between gmLI and exLI (a method I had mentioned last week), as it accounts for pitchers increasing the leverage of a situation by getting into trouble and ignores the impact of his team’s hitting on LI.
As expected given that WPA values high-leverage success, we see a lot of closers among the league leaders. However, we again see Pestano, as well as a few other successful high-leverage relievers like Darren O’Day and Mike Adams, among others. One strong indicator that this method is better is that you see very few high ERAs when you sort by WPA, indicating that this is probably a better measure of actual performance.
We can also utilize WPA/LI, which is just WPA brought to a leverage-neutral context. I should note that it is not simply WPA divided by LI, but the sum of WPA divided by LI in each situation. This makes it difficult to calculate as a back-of-the-envelope calculation, but luckily Fangraphs provides it on its leaderboards. WPA/LI can be utilized to identify relievers who may be successful if given more high-leverage situations. After all, since a reliever doesn’t control when they enter the game, their total WPA is somewhat dependant on how they’re deployed.
Here we see many closers among the leaders, but we also see relievers who have had strong success but have not necessarily been given high-leverage situations to work with. An example isDarren Oliver of the Blue Jays, who has just a 1.70 WPA despite a 1.62 ERA, mostly because his pLI is just 1.32. Thus, it seems Oliver could be trusted in more leveraged situations to add more win probability to the team.
Utilizing LI and WPA Stats
Again referencing Tango’s responsorial piece, I want to emphasize that using any one of these statistics in isolation is ill-advised. While gmLI can tell you how leveraged the situations have been when a reliever has been deployed, which may be a proxy for manager trust, it tells us nothing of his success in those situations. WPA alone can be effected by manager usage, while WPA/LI is not sufficient alone either as it doesn’t tell us much about usage patterns. Thus, we have to look at our entire menu of LI and WPA related stats when trying to identify high-performing relievers and potential future closers.
I realize it might be frustrating to fantasy owners looking to Sabermetric Mining for fantasy tools, to have read back-to-back pieces that essentially summate to “you have to look at a lot of stats and contextualize,” but that’s the nature of the saves chase and reliever performance. As Gary had pointed out in the comments, too, it is sometimes as simple as going off of what managers say, whether or not we trust their word or agree with their choice. Even still, these stats provide us with a means of evaluating relievers and let us speculate on manager trust and potential future closers, while also letting us appreciate reliever performance outside of the fantasy context (Vinnie Pestano!).
Predictive Ability
At Tom’s suggestion, I pulled 2011 LI and WPA stats and compared them to save totals from this year. If there is predictive ability in leverage stats and win probability stats when it comes to the save chase, we should see some of 2011′s top non-closers in LI and WPA get save opportunities in 2012. While a regression analysis showed that the most predictive of the stats (WPA alone) only accounted for 17% of the variance in 2012 saves, regression is a somewhat flawed tools since save opportunities are finite. Instead, we’re more interested in individual closers who earned jobs in 2012. The chart below shows relievers who had less than 10 saves in 2011 but more than 10 saves in 2012, as well as their LI and WPA stats (heat-scaled for all relievers to give context to each mark, with darker green being a higher rank). Here we see that while not all relievers with strong LI and/or WPA got saves in 2012, most who did get saves previously had strong WPA and/or LI numbers. Casilla (injury-related), Frieri (unexpected improvement), and Cishek (Ozzie Guillen bullpen nightmares) are the exceptions, but with reason.
Buy/Sell Candidates
Because I want to provide some sort of fantasy utility here anyway, at least for those in dynasty formats looking for potential 2013 closer candidates, I have provided the chart below. I have shown the top few in WPA and WPA/LI while highlighting current non-closers that show the potential to close, while also providing the laggards in WPA and WPA/LI, highlighting current closers who seem to have floundered in the role.
Blake Murphy Sports Writing 