Mark T. Williams, the James E. Freeman Lecturer in Management at Boston University Questrom School of Business, and a team of graduate students at the Questrom School of Business experienced in data mining, analytics, and statistics spent more than two months analyzing almost 4,000,000 pitches over the past 11 MLB seasons. (all emphasis below is mine)
Baseball is here, another season of amazing catches, overpowering pitching, tape-measure home runs, overpriced beers, and, yes, television replays of every missed call by umpires, revealed in painful, high-definition slow motion.
It's time for Major League Baseball to put an end to the agony caused by at least some of those blown calls—the balls and strikes. ...
In 2018, MLB umpires, made 34,294 incorrect ball and strike calls for an average of 14 per game or 1.6 per inning. Many umpires well exceeded this number. Some of these flubbed calls were game changing. ...
Blown calls only undermine the integrity of the game, slow down pace, hurt averages, and prevent the athletes from being able to maximize their potential performance. ...
In 2018, a total of 55 games were ended when umpires made incorrect calls. ...
For this research, we looked at game data from Baseball Savant, MLB.com, and Retrosheet. The time period chosen, the most recent 11 baseball regular seasons (2008-2018), presented nearly four million called pitches. Similar to players, MLB umpires were assigned numbers, so that games behind the plate could be easily tracked. All active umpires were included in this performance study, and their ability to accurately call balls and strikes was closely observed. All 30 major league parks are outfitted with triangulated tracking cameras that follow baseballs from the pitcher's hand to across home plate. Ball location can be tracked up to 50 times during each pitch and accuracy is claimed to be within one inch. ...
This deep-dive analysis demonstrated that MLB umpires make certain incorrect calls at least 20 percent of the time, or one in every five calls. Research results revealed clear two-strike bias and pronounced strike zone blind spots. Less-experienced younger umpires in their prime routinely outperformed veterans, and umpires selected in recent World Series were not the best performers. ...
Research results demonstrate that umpires in certain circumstances overwhelmingly favored the pitcher over the batter. For a batter with a two-strike count, umpires were twice as likely to call a true ball a strike (29 percent of the time) than when the count was lower (15 percent). These error rates have declined since 2008 (35.20 percent), but still are too high. During the 2018 season, this two-strike count error rate was 21.50 percent and repeated 2,107 times. The impact of constant miscalls include overinflated pitcher strikeout percentages and suppressed batting averages. Last season, umpires were three times more likely to incorrectly send a batter back to the dugout than to miss a ball-4 walk call (7 percent). Based on the 11 regular seasons worth of data analyzed, almost one third of batters called out looking at third strikes had good reason to be angry.
Umpires from 2008 through 2018 also exhibited a pronounced and persistent blind spot with a number of incorrect calls at the top of the strike zone. Remarkably, pitches thrown in the top right and left part of the strike zone were called incorrectly 26.99 percent of the time on the right side to 26.78 percent on the left. And while there was marked improvement in umpiring, the incorrect calls around the bottom right strike zone in 2018 was still a mind-boggling 18.25 percent. Data results confirm that strike zone blind spots penalized certain pitchers more than others. This time, however, batters benefited from such flubbed calls, as strike zones shrank, forcing pitchers to throw fewer pitches up in the zone. ...
[R]esearch uncovered that umpires on the Bottom 10 MLB performance list (2008-2018) had an average experience level of 20.6 years, were 56.1 years of age, and had an average BCR of 13.96 percent. This group's error rate was a staggering 56 percent higher than the top 10 MLB performers. Umpire Jerry Layne, with 29 years on the job and at age 61, sported the highest BCR, 14.18 percent. This performance research clearly indicates that more experience and age does not necessarily produce the best umpires. ...
For 2018, Ted Barrett and Joe West were the top poor performers, making 495 and 512 incorrect home plate calls, for an average of 17.7 and 16.5 errors per game, respectively. Such bad call numbers can produce an array of new outcomes. For example, incorrect calls can extend pitch count and impact pitcher rotation and the reliance on relievers. As a starter gets deeper into his pitch count, one or two more balls can change game outcome. Bad calls in favor of batters can extend innings, and increase scoring opportunities. ...
The statistics show that West made more incorrect calls than most. In fact, behind the plate, over the last 11 seasons he has averaged 21 incorrect calls a game, or 2.3 per inning. And while Angel Hernandez receives similar fan dislike, averaging 19 incorrect calls a game, or 2.2 per inning, even with this high error rate, compared to his peers, he performed better than others, escaping the 2018 Bottom 10 MLB list. ...
Recently, Hernandez stated he only gets four calls wrong per game. His actual error rate, as evidenced in this research, was almost five times higher. ...
Despite the hard evidence, each season, MLB continues to keep questionable performers, some past their prime, on the job. ... Game by game, season by season, poor performing umpires remain on the field. ... In 2018, 2 percent of all major league games (55) were ended by incorrect calls, an increase of 41 percent from the previous year (39). ...
If the league is truly committed to game improvement, its officials should aggressively recruit and retain high-performing umpires, as any smart industry does. ...
Despite years of data-driven evidence, MLB has been slow to expand the ranks of younger umpires, missing the opportunity to rapidly lower unacceptably high bad call rates. The league has also dragged its feet in putting strike zone–assisted technology behind the plate. In a thinly veiled attempt to silence vocal critics, MLB recently announced it will begin to test robot umpires, but only on a small scale, through the unaffiliated Atlantic League farm program. Instead of addressing this pervasive big-league problem now, MLB continues to stall. ...
Technology does not have to mean the death of umpires. Rather it's a tool to allow them to do a better job. ...
Major League Baseball has been a follower and not a leader in adopting innovative technology. In contrast, other professional sports have increasingly relied on high-tech aids, rapid communication, and centralized control rooms to improve officiating. ...
Major League Baseball's goal should not be to resist change, but to adhere to the official strike zone that its own rules make clear–on every pitch. ... Imagine player and fan experience and what baseball would look like if each year the more than 34,000 incorrect calls vanished. ...
It is unrealistic to assume that home-plate umpires, unassisted, can collectively achieve the accuracy rates increasingly demanded by the sports industry and deserving fans. ... Adopting a stronger performance-based system coupled with readily available technology would allow the human aspect of the game to remain while respecting the benefits that can come with advancing technologies. At minimum, using a tech-assisted approach would produce results no worse than our existing band of MLB umpires.
Excellent stuff. Like you, I've taken to muting NESN and listening to the radio feed. On Tuesday the booth had a half-inning with Manfred and I thought it was pretty good. Asked him all sorts of questions, which of course he mainly skated around but did sound surprisingly sane. However, the one thing never mentioned was the robot balls and strikes calling, which surprised me, since Joe and Lewen even brought up tanking. An opportunity missed for sure.
ReplyDeleteIf I had any computer skills I'd somehow "viral" this baby everywhere.
It's a fascinating read, and the conclusions are awful in many ways.
ReplyDeleteBut it also buries something major: The numbers (both ball-called-strike percentage and bad-call rate) have gone down every year in the survey period, and the cumulative effect is huge. From 2008 to 2018, the ball-called-strike percentage dropped more than 41.5 percent; the bad-call rate fell 43.7 percent.
I can think of two obvious explanations, with the first being the age demographics of the umpiring pool. The second is the likely effect of accountability (in the form of widely dispersed data, both on TV broadcasts and after the fact) on behavior; if umpires don't want to look like fools, they need to (and apparently can) do better.
I know you favor robo-umpires, and I don't disagree, but the silver lining here is that ball/strike umpiring has improved substantially over the past decade -- and the trend doesn't appear to be abating.
The numbers (both ball-called-strike percentage and bad-call rate) have gone down every year in the survey period, and the cumulative effect is huge. ... The second is the likely effect of accountability (in the form of widely dispersed data, both on TV broadcasts and after the fact) on behavior
ReplyDeleteI agree. Those shrinking percentages were very surprising. And encouraging - though of course robots would bring them much further down. (Also, I don't recognize the names of a lot of those better umps. They must work in the NL mostly.)
You know what I'd love? BrooksBaseball pitch/fx diagrams of Eric Gregg's infamous 1997 NLCS game. That would be insane. (Some video.)
I wonder if robocalls would actually benefit a particular kind of hitter. There must be guys out there who actually see and know the strike zone almost as well as a machine, and those players have been cheated out of a chance to demonstrate a legitimate baseball skill for years.
ReplyDeleteThis is sort of a barroom argument, but...
If you can look past #9 and evaluate just the current talent, who would you nominate for the finest batter's eye in baseball, right now?
A lot of people would say Joey Votto (.426 lifetime OBP, including .414+ in 9 of the last 10 seasons). Also, Votto treats TSW's book like the Bible. (Crazy stats!)
ReplyDeleteInterestingly, speaking to my friend and fellow STH at Tuesday's game, he actually was on a plane going to LA last year, and he happened to be sitting next to a person who works for Joe Torre and handles the umpires (I didn't catch the name but I'm sure we could research it). The discussion from there proceeded to address part of this issue related to the union, which similar to other unions makes it very difficult to get rid of long-standing members like West, Barrett, and Hernandez despite negative performance.
ReplyDeleteMLB has only been able to hire a handful of new umpires over the past number of years for this reason. In other words, until these ass-clowns decide to hang up the chest protector, we will continue to deal with bad balls unless MLB enables the use of StatCast to call the balls and strikes.
(I'll be clear: this is not a rant against unions. I have worked with several union members at my job and the vast majority of them are good, hard-working people who would get the shaft from businesses without the assistance of union representatives. That said, my mother was a steward with a nurses union for several years, and even she recognizes that one of the negatives is that it often protects people she dubbed "the lowest common denominators," and it becomes more and more difficult as they become more senior in the organization to get rid of them.)