Are NFL Referee Crews Biased?Examination of Referee Tendencies and Biases in the National Football LeagueJacob KleinBlockedUnblockFollowFollowingJan 30By: Thompson Bliss, Connor Daly, Jacob Klein, Patrick LewisNext weekend, millions of people around the world will tune in to watch Super Bowl LIII.
Following what many are considering to be two of the most controversial playoff games in the history of the NFL, this year’s event features the Los Angeles Rams and the New England Patriots.
Both of these teams managed to advance to the prestigious game with some thanks to favorable penalties (or lack thereof) called by the referee crews officiating their respective games.
Throughout recent years, an increasing amount of attention has been placed on penalty calls in the NFL.
Penalties can dramatically alter the flow and outcomes of games (as was arguably the case in both the AFC and NFC Championship games last weekend), so ensuring that penalties are called fairly and consistently is crucial to maintaining the integrity of the game.
The purpose of this analysis is to examine tendencies, biases, and patterns in penalties called by NFL officials.
Specifically, we wanted to examine whether all teams are penalized at similar rates, whether there are differences in penalty calls against home and away teams, and frequencies of penalty calls by both penalty type and field position.
Some background: Every NFL game is watched by a group of seven officials, which includes one referee, one umpire, and five line judges.
Although NFL officials are labeled by name in the following analyses and visualizations, it is important to note that these names represent the head referees of the crews; thus, this analysis focuses on trends and biases by referee crews, rather than by individual NFL officials.
Data for this analysis was collected from the beginning of the 2013–2014 NFL season through week 9 of the 2018–2019 NFL season.
We obtained play-by-play data from NFL Savant and referee game data from Pro-Football-Reference.
Which penalty types are most common?Weekly viewers of NFL football are sometimes confused about why certain types of penalties occur more frequently in one week versus another.
This inconsistency certainly could be a function of luck combined with sloppiness by the players, but it also could have to do with biases by referee crews towards calling or not calling certain penalty types.
For example, some referee crews may throw the flag for any action nearing an illegal tackle while others may be compelled to let questionable actions slide and go uncalled.
Before turning to the differences among referees, it is helpful to have a sense of the overall distribution of penalty types.
Offside and Illegal Block are the most common penalties.
This commonality is not surprising as running across the line of scrimmage and blocking are two fundamental aspects of the game and thus are bound to be done incorrectly sometimes.
Moreover, it is also not surprising that particularly sloppy penalties such as Too Many Men on the Field, Illegal Formation, and Delay of Game are rare.
Looking at the distribution by referee, we learn that not every referee crew calls penalties at the same rate.
The above graph provides insight regarding tendencies of NFL referee crews to call or not call certain types of penalties.
For example, the crew of Bill Vinovich seems to call more penalties than average regardless of penalty type.
Additionally, Walt Anderson’s crew seems to call less penalties than average regardless of type.
Are more penalties called against the away team?It is common to think that teams are often called for more penalties when they are playing on the road as opposed to playing in their home cities.
This difference can be a function of player familiarity with the field, miscommunication between members of the away team due to loudness of the home crowd, and/or referee crew reaction to crowd noise encouraging or discouraging certain calls.
The percentages of calls against home teams and away teams by penalty type are shown below.
We see that a timingpenalty (i.
e.
Delay of Game) is the most unfavorable towards the away team, which can likely be attributed to home crowds often increasing their noise levels in attempts to cause such penalties on the away teams.
However, the higher degree of Illegal Tackle penalties that we observe is fairly surprising.
An illegal tackle penalty is not based on familiarity with field or pre-snap timing, which suggests that it may be a function of referee crew responses to crowd noise.
Let’s now turn to the percentages of penalties called against home teams and away teams by referee crew.
Here we see that nearly every referee crew calls more penalties against the away teams than the home teams.
The crew led by John Hussey is the only exception and is just below the 50% mark, while Pete Morelli’s crew has the highest percentage of penalty calls against away teams at nearly 55%.
As a whole, all of the referee crews call home and away penalties at rates around 50%, which suggests that penalty calling is quite fair even though we would expect a slight trend toward more penalties called against away teams because of the aforementioned points.
Where on the field are penalties most often called?Example of the interactive component in use.
This component serves to act as an interactive application that users can use to learn about how penalties within the NFL occur at different frequencies depending on field position.
To play with this interactive component yourself click here.
ConclusionAlthough many technological advances are constantly being made in just about all sports (such as the introduction of instant replay), referees are an inherently human part of football.
Thus, at the core of the exploratory data analysis conducted in this project lies human behavior.
No two humans see things the exact same way, which serves as some justification for the inconsistencies we see in referee penalty calls, despite much effort taken by the NFL to define the constitution of particular penalties in objective terms.
Therefore, we must err on the side of caution when identifying anomalies or inconsistencies in our dataset.
John Parry, the head referee of this weekend’s Super Bowl, will try to lead a consistent crew throughout the game.
According to the data, his crew tends to call a high number of pre-snap penalties such as offsides, delay of game, and illegal formation.
Moreover, his crew tends to call fewer pass interference penalties than average.
Will a missed pass interference penalty have a significant impact on Sunday’s game, as was the case last weekend?.We will have to wait and see.
If interested in exploring the dataset, code, or analysis in more detail, you can find all work located here.
.