DVOA EXPLAINED
One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.
Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?
Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.
The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Beanie Wells takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Beanie Wells done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Wells is getting credit for the work done by the passing game.
Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.
We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).
(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)
Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.
Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.
The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)
The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.
For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.
Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +33.8%) and the best defenses have negative DVOA ratings (with Baltimore number one in 2011 at -17.1%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.
DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.
Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.
Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Kevin Kolb and John Skelton can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.
Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1991) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which team had the better offense: the 2011 New Orleans Saints or the 1998 Denver Broncos? Going by total yardage (7,474 vs. 6,092) or even yards per play (6.7 vs. 5.9), it's not even a contest. The Saints were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2011 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that Denver's offense was slightly better relative to the rest of the league (34.5% to 33.0%).
