State of Analytics: How the Movement Has Changed the NFL and Where it Has Fallen Short

Football players often focus on getting bigger, stronger and faster in an effort to dominate whoever lines up across from them on the gridiron.

Or, as San Francisco 49ers defensive end Dee Ford told Sports Illustrated: “Old-school coaches say, ‘You need to be tough.’”

The last thing football teams need is a computer spitting out numbers or an analyst with a master’s degree who has never played the game calculating ways to capture a winning edge.

At least that’s what the game’s macho culture, ripe with war analogies that stem from its violent nature, has told us about how the sport has long perceived stacks of data and advanced analytics.

But that’s not entirely accurate.

There is little doubt that football has a complicated history with advanced metrics. In one respect, the practice of quantitative analysis as a way to gain a tactical advantage has been part of the game for decades, and some organizations would be offended by an out-of-touch label. The league itself toyed with a system to better rank quarterback play for 30 years before commissioner Pete Rozelle’s committee on the subject finally established the groundbreaking passer rating formula to determine the NFL’s passing leader starting in 1973.

Paul DePodesta played a key role in baseball’s ‘Moneyball’ movement before coming over to the NFL. (AP)

The flip side, however, is that general managers and coaches aren’t likely to fully embrace analytics until they have the confidence to make unconventional, data-driven roster moves, player evaluations, lineup changes and game-day decisions without fear of getting fired. And that’s entirely understandable: Careers are short and the expectation to win immediately is ever-present. In those circumstances, embracing what may be seen as unconventional is a difficult step to take.

Things came to a head in the early part of this century when owners across sports wondered if the way baseball general manager Billy Beane, assistant Paul DePodesta and the cash-strapped Oakland A’s succeeded by embracing an analytics-heavy scouting and player evaluation model could help them get the most out of their spending.

“The most entertaining thing you can do is win games, and if you’re an executive you should do everything you can to make the smart and most efficient decisions you can make.” – Baseball executive Billy Beane

The A’s game-changing strategy was the subject of Michael Lewis’ 2003 award-winning book ‘Moneyball: The Art of Winning an Unfair Game’ and featured in the ’11 film adaptation that was nominated for six Academy Awards, including Brad Pitt for playing the role of Beane. Though faith in the value of modern metrics spread at different speeds and to various degrees, franchises across sports began to adopt the mindset that there is indeed value in considering – and trusting – the data.

Football teams were eventually no different, though the more innovative front offices have predictably faced more of an uphill climb than those ahead of them in baseball and basketball. Still, it would have been difficult to imagine how football analytics (and rule changes) would change the way front offices are structured, influence offensive strategy and seemingly devalue what was once one of the most important positions on the field. There was also no way to predict how technology and tracking data would alter the way some franchises view the game.

“The fact that this happened is not a surprise at all,” Beane told The Washington Post about the spread of analytics across sports. “Initially, it took longer than I would have expected. But once it gained momentum, it went faster than I would have expected.”

Bill James sought to expand the thought process beyond baseball card numbers to the “ever-expanding line of numerical analysis.” (AP)

“The most entertaining thing you can do is win games, and if you’re an executive you should do everything you can to make the smart and most efficient decisions you can make,” he added. “If that results in a less entertaining game to one or two people, there will be three people who like it more. Losing games isn’t entertaining for anybody.”

It was starting in the 1980s when Bill James, a baseball fan and aspiring writer, attempted to expand the thought process beyond the numbers on the back of a baseball card and into what he called the “ever-expanding line of numerical analysis.” James eventually worked with STATS, Inc. – now Stats Perform – to publish books about his revolutionary statistics.

Because of his work, he would become known as the Godfather of Sabermetrics – the Society for American Baseball Research (SABR). James inspired others to follow with their own ideas, algorithms, statistics, articles and books, like John Thorn, Pete Palmer and David Reuther’s ‘The Hidden Game of Baseball: A Revolutionary Approach to Baseball and Its Statistics.’ The flood of new information continued to evolve in the ’90s and accelerated from there.

Some 30 years before Moneyball pushed the numbers craze into mainstream culture and across sports, Virgil Carter, a quarterback for the Cincinnati Bengals who had a master’s degree from Northwestern, and Robert Machol, a systems engineer credited with pioneering research in wake turbulence, wrote an academic paper called “Operations Research on Football” that featured data on the value of possession and introduced expected points.

Thorn and Palmer’s influence expanded to football in 1988 when they teamed with Bob Carroll to write ‘The Hidden Game of Football,’ which built on Carter and Machol’s concepts and popularized expected points added (EPA). EPA takes several factors such as down, distance to go, field position, home-field advantage and time remaining on a play-by-play basis and assigns a numerical value for each play result.

The formula was resurrected by ESPN’s Brian Burke in 2010, and the rise of the advanced analytics movement along with social media helped it gain popularity it never had in its early days. In the early 2000s, coverage of football had followed after the Moneyball trend as websites dedicated to analyzing statistics like Burke’s Advanced NFL Stats, Football Outsiders, numberFire and Pro Football Focus were created.

At ESPN, Brian Burke helped EPA gain popularity it never had in its early days. (espnpressroom.com)

The concept of EPA became the foundation for ESPN’s total QBR, a statistic that sought to better measure a quarterback’s performance and provide a more accurate, data-driven alternative to the once-innovative passer rating. Already considered by some to be the Bill James of basketball, Dean Oliver brought his analytic mind to football as part of ESPN’s Stats & Information Group that developed total QBR – along with input from analysts Trent Dilfer, Jon Gruden and Ron Jaworski. Oliver had previously sought to break the mold in 2004 with his groundbreaking work ‘Basketball on Paper’ during a time when modern metrics had yet to become mainstream in NBA front offices and within the fandom.

Much like it had done in the early collaboration with James, Stats Perform was at the cutting edge when it began collecting X-Info football data as early as 1992 before adding things like defensive targets, broken tackles, defensive-line pressures and formations, defenders in the box and quarterback release time and player involvement. Eventually, things as advanced as offensive and defensive play schemes and routes were tracked.

“You better get a head coach that really believes in (analytics), or you’re going to have real serious conflicts.” –  Former NFL executive Joe Banner

It was around this time that innovative Philadelphia Eagles executive Joe Banner looked to gain a strategic edge from the data by establishing what is believed to be the NFL’s first analytics department and contracting MIT students for studies. Banner proclaimed there was “a competitive advantage in analytics” and was determined to help the information gain headway against the fear that data scientists would take decision-making power away from coaches and that computer nerds would take their jobs.

“Analytics are sophisticated, accurate and quality information,” Banner told The Washington Post. “There should be no coach afraid of that.”

“Football is very, very much driven by conventional wisdom. Most industries are this way. But in football, it’s true to an extreme degree.”

Former Eagles executive Joe Banner established what is believed to be the NFL’s first analytics department. (AP)

The Cleveland Browns made their lasting commitment to analytics evident by naming Banner the franchise’s CEO in 2012 and hiring DePodesta – Beane’s right-hand man early in the Moneyball era – as its chief strategy officer in ‘16. After leaving the Browns to join the Atlanta Falcons, Banner told ESPN in the wake of the landmark hire of DePodesta that although the use of analytics should increase in the NFL, the consequences of a forced data-driven commitment could be disastrous.

This was his eye-opening comment on ESPN radio just five years ago that revealed how far the league still had to go in terms of embracing modern metrics: “You better get a head coach that really believes in this, or you’re going to have real serious conflicts. … The analytics will tell (you to) trade away players as they get older for future draft picks and accumulate as many draft picks as you can. … It’s going to take a unique coach in the NFL. There isn’t one of the 32 right now that’s going to walk in and say, ‘Hey, I believe in that.’”

Still, there is some evidence that the analytics evolution has trickled down from front offices to coaching staffs, leading to a revolutionary shift in strategy that likely should have occurred much sooner. The rise of smart young general managers and offensive-minded coaches has played a role in this glacially paced movement.

The New England Patriots are believed to have incorporated some level of analytics for years because their decisions often fall in line with what the data would suggest, though there’s so much secrecy surrounding the organization that no one really knows for sure. The Jacksonville Jaguars are also considered one of the leaders in the use of modern metrics after they openly launched one of the league’s most notable analytics departments under executive Tony Khan. And Minnesota Vikings general manager Rick Spielman has been outspoken about his use of a data hub at their practice facility.

“Now we have countless models ingesting thousands of data points per game, generating more storytelling elements and better predictions for future games.” – Stats Perform AI Data Analyst Kyle Cunningham-Rhoads

Stats Perform played a key role in this advancement by providing statistical analysis and services for many teams, progressing from collecting and distributing raw data to building a user interface to present the raw data to finally using the data to project game and player outcome.

“Historically, football is extremely limited in terms of the types of statistics available,” Stats Perform AI Data Analyst Kyle Cunningham-Rhoads said. “There are 22 players on the field for every play, but in a box score only two or three are actually getting a stat on each play. We wanted to have more context about what players were doing, and a greater ability to tell the story of a game.

“We had to figure out what we had to collect to answer questions about player evaluation and prediction, and then we had to test and refine those data points to figure out which ones were valuable and how to use them. Now we have countless models ingesting thousands of data points per game, generating more storytelling elements and better predictions for future games.”

The league went through another gradual development that began with the 1978 rules changes that made it illegal for defenders to make significant contact with a receiver more than five yards past the line of scrimmage and eventually led to the fall of many passing records in recent seasons.

In between, we’ve witnessed the rise of the West Coast offense, the no-huddle, the run-and-shoot, the K-Gun and more rules that limited defensive backs and protected quarterbacks and wide receivers. There was the Greatest Show on Turf, the spread offense, the use of tight ends and running backs as receivers, the RPO and more rules cracking down on contact in the secondary.

Ultimately, the NFL’s purposeful rule changes and the ensuing schematic innovations birthed the desired impact: Fans flocked to watch as more passing led to more touchdowns and television ratings spiked substantially. However, it also led to something that would have been unimaginable 30 years ago – what was once one of the game’s most prestigious positions became one of its most replaceable.

Ten of the 11 years with the most passing attempts (68.3-71.5) occurred between 2011-20 and the NFL seasons with the fewest rushing attempts per game league-wide (51.8-54.6) all took place from 2010-20. The 10 seasons with the most rushing attempts (75.0-82.5) all occurred during the three-yards-and-a-cloud-of-dust era from 1935-51.

With fewer runs, the value of a good running back appears to have plummeted. After 43 backs were taken in the first round of the NFL draft between 1970-79 and 50 were selected from 1980-89, that number dropped into the 30s in the ’90s and 2000s before falling all the way down to just 16 between 2010-19. The last running back to be taken with the No. 1 overall pick was Penn State’s Ki-Jana Carter in 1995 and since then, there have been 17 quarterbacks picked first overall after Cincinnati took LSU’s Joe Burrow in 2020. And with some exceptions, teams have become no longer willing to pay running backs the big bucks they did in the past. According to Spotrac, 12 of the 13 highest-paid players last year were QBs, while the top-earning back was Christian McCaffrey at No. 58.

In some ways, the drafting habits of front offices in recent years expose a shrinking but still lingering divide between them and their coaching staffs. There remain moments that can’t be ignored, revealing the game’s ingrained conservatism, and the security blanket of the running game hasn’t completely dissipated.

It wasn’t long ago when coach Lovie Smith proclaimed that his Chicago Bears, who have long featured star running backs throughout their history, need to “get off the bus running.” In 2019, Chicago media and fans blasted coach Matt Nagy for not running the ball more during a disappointing 8-8 season. All this even after the data shows teams are still running too much on first downs and studies by footballoutsiders.com and Seahawks Advanced Stats writer Sean Clement, who would become an analyst for the Baltimore Ravens, revealed that “establishing the run” early in games doesn’t actually “open up” the passing game later on.

Over the past five years, passing on first downs have averaged an eye-opening 7.6 yards per attempt while rushing plays in the same situations gain 4.3 yards. And 30.4% of pass attempts on first down have ended up moving the chains, but only 12.8% of running plays have picked up another first down. Still, NFL teams ran the ball on 50.3% of their first-down plays in 2020 and passed the ball on only 49.7%.

So when should teams run the ball? Well, the data is begging them to do it when it’s fourth-and-2 or less to gain. In those situations, the league has converted 63.3% of its attempts since 2012 – 67.7% on run plays and 56.2% on pass plays. Despite the high success rate, teams have opted to punt or try a field goal 63.1% of the time during that span.

Fourth–and-2 or Fewer Conversion Rates – Since 2012

So the only thing that kept Bill Belichick’s famous call to go for it on fourth-and-2 at the Patriots’ own 28-yard line against Peyton Manning and the Indianapolis Colts in 2009 from being a sound data-driven decision is that he opted to pass instead of run.

Data analysts have compared this decision to the one baseball coaches have faced with bunting, because in that sport many teams have learned that they shouldn’t give up one of only 27 outs they have to work within a game. Likewise, the data says NFL teams should do everything they can to keep possession of the football.

“Every one of those (attempts was a) clear analytical decision to go for two. I could just tell you analytically, like if you look at the numbers, it’s not even close.” – Ravens coach John Harbaugh

The same could be said about 2-point tries as the NFL’s conversation rate over the past three seasons has been 49.1% (183 for 373) for an expected value of .982 points. Over that same span, kickers have made 93.7% of their extra-point attempts for an expected value of .937. Still, teams chose to kick the extra point 91.1% of the time from 2018-20.

Just because it doesn’t work out doesn’t mean it’s wrong. Ravens coach John Harbaugh used the data to defend his decisions after his team went 3 for 4 on fourth downs but failed on three 2-point tries in a 33-28 loss to the Kansas City Chiefs on Sep. 22, 2019.

“The point was to score as many points as we could,” Harbaugh told ESPN. “Every one of those (attempts was a) clear analytical decision to go for two.”

“I could just tell you analytically, like if you look at the numbers, it’s not even close,” he added. “So you understand in terms of the percentage of chances to win the game, I’m just telling you that’s what the analytics say. That’s what it says. That’s how it works.”

Ravens coach John Harbaugh used the data to defend his decisions after a loss to the Chiefs on Sep. 22, 2019. (AP)

Some teams are still trying to figure out how the NFL’s player-tracking data works after only having access to Next Gen Stats (NGS) over the past three seasons. NGS, which first became a league initiative in 2014, allows forward-thinking franchises to build models to analyze plays and players differently.

The league itself devised the service in collaboration with Zebra Technologies to collect data from nickel-sized tracking devices on the ball and in every player’s shoulder pads. It can reveal information such as the path and speed of a pass, a player’s acceleration, how tight a window a quarterback was comfortable throwing into and which receiver gained the most separation on which routes in which situations.

NGS has been integrated into game broadcasts and NFL.com, and it provided media and fans an immensely popular note in February by revealing that Chiefs quarterback Patrick Mahomes had run for a total of 497 yards before throwing the ball or being sacked in the Chiefs’ Super Bowl 55 loss to the Tampa Bay Buccaneers.

The league also created the Next Gen Stats Draft Model to help teams in the scouting process and determine how a player projects to the NFL based on his athleticism, production and size profile.

“It’s a hand-and-glove relationship. Film is always a part of it. … But to ignore the power of the information and data available would be for time to pass you by.” – Zebra Technologies Vice President John Pollard

“It’s a hand-and-glove relationship,” Zebra Technologies Vice President John Pollard, a former general manager of sports solutions at Stats Perform, told The Washington Post. “Film is always a part of it. … But to ignore the power of the information and data available would be for time to pass you by.”

Despite some remaining resistance from old-school contemporaries, the thirst for knowledge in football is as excessive as any other sport and the desire to get the most wins per dollar is just as high. With a vast expanse of information still undiscovered, the smart teams will continue to leave no rock unturned as they push the limits on how far the data can take them.

“In every field there’s way too much, ‘This is the way we’ve always done it,’ and in the NFL that’s particularly extreme,” Banner said. “Now comes the major shift.”

Enjoy this? Subscribe to The Analyst to receive five stories each Friday from Stats Perform. It’s free.