Academic research hints Chart is flawed, trading down wise, best value high 2nd round

[Awsi Dooger]

Massey and Thaler significantly updated their 2005 research a week or two ago, with the release of a new paper. I've seen sampling it for the past few days but haven't seen it mentioned here, or on other draft-oriented sites:

"The Loser's Curse: Overconfidence vs. Market Efficiency in the National Football League Draft"

Obviously it's vastly different than typical fare, and relies on some lazy/questionable assumptions, i.e. the value and validity of Pro Bowl status. It's also salary cap minded, which takes a detour this year. But overall it's a ton of work and leads to many conclusions that I agree with, notably that the famous Chart is flawed (too much weight at the top), trading down is wise, and that late 1st/early 2nd round picks hold the most value. Similarly, the previous paper argued that slots 25-50 held the highest return on investment.

The paper can be downloaded at the top of this link: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=697121

Highlights:

* "More central to the thrust of this paper is the fact that while both performance and compensation decline with draft order, compensation declines more steeply. Consequently, surplus value increases at the top of the order, rising to its maximum of approximately $1,000,000 near the beginning of the second round before declining through the rest of the draft. That treasured first pick in the draft is, according to this analysis, actually the least valuable pick in the first round!"

* "We analyze 12,754 potential trades over the 14-year period and find overwhelming evidence that a team would do better in the draft by trading down. The average gain from trading down is 3.87 starts per season. As shown in Figure 10, we estimate this gain to be positive for all 32 draft-pick positions in the first round.

* "The Chart is an example of an especially interesting type of social phenomenon in which some biases or wrong beliefs lead to perceived consensus or conventional wisdom which then becomes a norm. ... The early trades on which the original Chart was based were priced according to the intuitions of team decision-makers. As we have argued, we have reason to expect these intuitions are overconfident. ...Hence, The Chart appears both a symptom of biased judgment and also a self-perpetuating cause."

* "The discrepancy between the market prices implied by The Chart and values we estimate suggest that teams who successfully exploit this difference can substantially improve their on-field performance."

***

I'll throw in this related link from 3 years ago, a study I remembered, based on the first Massey-Thaler paper: http://www.pro-football-reference.com/blog/?p=276

Pro Football Reference.com looked at every draft from 1993-2007 and concluded that the 1997 Dolphins were in better theoretical draft position than any team during that span:

"For example, the team with the best overall collection of slots was the 1997 Dolphins; they had 14 total picks, including four in the third round. By my accounting, the value of the Dolphins draft was 170.6 on a scale where league average is necessarily 100. To be more precise, I took the total Massey-Thaler value of all picks in the draft, then I divided that by the number of teams in the league at the time. Then I divided the total Massey-Thaler value of the Dolphins picks by the league average. Finally, I multiplied it by 100 to make it easier on the eyes. To summarize: 170.6 means that they had 70.6% more draft value than an average team."

***

Now, we had some busts in 1997, but it was the Jason Taylor/Sam Madison draft, which makes it among the most successful in decades.

There's great value in doing the right thing. That's the point of this thread. Picking very high to medium high in the first round combined with stockpiling late rounders is hardly the correct approach yet we seem to be stuck in that rut. The overflow needs to be in the late 1st, 2nd and 3rd round area, similar to New England's approach. In fact, the revised Massey-Thaler paper singles out New England several times in the late going. There's also a segment arguing that the standard practice of trading next year's pick at a one round discount for this year's pick, another Patriot specialty, is immensely favorable to the patient team.

 

[Mr. Magoo]

Very interesting stuff.

My contention would be that in a salary cap less environment picks would reflect something more akin to their absolute value, though, since as the research and common sense indicates performance declines on average with draft position. Essentially, the draft trade chart might actually be more relevant this year than it's been in the past few years, since the urge to get the maximum amount of performance from every dollar isn't squeezing you from a rules standpoint, only a financially solvent one.

Another thought: the commonly accepted idea that future picks are worth a round lower than they would be this year. E.g., Brandon Marshall trade functionally being for a two and a three this year.

I get why that's accepted, since FOs are understandably worried about getting fired every year. But this works out to be a huge advantage to regimes that aren't worried about job security. Ideally you'd like to be in a position from a depth perspective not to have dire needs kicking you in the face every offseason, but the long term benefits to sucking it up for one year have shown up dramatically for teams like New England, San Diego and Philadelphia. The smart FO's know that over the long view, a second round pick is worth a second round pick, so if you can trade this year's three for next year's two, you do it, no question.

The only silver lining to New England's smart wheeling and dealing is that they really haven't had a great draft in a few years, despite the great positioning. They really need a great draft this year, and I know I speak for all of us when I say I hope they don't have one.

 

[Elliott 1]

The chart is a friggin joke. It's for sites like this and video games.

Real teams do their own assessments and make their own decisions based on their own research. They don't give a flip about anybodys chart.

 

[Awsi Dooger]

The chart is a friggin joke. It's for sites like this and video games.

Real teams do their own assessments and make their own decisions based on their own research. They don't give a flip about anybodys chart.

The flimsy origin of the Chart is amazing. Apparently it was based on only a handful of years data, and never intended to become a rigid guideline.

This from the research paper:

"The Chart was originally estimated in 1991 by Mike McCoy, then a part-owner of the Dallas Cowboys. He was kind enough to provide us with a brief history of the chart (McCoy, 2006). An engineer, McCoy estimated the values from a subset of the trades that occurred from 1987 to 1990. His goal was merely to characterize past trading behavior rather than to determine what the picks should be worth. The Chart then made its way through the league as personnel moved from the Cowboys to other teams, taking The Chart with them."

***

An interesting implied point from the paper is teams get overly sure about their player evaluation, and eager to trade up, due to so much time invested in evaluation: "Another robust finding in psychology, similar in spirit to the aforementioned tendency to make excessively extreme forecasts, is that people are overconfident in their judgments(Alpert & Raiffa, 1982). Furthermore, overconfidence is exacerbated by information—the more information experts have, the more overconfident they become. NFL teams face a related challenge – making judgments about players while accumulating increasing amounts of information about them as the draft approaches."

When I was a kid, the draft was held not long after the end of the season, in late January or early February. I don't remember many trades at all. It was a scramble to prepare and the conventional wisdom was the strong terms perpetually benefited because they were simply more equipped to do everything well. Moving the draft back to April was designed to level the playing field, giving the weak franchises opportunity to catch up.