After doing a whole lot more data collection and statistical analysis over the past few days, I've decided to alter a bit the plan for the "sequel" to this thread:
http://www.finheaven.com/forums/sho...hill-amp-Weeden-An-Objective-Analysis-(Part-I)
There will still be a comparison here among the five rookie QBs from this year, as I originally intended, but after looking at the data rather extensively, what I decided to focus on much more instead is the statistical probability that Ryan Tannehill will become a franchise QB during his career in the NFL, since that might just be a concern for some of us. :)
The idea here first was to develop a criterion variable: to identify which of the quarterbacks who started a significant number of games as rookies have gone on to become franchise QBs.
To do that, I used career QB rating as the crtierion measure. In other words, I focused on career QB ratings as a means of determining whether the QBs in the analysis have attained the "franchise" level during their careers. Here is the data set that was used for this analysis and all the other statistical analyses I'm going to cover later in this post:
The career QB ratings for the QBs who are in only their second years in the league were their QB ratings during their second season only, since using a "career" rating that involved their rookie season would confound the analysis.
What we see here is that the median career QB rating in the sample is 79.8 (the average was 77.9). Therefore every QB with a career QB rating of 79.8 or higher has been deemed a franchise QB in the sample. As you can see, that fairly well maps on to my sense of what people would generally believe, with Josh Freeman being the lowest man on the totem pole in that regard, and Ben Roethlisberger and Matt Ryan being at the top.
Interestingly, Kyle Orton and Alex Smith were barely below the "franchise" level, and both became backups this season. So not only does the criterion variable (career QB rating) map on to my sense of the general consensus of how these QBs are viewed; it also distinguishes starters from backups very well, even backups who were recently starters and who played well in the recent past (Smith and Orton).
Therefore I would say the criterion variable has what we call "construct validity." In other words, in my opinion it validly represents the concept of "franchise QB," at least for the purpose of the forthcoming statistical analyses.
So, now we have a whole lot of other variables (above) we can use to determine whether Ryan Tannehill may someday find his place in that category, as well.
What we find first statistically (using one-way ANOVAs) is that rookie QBs who later became franchise QBs differed significantly from those who did not with regard to only three of the above variables: 1) rookie QB rating, 2) pass attempts per TD pass, and 3) WPA (explained below).
The first stands to reason, since past behavior (rookie QB rating) is often a predictor of future behavior (career QB rating). The second (pass attempts per TD pass) stands to reason, as well, since TD passes are an element of QB rating.
The third, however -- WPA -- requires some explanation. "WPA," or "win probability added" is taken from a website called Advanced NFL Stats, and refers to the following:
Now, what we see here with some additional statistical analysis is that the correlation between rookie QB rating and career QB rating is 0.58 and is statistically significant at a p-value of 0.006.
So, you ask, perhaps, why not just use rookie QB rating to determine how likely a rookie QB is to become a franchise QB?
Two reasons: 1) the correlation between WPA and career QB rating is much stronger -- 0.75, with a p-value less than 0.0001, and 2) when we correlate rookie QB rating with career QB rating while partialling out the variance in both associated with rookie WPA, the correlation between rookie QB rating and QB rating drops from 0.58 to a lowly 0.19, which is not statistically significant.
In other words, the way a rookie QB plays in the clutch (i.e, his WPA) is a much stronger predictor of his later designation as a franchise QB than his rookie QB rating alone. What the partial correlation above points out is that there have been a decent number of QBs whose rookie QB ratings were low and whose career QB ratings are high, because their rookie WPA was relatively good (and vice-versa). Said differently, their play in the clutch as rookies was a better predictor of their later success than their overall QB ratings as rookies.
So now that we have an adequate predictor of "franchise QB" status based on the analyses above, let's take a look at how Ryan Tannehill did this year in that regard.
Ryan Tannehill's WPA this year was 1.36, which may seem meager at first glance when you compare it to the WPA numbers of RGIII, Russell Wilson, Matt Ryan, and Ben Roethlisberger, for example.
However, when you analyze it statistically, you find that the average "franchise QB" in the sample above had a rookie WPA of only 0.59, while the average "non-franchise QB" in the sample had a rookie WPA of -1.16 (yes that's negative 1.16).
We already know Ryan Tannehill's rookie WPA places him comfortably in the "franchise QB" category, but we can go one step further. We can see if his rookie WPA was significantly higher than that of the average rookie WPA of the "franchise QBs" in the sample.
The answer is not quite. The standard deviation of the rookie WPAs among the "franchise QBs" is 1.44, which means Ryan Tannehill's WPA of 1.36 isn't quite a standard deviation above the mean.
However, I think the general finding here, that Ryan Tannehill showed the kind of clutch play this year that suggested he has "the tools" to become a franchise QB, supports the perceptions of many here in the forum.
To provide a bit of additional context, Chad Henne's WPA in 2009 (his first season as a starter) was 0.90, which would've placed him, also, in the "future franchise QB" category, though not with the degree of confidence provided by Ryan Tannehill's WPA of 1.36. Of course Chad Henne was not a rookie that year, either, and he'd had much more experience at the position than Tannehill.
Also, and here's the clincher: Chad Henne's WPA the following season (2010) was -1.28.
So anyway, this is all just for fun and for our information. Hope you found it enjoyable. :up:
http://www.finheaven.com/forums/sho...hill-amp-Weeden-An-Objective-Analysis-(Part-I)
There will still be a comparison here among the five rookie QBs from this year, as I originally intended, but after looking at the data rather extensively, what I decided to focus on much more instead is the statistical probability that Ryan Tannehill will become a franchise QB during his career in the NFL, since that might just be a concern for some of us. :)
The idea here first was to develop a criterion variable: to identify which of the quarterbacks who started a significant number of games as rookies have gone on to become franchise QBs.
To do that, I used career QB rating as the crtierion measure. In other words, I focused on career QB ratings as a means of determining whether the QBs in the analysis have attained the "franchise" level during their careers. Here is the data set that was used for this analysis and all the other statistical analyses I'm going to cover later in this post:
QB | Rookie Starts | 1st Round? | Franchise QB? | Career QB Rating | Rookie QB Rating | WPA | Total Attempts | Attempts Per Start | Attempts Per TD | Attempts Per INT | Defensive Rank (Points) | Rushing Rank (Yards) | Deep Ball %age |
Tannehill | 15 | Yes | 76.1 | 1.36 | 479 | 31.93 | 39.92 | 36.85 | 7 | 17 | 8.77 | ||
Luck | 16 | Yes | 76.5 | 4.5 | 627 | 39.19 | 27.26 | 34.83 | 21 | 22 | 13.24 | ||
Wilson | 16 | No | 100 | 3.03 | 393 | 24.56 | 15.12 | 39.3 | 1 | 3 | 14.25 | ||
RGIII | 15 | Yes | 102.4 | 3.1 | 393 | 26.2 | 19.65 | 78.6 | 22 | 1 | 8.4 | ||
Weeden | 15 | Yes | 72.6 | -0.47 | 517 | 34.47 | 36.93 | 30.41 | 19 | 24 | 10.06 | ||
Newton | 16 | Yes | Yes | 86.2 | 84.5 | 2.05 | 517 | 32.31 | 24.62 | 30.41 | 27 | 3 | 15.47 |
Dalton | 16 | Yes | Yes | 87.4 | 80.4 | -0.13 | 516 | 32.25 | 25.8 | 39.69 | 9 | 19 | 10.85 |
Bradford | 16 | Yes | Yes | 82.6 | 76.5 | 0.09 | 590 | 36.88 | 32.78 | 39.33 | 12 | 25 | 6.61 |
Palmer | 13 | Yes | Yes | 86.2 | 77.3 | 0.06 | 432 | 33.23 | 24 | 24 | 21 | 17 | 10.88 |
Roethlisberger | 13 | Yes | Yes | 92.7 | 98.1 | 2.12 | 295 | 22.69 | 17.35 | 26.82 | 1 | 2 | 14.92 |
Ryan | 16 | Yes | Yes | 90.9 | 87.7 | 3.75 | 434 | 27.13 | 27.13 | 39.45 | 11 | 2 | 12.67 |
Flacco | 16 | Yes | Yes | 86.3 | 80.3 | -0.13 | 428 | 26.75 | 30.57 | 35.67 | 3 | 4 | 11.45 |
Ponder | 10 | Yes | Yes | 81.2 | 70.1 | -0.61 | 291 | 29.1 | 22.38 | 22.38 | 31 | 4 | 12.71 |
Stafford | 10 | Yes | Yes | 82.8 | 61 | 0.45 | 377 | 37.7 | 29 | 18.85 | 32 | 24 | 10.34 |
Gabbert | 14 | Yes | No | 70.2 | 65.4 | -2.21 | 413 | 29.5 | 34.42 | 37.55 | 11 | 12 | 6.78 |
Sanchez | 15 | Yes | No | 71.7 | 63 | -1.02 | 364 | 24.27 | 30.33 | 18.2 | 1 | 1 | 10.71 |
Young | 13 | Yes | No | 74.4 | 66.7 | 0.96 | 357 | 27.46 | 29.75 | 27.46 | 31 | 5 | 14.29 |
Leinart | 11 | Yes | No | 70.2 | 74 | 0.91 | 377 | 34.27 | 34.27 | 31.42 | 29 | 30 | 10.08 |
Freeman | 9 | Yes | Yes | 79.8 | 59.8 | -1.08 | 290 | 32.22 | 29 | 29 | 27 | 23 | 11.72 |
Gradkowski | 11 | No | No | 65.8 | 65.9 | -1.3 | 328 | 29.82 | 36.44 | 36.44 | 21 | 28 | 10.98 |
Orton | 15 | No | No | 79.7 | 59.7 | -2.66 | 368 | 24.53 | 40.89 | 28.31 | 1 | 8 | 11.96 |
Edwards | 9 | No | No | 75.5 | 70.4 | -0.42 | 269 | 29.89 | 38.43 | 33.63 | 18 | 15 | 6.69 |
Clausen | 10 | No | No | 58.4 | 58.4 | -1.83 | 299 | 29.9 | 99.67 | 33.22 | 26 | 13 | 7.02 |
Walter | 8 | No | No | 52.6 | 55.8 | -2.49 | 276 | 34.5 | 92 | 21.23 | 18 | 29 | 15.94 |
Smith | 7 | Yes | No | 79.1 | 40.8 | -1.57 | 165 | 23.57 | 165 | 15 | 30 | 17 | 13.94 |
EManning | 7 | Yes | Yes | 82.7 | 55.4 | -0.05 | 197 | 28.14 | 32.83 | 21.89 | 17 | 11 | 9.64 |
The career QB ratings for the QBs who are in only their second years in the league were their QB ratings during their second season only, since using a "career" rating that involved their rookie season would confound the analysis.
What we see here is that the median career QB rating in the sample is 79.8 (the average was 77.9). Therefore every QB with a career QB rating of 79.8 or higher has been deemed a franchise QB in the sample. As you can see, that fairly well maps on to my sense of what people would generally believe, with Josh Freeman being the lowest man on the totem pole in that regard, and Ben Roethlisberger and Matt Ryan being at the top.
Interestingly, Kyle Orton and Alex Smith were barely below the "franchise" level, and both became backups this season. So not only does the criterion variable (career QB rating) map on to my sense of the general consensus of how these QBs are viewed; it also distinguishes starters from backups very well, even backups who were recently starters and who played well in the recent past (Smith and Orton).
Therefore I would say the criterion variable has what we call "construct validity." In other words, in my opinion it validly represents the concept of "franchise QB," at least for the purpose of the forthcoming statistical analyses.
So, now we have a whole lot of other variables (above) we can use to determine whether Ryan Tannehill may someday find his place in that category, as well.
What we find first statistically (using one-way ANOVAs) is that rookie QBs who later became franchise QBs differed significantly from those who did not with regard to only three of the above variables: 1) rookie QB rating, 2) pass attempts per TD pass, and 3) WPA (explained below).
The first stands to reason, since past behavior (rookie QB rating) is often a predictor of future behavior (career QB rating). The second (pass attempts per TD pass) stands to reason, as well, since TD passes are an element of QB rating.
The third, however -- WPA -- requires some explanation. "WPA," or "win probability added" is taken from a website called Advanced NFL Stats, and refers to the following:
Advanced NFL Stats said:Although it's been used in baseball sabermetrics for several years now, Win Probability Added (WPA) is a new, or at least rediscovered, concept for football stats. It measures each play in terms of how much it increased or decreased a team’s chances of winning the game.
WPA starts with a Win Probability (WP) model of the game of football. Every situation in a game gives each opponent a particular chance of winning, and a WP model estimates those chances. The model created here at Advanced NFL Stats uses score, time, down, distance, and field position to estimate how likely each team will go on to win the game. For example, at the start of the 2nd quarter, a team down by 7 points with a 2nd down and 5 from their own 25 will win about 36% of the time -- in other words a 0.36 WP.
On that 2nd down and 5, let’s say there is a 30-yard pass, setting up a 1st down and 10 on the opponent’s 45. Now that team has gone from a 0.36 to a 0.39 WP. The WPA for that play would be +0.03.
If instead the quarterback throws an interception returned back to the line of scrimmage, the opponent now has the ball at the 25, giving the trailing team a 0.28 WP. The WPA for the interception would be -0.08.
WPA is very sensitive to the context of the game. That same interception that cost -0.08 when a team was down by 7 points in the 2nd quarter would cost much more if it the offense was leading by a point late in the 4th quarter. Putting your opponent in immediate field goal range would be nearly fatal.
Stats are tools, and each tool has its own purpose. WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing.
WPA has a number of applications. For starters, we can tell which plays were truly critical in each game. From a fan's perspective, we can call a play the 'play of the week' or the 'play of the year.' And although we still can't separate an individual player's performance from that of his teammates', we add up the total WPA for plays in which individual players took part. This can help us see who really made the difference when it matters most. It can help tell us who is, or at least appears to be, "clutch." It can also help inform us who really deserves the player of the week award, the selection to the Pro Bowl, or even induction into the Hall of Fame.
As interesting as that might be to us as fans, WPA might be even more useful to coaches and strategists. We can measure whether a kicker’s high field goal accuracy was worth a trade-off in short kickoffs. And for the first time, we can measure the effectiveness of clock management strategies — is it better to run three times and punt, or pass and go for the first down to end the game?
Even better, WPA can measure the risk-reward balance of a given situation. For example, when is it best to take a sack, and when is it best to take your chances throwing into traffic? Or for defenses, when is it best to roll the dice on a big blitz and risk a long completion due to a big hole in the secondary? We can measure which interceptions were the most costly, and which quarterback tends to throw the costliest ones. WPA may actually be able to measure decision-making ability on the field and on the sideline.
WPA is not 'the one stat to rule them all,' and it does have its disadvantages. It can certainly be improved in several respects, and I’m already working on several ideas. As time goes on, we’ll find even more new and interesting applications for it, and this is only the beginning. More than any other sport, football is about strategy, risk, and reward, and WPA is well suited to capturing much of what makes football so compelling.
Now, what we see here with some additional statistical analysis is that the correlation between rookie QB rating and career QB rating is 0.58 and is statistically significant at a p-value of 0.006.
So, you ask, perhaps, why not just use rookie QB rating to determine how likely a rookie QB is to become a franchise QB?
Two reasons: 1) the correlation between WPA and career QB rating is much stronger -- 0.75, with a p-value less than 0.0001, and 2) when we correlate rookie QB rating with career QB rating while partialling out the variance in both associated with rookie WPA, the correlation between rookie QB rating and QB rating drops from 0.58 to a lowly 0.19, which is not statistically significant.
In other words, the way a rookie QB plays in the clutch (i.e, his WPA) is a much stronger predictor of his later designation as a franchise QB than his rookie QB rating alone. What the partial correlation above points out is that there have been a decent number of QBs whose rookie QB ratings were low and whose career QB ratings are high, because their rookie WPA was relatively good (and vice-versa). Said differently, their play in the clutch as rookies was a better predictor of their later success than their overall QB ratings as rookies.
So now that we have an adequate predictor of "franchise QB" status based on the analyses above, let's take a look at how Ryan Tannehill did this year in that regard.
Ryan Tannehill's WPA this year was 1.36, which may seem meager at first glance when you compare it to the WPA numbers of RGIII, Russell Wilson, Matt Ryan, and Ben Roethlisberger, for example.
However, when you analyze it statistically, you find that the average "franchise QB" in the sample above had a rookie WPA of only 0.59, while the average "non-franchise QB" in the sample had a rookie WPA of -1.16 (yes that's negative 1.16).
We already know Ryan Tannehill's rookie WPA places him comfortably in the "franchise QB" category, but we can go one step further. We can see if his rookie WPA was significantly higher than that of the average rookie WPA of the "franchise QBs" in the sample.
The answer is not quite. The standard deviation of the rookie WPAs among the "franchise QBs" is 1.44, which means Ryan Tannehill's WPA of 1.36 isn't quite a standard deviation above the mean.
However, I think the general finding here, that Ryan Tannehill showed the kind of clutch play this year that suggested he has "the tools" to become a franchise QB, supports the perceptions of many here in the forum.
To provide a bit of additional context, Chad Henne's WPA in 2009 (his first season as a starter) was 0.90, which would've placed him, also, in the "future franchise QB" category, though not with the degree of confidence provided by Ryan Tannehill's WPA of 1.36. Of course Chad Henne was not a rookie that year, either, and he'd had much more experience at the position than Tannehill.
Also, and here's the clincher: Chad Henne's WPA the following season (2010) was -1.28.
So anyway, this is all just for fun and for our information. Hope you found it enjoyable. :up: