College Football Playoff: Lessons learned in judging teams as a member of the mock selection committee

GRAPEVINE, Texas — I was a student journalist at Baylor in 2014 when the final chart on ESPN moved to #4 in Ohio State. The Bears, the infamous co-champions of the Big 12 with the TCU, found themselves sitting outside of their first College Football Playoff. Eight years later, to my great disappointment, the College Football Playoff mock selection committee did the same.

But we’ll come to that.

Twelve media members, myself included, were invited to participate in the annual mock election held by the College Football Playoff. The exercise takes place behind the doors of the same opulent boardroom at Gaylord Texan as the actual committee that holds the destinies of countless athletic departments.

That day, I played the role of Chet Gladchuk, Navy athletic director and current CFP committee member. “Be a little rude to play the part of Chet,” the 72-year-old CFP employee I was sitting in joked. But if what they wanted was vulgarity, a deep-throated rewrite of the controversial 2014 playoff was the perfect place to start.

The difficulty of ranking the four “best” teams

The process of ranking teams has been extensively reported, but here’s a brief rundown: The standings are divided into seven groups – 1-9 in all three spots and 10-25 in all four. For each trio, we vote to evaluate six teams at once. First group: Alabama, State of Ohio, Oregon, State of Florida, Baylor and TCU. As it’s supposed to be.

Boo Corrigan, CFP president and NC State athletic director, tasked us with finding the 25 “best” teams in the country. But from the very beginning, it turns out that each of us has different definitions of this metric. While some value success throughout the season, others wanted the best team from Sunday’s Pick. Regardless, the resume has become a far more important consideration than which team will win on any given Saturday.

Discussing the criteria with some of the most famous names in the sports world was perhaps the part I was most unprepared for. The former players in the room were adamant that Florida State’s undefeated season was a trump card. I did not agree. Don’t tell the Pro Bowl, who beat Deuce McAllister and former first-round quarterback EJ Manuel, that winning isn’t everything because of metrics, try being a jerk like me.

CFP has an analytics system built by SportSource Analytics that can compare up to four teams simultaneously with a wide variety of metrics on giant screens in front of the group. Baylor vs. TCU vs. Ohio State vs. Everything that happened in 2014 made sense when the Florida State benchmark was screened to spot the difference between points 3 and 4.

Jeyarajah (left) and former NFL defender Kirk Morrison

Kevin Jairaj/College Football Playoffs

Team schedules are displayed on the page with a list of results and common competitors or head-to-head highlighted. However, the most striking part of the page is a color gradient where teams are highlighted from green (good) to red (bad). When the programs for TCU and Baylor came along, the amount of red stood out against the rest of the field.

Never underestimate the power of data visualization!

Also, one of the only real pieces of criteria listed by the committee is to prioritize similar teams with head-to-head victories or conference championships. CFP director Bill Hancock confirmed to the group that since Baylor and TCU were presented as co-champions by the Big 12, teams should each be considered as having effectively 0.5 championships.

Compared to Ohio State and Florida State, which were clearly champions, the decision was not difficult. We could argue about the program the committee used and the specific strength of the rating criteria, but that gave us a complete picture of the committee’s decision.

Perhaps guided by Manuel’s Florida State hand in the room, ‘Noles actually climbed to #2 on our fake rankings, behind only Alabama. Florida State No. I voted 5; 12 metrics that CFP identified as most associated with winning hated FSU and its narrow margins of victory were not enough to convince me of other (better) conference champions.

Full disclosure, I tended to rely more on efficiency numbers and game quality than other voters. SportSource Analytics, neatly organized and color-coded statistics, relative scoring and leading games in offense-per-point that are most associated with winning after extensive historical research. (A special teams metric was added to the committee’s top 12 stat factors at the behest of former Nebraska coach Tom Osborne.) Other bogus committee members talked about the quality of their programs. Some leaned on the quarterback game or tried to expose their NFL prospects.

Ultimately, this diversity of ideas is the issue. Seven of the 13 CFP committee members are athletic directors. Seven are former NFL players and MIT Ph. D are former college football players, including mathematician John Urschel. Both are trainers. One, former USA Today columnist Kelly Whiteside, is a journalist. Everyone sees the game differently.

The combination of former actors, writers, and TV personalities gave our group a similarly interesting context. San Diego State legend and NFL veterinarian Kirk Morrison wanted to ensure that the Group of Five competitors got the attention they deserved. Ari Wasserman of The Athletic asked us to assess hiring vacancies. AL.com editor John Talty wanted us to remember Katy Perry was in The Grove to match Ole Miss’s Alabama. McAllister and former Michigan quarterback Devin Gardner got us thinking first and foremost about winning. I’ve raised the issue that Ole Miss was left blank 30-0 by a Bret Bielema Arkansas team that achieved a “borderline erotic” bowl win in Texas. Everyone has a role.

Boo Corrigan, NC State athletic director and CFP president

Kevin Jairaj/College Football Playoffs

How to rank the best of the rest

If you’re wondering why there might be inconsistencies in a ranking, that’s why. Each secret ballot has different voters weighing their own criteria as they try to convince other voters that their point of view is correct. And indeed, this made the 7-25 conversation much more interesting and competitive.

“We would have been very happy to be in the top 25 when we were athletic director in the Army,” Corrigan told the group. “We have to give as much importance to the bottom of the standings as to the top.”

What we do. The process required it.

Some participants tried to add the power of the conference to the picture, or they rejected some programs because they were from the Group of Five. CFP chief executive Bill Hancock and Corrigan struck them quickly. After reviewing the matches and seeing the green to red timing gradient again, it became clear that this was obviously not necessary.

The data visualization had records against over .500 teams, the committee’s previous top 25, and the previous top 10, but really there were very few of these top data points to choose from. Instead, programs filled with green caught our attention. Ranked games – or border-ranked games – were discussed on a case-by-case basis.

We removed Boise State, and some were shocked by the amount of quality competitors in the “Group of Five” conference. It moved up. Conversely, Wisconsin’s “Power Five” program was full of red. SEC contenders Ole Miss and Georgia were filled with green wins and red losses. Michigan State and Kansas State failed to win the green, but their only flaws came from the nation’s top teams.

We used to vote for new teams after each round. Finally, we had the opportunity to discuss major inequalities. UCLA initially featured behind an Arizona State team that the Bruins scolded in Tempe, Arizona. This has been fixed. Arizona went up a few points while Georgia slipped even lower. Marshall entered the field after comparing well with teams like Minnesota and Louisville on a factor of 12.

By the way, this discussion lasted more than five hours. In fact, it includes six weeks of talk, hours of discussion, and countless weekends watching clips of every relevant football game imaginable. Hancock noted that there was a committee member who decided to rate each player individually in each game he watched. It’s different every year.

What will change (and what will not)

Naturally, that leads us to the 12-team playoff. In many ways, the committee’s decisions are about to change. With six automatic bids, less emphasis will be placed on figuring out which teams are taking the field, and more emphasis on ranking and ranking the teams.

When the playoffs expand, the committee will remain at 13 members. They still plan to rank 25 teams and feel comfortable that at least six conference champions will make it to that number. Instead of filling bowl matches with next teams, these will now fill major playoff spots. Basically, the process won’t change much when the playoffs expand, which those in the room hope sooner or later.

After all, I respected the process a lot. Not perfect, not perfect, but complete. With the amount of information and video available to committee members, I feel more confident that they have all the necessary tools to make the best possible decisions. More importantly, the secret ballot process makes it a little more difficult to shuffle rankings for branding or matching purposes. Transparency won’t console any conspiracy theory, but it gives me more confidence in the process.

And to Hancock, whenever you need a new name to add to your committee members list… I’m available.

Leave a Comment

Your email address will not be published.