Reviewing the Rankings Challenge: A Recap and a discussion of Poll of Polls

The Tournament came to a close tonight with a surprising winner in Connecticut, so it's time to recap the Rankings Challenge and determine a winner.

First, I'll quickly run through the computers' Final Four projections:

Only ESPN BPI projected a Kentucky victory over Wisconsin.
All computers incorrectly predicted Florida would handle UCONN.
Only myself, Massey & Silver predicted that UCONN would defeat Kentucky tonight.

The final standings:

Silver (Champion)
KenPom -4
Stanford - 4
Massey - 4
Trevor - 5
Myself - 6
ESPN BPI - 6
Sagarin -7

At least I didn't finish last.

The finish raises an interesting question about Silver's method.  Was he really that much better?  In a way, his dominance comes with an asterisk.  Due to the number of permutations, I hadn't gone through and analyzed hypothetical match-ups on Silver's site; as each round came up, I looked at who Silver had winning each game (except for his initial Final 4 & Championship projection).  What I initially failed to take into account is that Silver's bracket updated with new information; something that the other 7 computers did not have.  Essentially, Silver's projection had a head start on the rest of the computers by the end.

Silver's blog (the FiveThirtyEight) has since released an archive of their various projections.

I went back and cross-checked their initial projections against what their up-to-date projection had and they flip-flopped on four picks (Iowa State-UConn, Michigan-Kentucky, Uconn-Kentucky, Dayton-Stanford) to the correct pick.  If you were to use their initial projection, the new standings look quite a bit differently:

Silver (Co-Champion)
KenPom (Co-Champion)
Stanford (Co-Champion)
Massey - (Co-Champion)
Trevor -1
Myself -2
ESPN BPI -2
Sagarin -3

Using this adjustment, Silver is effectively co-champion as opposed to outright champion and no computer embarrassed itself (unlike the RPI last year).

As the FiveThirtyEight notes in their initial preview, the projection's core methodology is centered around other projections.   For kicks, I stripped Silver's projections from the rest and built a poll of the remaining 7 computers and what I found was fascinating (to me at least).  The poll of the 7 computers would have also been a co-champion of the contest.  This seems to confirm that whenever possible, multiple (and even diverging) points of views and projections should be used.

This was a fun exercise and I expect that I will repeat it next year.