Date: Thu, 30 Nov 2000 04:34:55 -0600 From: Mark Hopkins Subject: Why All Rating & Polling Methods Are Wrong A little incisive commentary for your enjoyment. And a challenge (which I believe is an impossible challenge). CONJECTURE: All rating & polling methods, no matter how conceived (be they defined by human voting, checking animal entrails or star charts, coin tosses, mathematical formula, computational algorithm or otherwise) will ALL fail in one of the following three ways: (a) The "Mount Union" effect. Exaggerated ranking of undefeated/untied teams. Particularly, Division III Mount Union being placed in the midsts of Division I teams in recent years; but also cases like 1956 Oklahoma being ranked #1 even though it played only 1 (nowhere-near-contender) winning team in 1956! (b) The "Tennessee" effect. Exaggeration of schedule strength. Particular, in 1997, the SEC was so completely dominant over everyone else, as to be virtually an entire division onto itself. A rating method which puts too much emphasis on schedule strength will place Tennessee at #1 even over Nebraska. Nebraska won its bowl game against Tennessee. A similar thing happened in 1919 with the Western Conference. or (c) It will be at odds with the Won-Loss percentage, even in round-robins where the "Pct." is the nearly universal standard definition of "Best" and (more importantly) in tight-enough schedules where its use is equally universal (like conferences). More generally and more to the point: (c') It will be at odds with the Won-Loss Percentage (even taking into account opposition strength) in not-so-loose schdules (like Division I-A Overall). There is simply no rating method which can avoid all 3 failures.