Do I really teach at America’s best liberal arts college? Absolutely, according to the annual college ranking system published by the U.S. News and World Report, which assesses academic quality by looking at such factors as class size, graduation rate, and student SAT scores. For those of us on whom the list smiles, it seems to have the finely calibrated authority of an astronomical instrument. For those further down the list, it seems on the order of goat entrails, or something even less innocent. To be sure, a slight change of position on the list—especially one into or out of the top ten—can have dire consequences for student applications, institutional morale, and even the job security of administrators.
Now a revolt against the ranking system is in full swing. Last week, a meeting of college presidents and administrators in Annapolis discussed a boycott of the questionnaire the magazine uses to compile its annual ranking. Although a total boycott was rejected, most colleges represented at the meeting pledged that they will cease cooperating with the most controversial aspect of the magazine’s ranking, its “peer assessment score.” Whether this will make the ranking a better or worse proxy of academic quality remains to be seen.
The revolt was set in motion earlier this year when Sarah Lawrence (currently ranked 45) decided it would no longer use the SAT as a basis for admission. Many schools do not require SAT scores but Sarah Lawrence was the first to refuse to even look at them. Since the collective SAT is a major component of the U.S. News ranking, the magazine decided to find another way of measuring it. According to the indignant president of the college, she
was recently informed by the director of data research at U.S. News, the person at the magazine who has a lot to say about how the rankings are computed, that absent students’ SAT scores, the magazine will calculate the college’s ranking by assuming an arbitrary average SAT score of one standard deviation (roughly 200 points) below the average score of our peer group.
“In other words,” as she put it, “in the absence of real data, they will make up a number.” It was precisely this sort of high-handedness that has led colleges to consider pulling out of the U.S. News survey entirely, as Reed College has. Now at Annapolis, they have chosen to boycott the “beauty contest” aspect of evaluation, the peer-assessment of reputation that tends to be a self-perpetuating phenomenon.
Meanwhile, the beleaguered news magazine has responded, arguing that peer assessment is “by nature subjective, but the technique of asking industry leaders to rate their competitors is a commonly accepted practice,” which measures the “‘intangibles’ of a college that we can’t measure through statistical data.”
The stakes are high here, and it is not easy to separate self-interest from principle—on either side. Expect this story to play out over the coming academic year.