College Rankings Permeate Decisions in Higher Education

Newswise — While many college and university administrators object to the growing influence of rankings in higher education, new research by a University of Iowa professor shows those administrators often make decisions they hope will improve their own school's ranking.

Michael Sauder (left), assistant professor of sociology in the UI College of Liberal Arts and Sciences, found rankings in publications like U.S.News and World Report were behind decisions that affect everything from how money is spent to which students are admitted to when faculty members take leave.

"U.S.News's motivation was to make a profit and to provide consumers with a rough idea of how school X compares to school Y in a market that's more and more nationalized," Sauder said. "The magazine had no intention of changing education."

Sauder and Wendy Nelson Espeland, associate professor of sociology at Northwestern University, examined the impact of U.S.News's law school rankings. They outline the unexpected consequences of the rankings in a paper titled "Rankings and Reactivity: How Public Measures Recreate Social Worlds," published in the July issue of the American Journal of Sociology.

"The rankings seem to permeate all of the schools' decisions, to loom over everything," Sauder said. "Law school deans feel trapped. If they don't pay attention to rankings, there's a good chance their school will drop. Because students look at the rankings to decide where to go, it could have a spiraling effect, where they get worse students and drop farther, and it really will affect the quality of the school. So even though they don't agree with the rankings, they feel obligated to do as well as they can. Deans take different approaches, but almost every dean we interviewed does something to make sure their numbers are as good as they can be when they hand them in to U.S.News."

The professors conducted confidential interviews with 136 law school administrators, faculty and staff at 70 accredited law schools, with roughly equal numbers from all four tiers of the rankings. They sought people from various sizes of schools, public and private, in different geographic regions. Most interviews were conducted by phone, but a grant from the Law School Admissions Council funded trips to seven focus schools, where the professors conducted in-depth interviews. Sauder and Espeland monitored chat rooms dedicated to prospective law students for a year to see if rankings influenced school choice and interviewed students and admissions officials at a career fair. To see how rankings affect marketing, they analyzed law schools' Web sites, promotional publications and press releases, and resulting news coverage.

Reallocation of resources was one unintended consequence of the rankings. Law schools started to sink significant amounts of money into promotional magazines and brochures. The materials aren't just distributed to prospective students, but to practicing lawyers and to administrators at other law schools -- the people who participate in reputational surveys for the U.S.News rankings.

"The strategy may have worked for the first couple of schools, the innovators, but since then, everybody got so inundated with these materials, they just throw them away," Sauder said. "Everybody sees it as a waste of money, but they still do it because they're afraid not to -- just in case it has some effect."

Rankings also changed how law school staff do their jobs. Admissions officers say they have less discretion about which students they admit. They prefer to consider other characteristics along with LSAT scores but feel obligated to admit students with the highest scores because the scores count in rankings.

Career service officials now spend inordinate amounts of time tracking down law school graduates to report employment rates for the rankings. One official considered hiring a private investigator, but found it too costly. Some felt their job security depended on delivering complete employment statistics.

"Virtually everybody who graduates from law school finds a job, so the difference between, say, 99 percent employed and 98.6 percent employed could move you quite a bit in the rankings," Sauder said. "That's why they really feel pressure to track down every last graduate. They spend weeks doing this, and this is time they don't spend advising students or doing other things that are much more directly beneficial to students, like networking with law firms. They feel like their job has changed, and not in a good way."

Some law schools admitted to what the professors define as "gaming strategies" -- manipulating rules and numbers in ways that undermine the motivation behind them -- in order to maximize their rank. Some stretched the definition of "employed" to include any job, whether or not it was in the legal field. In some cases, they hired their unemployed graduates to boost their employment numbers.

The researchers discovered that schools sometimes encourage faculty to take leave in the spring, rather than the fall -- when the U.S.News data is collected -- to ensure a better student-faculty ratio. They found that some schools move students with lower LSAT scores or GPAs to part-time or night programs, since U.S.News only collects such data for full-time day programs.

"A lot of schools engage in some type of gaming strategy, but it's impossible to measure exactly how common these practices are because administrators don't like to admit to them," Sauder said. "What we can say is that almost every school is very concerned about their ranking and is taking actions to make sure they improve, or at least maintain, their ranking. And because everybody's doing this, it becomes an arms race of rankings."

Sauder hopes the research will encourage administrators to think about the unexpected consequences of the rankings.

"What does it mean that administrators are changing their behavior because of measures made by a news magazine with no expertise in education?" Sauder said. "It's ironic to have journalists and statisticians who aren't experts in education create rankings that determine how leaders in higher education think."

The research also has implications for students. Students should be aware that "there's not a huge difference between a school that ranks No. 22 versus a school that ranks No. 24," Sauder said.

"Rankings are popular because they're a simple way to convey lots of information," he said. "But don't just assume you should go to School No. 31 instead of School No. 33 because it must be better. Think about what's important to you, and use as many sources as you can -- not just rankings -- to guide your decision."