How do university rankings work — and do they really matter?

University rankings are a controversial criteria for prospective students. For some, rankings are an important factor when choosing schools, while for others, they are confusing and undecipherable, or only useful with regard to program rankings. So where does the controversy stem from, and how should we really be looking at them?

At the time of publication, Maclean’s Education, one of the most prevalent Canadian ranking publications, ranks UW the third best comprehensive university in Canada, the second best Canadian university overall, and the number one most innovative university in Canada.

According to data from the Canadian University Survey Consortium, in 2022, 21 per cent of first-year students considered Maclean’s university rankings an important source of information when choosing which university to attend.

“Ranking definitely played a role in m[e] choosing Waterloo,” said Gabrielle LaRosa, a first-year science student at UW. “I wanted to ensure . . . that I was getting the best education I can with the choices I have.”

In light of the influence rankings can have on recruitment, it becomes clear why UW has an entire section dedicated to rankings on its website. The school makes its standings as first-place in innovation and second-place in best overall Canadian university clear to students, and also highlights its programs’ spots in other rankings including US News Global University Rankings (second in engineering and nanoscience) and QS World University Rankings (second in Canada for electrical engineering and fourth for various programs including geography and environmental sciences).

Despite these titles, rankings can still create confusion due to the multitude of factors that go into the calculations. First-year Waterloo students Hailey Schmidt and Anika Nickel noted that university rankings can be complicated to decipher, as there are “a lot of factors” that go into creating these kinds of lists.

Annual university rankings published by Maclean’s Education take into account 12 performance indicators, including the ratio of full-time students to full-time faculty members, the number of scholarships and awards students have won in the previous five years, the number and value of research grants awarded to faculty over the past year, the percentage of the university’s operating budget spent on student services, scholarships, and bursaries, as well as the overall reputation of the university based on online surveys responses from faculty and senior administrators at Canadian universities and business people across Canada.

Changbao Wu, chair of the department of statistics and actuarial science at UW, emphasized the importance of determining indicators within each university ranking. For example, students should determine what it means when a school ranks highly in research: “Is [it] the total number of publications? Is [it the] average number of publications per faculty? There are different measures,” he said.

Wu also explained that individual programs often rank differently than the university as a whole. “From a student perspective, it is always more important to pay attention to [the] particular subject area you’re interested in, rather than the overall [university] ranking,” he noted.

Ema Masic, a first-year student at UW, echoed his sentiments. “The university [as a whole] doesn’t really matter,” she said, explaining that she wanted to focus on schools’ specific program rankings instead.

Other critiques of university ranking stem from issues with methodology used by the companies that publish the rankings. Elite schools in the United States, including Harvard Law, Johns Hopkins University School of Medicine, and Yale Law have decided to pull out of university rankings systems all together over the last few years due to said criticism. Critics say these rankings encourage universities to sacrifice educational quality and diversity among the student body and to instead prioritize exclusivity in an effort to strengthen the school’s rankings. Coupled with the significance placed on rankings, universities have an incentive to cheat by cherry picking the statistics they share with the media.

This was illustrated most recently by the case of Columbia University in February 2022, where mathematics professor Michael Thaddeus found that Columbia had submitted misleading numbers to the US News rankings, for example, inflating the amount of money spent on instruction by including the cost of patient care in the medical school. Columbia University opted not to submit information for that year’s ranking. Independent research performed by US News for the rankings saw Columbia fall 16 places, from second to 18th.

In an interview with Beyond the Bulletin, UW president and vice-chancellor Vivek Goel stated that trying to elevate Waterloo’s rankings in publications like the Times Higher Education rankings would require adhering to what some would say “metrics that link back to what we define as an excellent university 100 years ago”.

Though UW continues to promote its rankings in its marketing material and on its websites, Nick Manning, associate vice-president of communications, said that this is because rankings can still be helpful particularly to international students choosing which school to attend abroad, and that the university does not use strategies to purposefully increase its rankings.

“The real issue is the absolute view of positions,” Wu explained, noting that the actual differences between scores used for rankings are often “not distinguishable.” However, this gets lost when schools are ranked numerically: “If you try to read [rankings] in the absolute sense, it says [university number] three is better than four. That’s misleading, and we should never mislead students [with] this kind of absolute positioning of universities.”

Manning further clarified Goel’s comments on not following outdated metrics, stating, “What he means is we’re not building strategies, so that we get into the top 10 or top 50 or top 100. What we’re doing is we’re continuing to deliver against our [benchmarks] … to be successful in the ways that we define it.”

With files from Alicia Wang