In the current globalized economy, rankings are an obsession supposed to help consumers make an informed choice. Apart from the well-founded criticisms of ‘virtual customers’ flooding some platforms with positive feedback, the sheer amount of ‘information’ available often transforms even positive choices into daunting decisions.
With the growing marketization of the UK Higher Education and the metamorphosis of students into customers, the latter need to be ‘guided’ through their choice of university. Students are now told they need to ‘invest’ in themselves rather than become educated. And what better guides are there than league tables and customer feedback? The latter is ensured by the National Student Survey (NSS) in which ‘student satisfaction’ is dominant, as if satisfaction were a good measure of education. As for the former, there is no shortage of that either. League tables have flourished ever since the first release of the Shanghai Jiaotong ranking in 2003 and there are now many different sets of rankings and criteria. Unfortunately, ranking universities is somewhat more complex than ranking tennis or chess players. Universities do not oppose each other in events with a specific outcome (win, draw or loss) and they have a series of complex missions not easily captured in scores.
How did we end up here? In part this is the result of the incorporation of the former polytechnics as Post-92 Universities, and the target of 50% of school leavers going on to higher education. The sector is bigger and universities need to work harder to differentiate themselves. A ‘market’ needs data to drive ‘customer’ choice. As with any online shopping platform, it does not matter how relevant the customer feedback review is as long as it is there. Nevertheless, students are led to believe that the 10th ranking university is better than the 15th one. One look at the Unistats website (https://unistats.ac.uk) reveals the vast amount of ‘information’ students are deluged with when trying to make an ‘informed choice’. Apart from the stress this creates for students, these rankings do other kinds of damage, as captured in the excellent Distinguished Lecture Professor Dorothy Bishop gave at the university last year about the TEF and its shortcomings.
University reputations are high-inertia beasts, very difficult to move, one way or another. Rankings on the other hand, by releasing data every year, can tempt us to establish short-term correlations between measures, strategies and rank variation. A quick look at some rankings’ time series (Figs. 1-3) mostly shows stability over the last 5 to 10 years, and it is very difficult to establish whether a year-to-year variation is significant.
Figure 1 – from Evolution of UoS rankings in the UK,
Figure 2 – from Evolution of UoS THE World University rankings
Figure 3 – from the Evolution of UoS QS World rankings
An exception is the sharp decrease in 2018 (Fig. 3), but even here the reasons are likely to be multiple and these data would need to be rigorously analysed before any serious conclusion could be drawn.
A worrying trend is how communications teams often cannot resist the temptation and use the yearly release of these rankings to spin the news. In last year’s news post, while we took a strong hit in ranking as mentioned above, the message was “Southampton’s ranking reflects its performance in the ‘pillars’ used by the THE to generate its list. Once again this year, the University’s strongest pillar is for International Outlook with a score of 92 – more than double the sector median score of 43.7. Southampton also scored well for citations at 86, some 39 points above the sector median with scores for Teaching and Research also above the median.” We might also have come third in the UK for coffee machine quality… The great thing with these rankings is that there is always something you can cherry-pick to make you look good.
But no worries, this year’s news saw the return of our glory. In this year’s financial statement, it is said (p. 3) that “In the spring/summer we received a flurry of good news, all pointing to the positive impact of our strategy and operational focus on quality: we rose 6 places in the Complete University Guide, 12 places in the Guardian University league table, 12 places in the Times Good University Guide, regained our place in the World’s Top 100 in QS and were awarded a Silver in the Teaching Excellence Framework (TEF). […]. All major achievements.” This is Baldrick (from the BBC series Black Adder) at his most cunning: ‘drop’ in the rankings, come back to near where you were and praise the university strategy for a huge leap forward.
It goes without saying, NSS can be spun too. In the same document, there is more fantastic news: “The focus and effort that we have put into improving the student experience has been positively reflected in a gain of 1% in the overall satisfaction score to 84%.” Of course, any of our students making this statement in a Statistics exam would fail to get the marks, so this is not setting a great example. The strong evidence provided by Prof. Dorothy Bishop [sharepoint site, password needed 25:00, slide 29] should encourage us to take these with a pinch of salt rather than use flawed statistical commentary to spin the data in the direction you want.
The dire competition imposed on the HE sector by gradually removing public funding has led our universities to become individualistic, hoping that they will do better than the others. A symptom of this trend was revealed during the pension dispute and recent news that some universities are tempted to ‘leave the USS boat’ to escape any solidarity. Unfortunately, this competition is a negative sum game. Our ‘competitors’ have the same degree of intelligence as we have and fighting them will only result in all of us taking the hit. The risk is that while we spend all our energy trying to be ‘better’ than the others using such short-term and flawed criteria, we cease to be good according to academic criteria.
Rather than compete, we should collaborate and lobby against the plan to turn our universities into businesses. We should work in a university where honesty and transparency takes precedence over spin. In times of financial hardship, the right leadership would respond as an academic community, not as a corporation. We do not need ‘business plans’, ‘performance metrics’ and senior managers having to sign Non Disclosure Agreements before engaging with university strategy. We need open and honest debates and transparent decision making. These issues are at the heart of what staff and students believe are at stake in the appointment of our new VC. They are why we are asking for better senior leadership and a stronger more robust Senate.