EXPERT COMMENT LAST UPDATED : 15 FEBRUARY 2019
Birmingham City University’s Professor Matt O’Leary shares his views on what his new research with Dr Vanessa Cui and Dr Amanda French has revealed about the Teaching Excellence Framework (TEF).
Regardless of where you stand on the TEF, there is no denying that it has been a catalyst for fervent debate since its launch in 2016.
It continues to be a topic of conversation on campus and to be debated by higher education (HE) stakeholder groups, the National Union of Students, Universities UK, university vice-chancellors and TEF policy wonks to name a few, but not HE teaching staff. The voices of HE teaching staff has been noticeably absent in the development, implementation and assessment of the TEF to date.
A key milestone in the TEF debate
Our research report “Understanding, recognising and rewarding teaching quality in higher education: an exploration of the impact and implications of the Teaching Excellence Framework”, published this week, therefore marks a key milestone in the TEF policy debate by capturing the voices of some of those most affected by this policy reform. The report includes the views and experiences of over 6,000 academic staff working in universities and college-based HE providers in England, Scotland, Wales and Northern Ireland.
Commissioned by the University and College Union (UCU) to provide an evidence base of the impact and implications of the TEF on their members working in university/college-based HE, myself and colleagues at Birmingham City University undertook the research from February to November 2018. The study investigated UCU members’ awareness, involvement and perceptions of the TEF and its impact on their professional lives in their workplaces to date.
Is the TEF fit for purpose?
So, what did we find out and how might our findings inform the Independent TEF Review?
Our research reveals that the TEF has proven to be a highly unpopular policy among the vast majority of the project’s participants, with only one in ten welcoming its introduction. It calls into question the fitness for purpose of the TEF as a vehicle for making valid and reliable judgements about teaching excellence and its ability to fulfil its purported aims of improving the quality of teaching and increasing student choice in HE.
Participants’ perceptions of the conceptualisation and methodology used to capture teaching excellence in the current TEF was that it failed to do so in any meaningful way. Not only is the TEF not a direct measure of the quality of teaching - as acknowledged openly by the chair of the TEF assessment panel, Chris Husbands, when interviewed for the study - but it was not considered a credible measure at all by the overwhelming majority of the study’s participants, as emphasised repeatedly across data sets. This is quite a damning indictment of a policy that, according to Husbands, is unlikely to disappear in the near future:
I’ve got no reason to suppose that the TEF is going to disappear … And I suspect, although I’ve not spoken to anybody in the opposition, that it’s too useful a tool for any incoming government to want to get rid of it. So I think this is part of the landscape and we’ve got to make it as effective a part of the landscape as it can possibly be.
Husband’s statement inevitably raises a number of questions about the future of the TEF. How and what is the TEF useful for? For whom? Beyond its use as a market signal for categorising HE providers according to its ranking system, who actually benefits from the TEF? And how can it become (according to Michael Barber) a “catalyst for the improvement and innovation in the quality of teaching,” when it is based on a unidirectional, episodic, desk-based assessment of a collection of data sets, much of which have only tenuous links to teaching and teaching excellence?
During a research interview I carried out as part of the project, Chris Husbands repeatedly used the analogy of the TEF assessment processes as a vehicle for ‘telling a story about the institution’. But who are the narrators of this story? Who is the audience? And to what extent do the protagonists of this story have the opportunity for their voices to be heard?
What actually is ‘teaching excellence’?
Any framework or approach aimed at improving teaching must include staff at all levels. Institutions need to ensure that the consultation, creation and communication of all TEF-related (or any other national teaching policy) work (from policy formation to implementation) include a broader representation of all staff and not just an elite group of those in (senior) management and/or professional services. This requires a more inclusive approach to involving staff in shaping institutional policies and practice.
Despite the flaws and criticisms of the TEF expressed by participants in our study, understanding, recognising and rewarding teaching excellence was repeatedly acknowledged as an important mission, and broadly welcomed by the HE workforce. Staff were very committed to the idea of improving teaching quality and were keen to share and collaborate on ideas and practices.
Lessons from other sectors
Across the study, participants provided a wealth of ideas and suggestions for alternative, more representative ways of considering, talking about and capturing teaching excellence. They also had a lot to say about alternative methodologies, approaches and/or considerations for understanding, improving, recognising and/or rewarding teaching. Such was the wealth of ideas generated by participants that a separate section of ‘Alternative visions of teaching excellence’ was created in the report.
There are valuable lessons for HE to learn from the schools’ improvement agenda about the counterproductive effects of relying too heavily on the use of metrics to assess and performance manage educational provision. One of these lessons is that there needs to be less focus on showcasing manufactured manifestations of excellence in response to accountability-led regimes and more on supporting collaborative and collegial work across institutions that is responsibility-led.
Whilst recognition and reward schemes may act as short-term incentives, there is little evidence that they lead to long-term, sustainable improvement either individually or institutionally. This is something that comes from supporting not sorting staff. Whether the TEF or a variation of it remains part of the future HE landscape or not, our research reinforces the need for a radical policy review that places greater emphasis on supporting HE providers to collaborate rather than compete with each other to improve the quality of teaching.