Monitoring, mystery shopping customers

Measuring the quality of a Higher Education experience: a complex product co-produced by the consumer

Higher education is a complex experience, offering an intensive, unstructured, interactional and uncertain consumer environment (Ng and Forbes, 2009). As highlighted in the December 2017 NAO report ‘The Higher Education Market’, public sector market failure can arise for a number of reasons, all present within the HE sector; users find quality difficult to discern when exercising choice because the ‘product’ is complex and personalised, and it is unlikely the purchase will be made more than once in their lifetime; users’ knowledge of the service is only discernible during, or after, ‘consumption’; users play an important role in co-producing the value that they derive from the service.

Facilitating the effective transition of students into the labour market is one of the top government higher education policy priorities of the moment. In 2018 the sector is primed for the launch of additional regulatory and measurement frameworks and consolidation of existing metrics that have the potential to not only impact institutional funding, but most crucially expose inefficiencies in the higher education sector to function as a public service market. Data have never been so crucial.

These unique and fundamentally complex characteristics of higher education sustain the perennial challenge of constructing adequate and appropriate tools to measure quality and satisfaction outcomes from its student ‘consumers’. However, as clearly outlined in the 2016 ‘Success as a Knowledge Economy’ white paper, the UK government believe strongly that ‘information, particularly on price and quality, is critical if the higher education market is to perform properly’.

Measuring student experience

In 2018 the Graduate Outcomes record will be rolled out by HESA; ‘a new model to capture rich, robust and innovative data about graduates’, to contribute towards the governments mandate for better informed student consumers. This new model presents an innovation in higher education outcomes measurement; the inclusion of optional questions aimed at quantifying ‘subjective well-being’ of students 15 months post-graduation.

The juxtaposed interpretations of a quality student experience, in terms of value for money or intellectual enrichment, present an ongoing methodological challenge for UK HE institutions navigating the current climate of financial survival or irreversible reputational damage and exit from the market.

Student experience surveys are increasingly the research tool of choice, used throughout the UK, North America and Europe to measure satisfaction, engagement or employment outcomes. However, as described in a 2015 EHEA report presenting an international comparison of national student surveys; ‘the widespread use of student experience and engagement survey data raises questions of reliability, validity and other quality characteristics of such data to be used as evidence in higher education decision-making’[1].

Tools to measure quality, value and satisfaction

Despite the proliferation of student experience surveys, we have identified several insightful examples of innovative research methodologies used by institutions around the world that facilitate greater understanding of what determines a quality higher education experience. For example, Nottingham Trent University Business School developed a ‘Student Value Model’. Building on theories of consumer service consumption, when applied to the higher education context, characterises value as a function of ‘results’, ‘service attributes’, ‘price’ and ‘acquisition and relationship costs’. The Nottingham Trent model, whilst acknowledging the ongoing debate around categorising students as legitimate consumers, presents interesting findings from domestic and international student participants. The study revealed that for UK domestic students ‘price’ remained key, whilst for international students ‘results’ were most closely linked to a high value experience. Suggesting a substantive difference in the way these two groups of students construe quality and value.

In the United States, the University of Illinois has since 2002 developed The Ethnography of the University Initiative. This unique institution wide initiative builds upon the ethnographic premise of generating socio-culturally grounded understanding from an emic perspective, or what is sometimes described as an ‘insider’ point of view. The theoretical construct of ethnographic research is employed using qualitative research methods throughout the institution to analyse the student experience from within, creating ‘a repository of student generated data on the University’[2].

In methodological contrast, the University of Sydney, Australia, have begun harnessing digital learning analytics to provide in depth understanding of the existing student experience and how this could be improved. The design includes a suite of active learning resources, such as videos, discussion forums, student information systems, library usage and assessments. The solution aims to profile students as multifaceted individuals characterised by a sequence of digital indicators, allowing the development of new pedagogical strategies to maximise student participation, and in turn their experience[3].

Horizon Scanning; the pursuit of methodological innovation

It is our historic involvement in the design and administration of surveys measuring student destinations, experience and outcomes, that has led the IFF Higher Education team to consider how we, as a research agency, can support institutions across the UK in the pursuit of methodological innovation. Horizon scanning is a systematic, data driven, future oriented methodology that can be used to examine the factors impacting your institution, enabling you to plan the most effective, strategic response.

Want to find out more about this approach? Download our brochure or visit our Horizon Scanning web page.

Alternatively, you can contact me directly by emailing elizabeth.shepherd@iffresearch.com

 

References:

[1] Klemenčič M., Chirikov I. (2015) How Do We Know How Students Experience Higher Education? On the Use of Student Surveys. In: Curaj A., Matei L., Pricopie R., Salmi J., Scott P. (eds) The European Higher Education Area. Springer, Cham

[2] http://www.eui.illinois.edu/

[3] http://sydney.edu.au/education-portfolio/quality-and-analytics/la-roadmap.shtml