Deciding where to study – Findings from IFF’s high profile evaluation of the TEF

This week, the annual mid-January scramble for students to finalise their UCAS applications has once more come and gone, with one key question causing stress in households across the country:  where should I study? The Teaching Excellence and Student Outcomes Framework (TEF) seeks to help students make this decision, assigning Gold, Silver and Bronze awards to HE providers across the UK that denotes the relative quality of the teaching that they provide, and the outcomes of their students. IFF are delighted to see that our evaluation of the 2016-17 provider-level TEF has been published by the Department for Education today, see here for the full report.

This research evaluated the second year of the provider-level TEF against its objectives, considering the views of two key stakeholder groups: applicants to Higher Education courses for the 2018/19 academic year (close to 3,000 were interviewed) and Higher Education providers (311 staff participated, covering 195 unique institutions).

The applicant side of the research evaluated applicant awareness of the TEF, how the TEF is used by applicants and its role in their decision-making processes. We found that around a third of all applicants to HE courses had some knowledge of the TEF at the time of their application while 15% of all applicants used the TEF in their decision making. A further 11% of all applicants reported that they changed their application choices as a result of seeing the TEF awards, for example submitting their choices in a different order of preference. These are promising signs for the TEF and, should awareness of the TEF increase as anticipated, it suggests that the framework could become a widely used tool to inform future applicants’ choices around where to study.

The provider side of the evaluation explored how providers felt the TEF was performing and what impacts had occurred as a result. While there was some debate regarding its design and the criteria that inform the TEF, a majority of providers felt that TEF will be able to achieve most of its objectives: for example, 68% thought the TEF would raise esteem for teaching while a further 57% felt that it would better inform student choice. More pertinently, the study showed that the TEF has already contributed to changing practices within institutions, especially among institutions that received a Bronze TEF award. As a result of the TEF award that they received, a number of providers had already started to develop and invest in new initiatives aimed at boosting the student experience and student outcomes.

On the same day that the report was published, the DfE announced the TEF independent review and IFF will be keenly following its progress in the wake of today’s report.

For more information speak to blog authors Andrew Skone-James and Sam Whittaker.

E: andrew.skonejames@iffresearch.com

E: sam.whittaker@iffresearch.com

Seven aspects of engagement pilot (Qualitative Evaluation) – Findings from IFF and the University of Derby published by the Department for Education

The findings from IFF’s qualitative evaluation of the seven aspects of engagement pilot have recently been published by the Department for Education, see here. We completed this study in partnership with Dr. Deborah Robinson at the University of Derby.

In 2017 an independent review led by executive headteacher Diane Rochford recommended removing “P scales” as the statutory assessment tool for pupils working below the standard of the national curriculum assessments. For pupils with profound and complex learning difficulties and disabilities, it suggested replacing “P scales” with assessment against the 7 aspects of engagement for cognition and learning. DfE piloted this new assessment approach during 2018 before deciding on whether to introduce it on a statutory basis.

The pilot was evaluated by IFF and the University of Derby, using depth interviews with teachers, and a series of school-based case studies. Pilot schools were largely enthusiastic about using the 7 aspects of engagement for formative assessment as it enabled them to deepen their knowledge of pupils’ responsiveness and identify ways they could increase engagement levels in a pupil-centred way. Where schools had concerns these were around resource requirements, how best to interpret the results of the assessments, and how to communicate these to parents.

On the day the report was published, the schools Minister, Nick Gibb, announced that this new assessment approach for pupils with complex disabilities will replace “P-scales” from 2020. Informed by the pilot findings, Diane Rochford will now lead an expert group to refine the approach before it is rolled out nationally.

Creative Spark: IFF’s Elizabeth Shepherd contributes to Higher Education Enterprise event in Georgia

IFF’s Elizabeth Shepherd is providing a UK sector overview on driving Creative Enterprise Research and best models from the higher education sector at the British Council’s Creative Spark: Higher Education Enterprise Programme.

The event brings together representatives from the UK and all participating countries involved in Georgia today to kick off their plans.

Each country will be granted a maximum of £50,000 across the 35 new international partnerships between higher education and creative institutions in the UK, Armenia, Azerbaijan, Georgia, Kazakhstan, Kyrgyzstan, Ukraine and Uzbekistan.

The winning partnerships including Georgian cultural institutions are: Creative Georgia and Tbilisi State Academy of Arts to partner with Advantage Creative; Georgian Technical University with Keele University; Creative Georgia with University of East Anglia, Ivane Javakhishvili Tbilisi State University and V. Sarajishvili Tbilisi State Conservatoire with University of Essex Change School and Ilia State university with University Of The West Of England in Bristol.

The initiative aims to address to an underdeveloped creative sector and a demand for entrepreneurship training in these countries. The UN has previously recognised the creative economy as being one of the most rapidly growing sectors of the world economy. Creative Spark aims to give the next generation opportunities to develop skills required to compete in a global job market.

IFF’s Elizabeth Shepherd is speaking at a number conferences across the UK and Europe this month with the contributing to the Graduate Employment Conference on November 22nd and the University Market Insight Conference on November 27-28. Click here for details of IFF’s Higher Education work and these upcoming dates.

Perceptions of the UK’s unique Higher Education offer – views from international students

A new report published today by IFF Research, conducted in partnership with UK NARIC, aims to uncover current international opinion of the UK’s unique qualification offer, including the existing provision of one-year Master’s courses and the possible increased future provision of Accelerated Degrees.

The report provides unique insights into how international students perceive UK qualifications. In collaboration with UK NARIC, via their unique database of individuals around the world seeking educational services, we have been able to evidence current international attitudes and perception of UK HE qualifications.

Key findings from the Perceptions of the UK’s unique Higher Education offer – views from international students are:

  • 59% of the international respondents had not heard of Accelerated Degrees
  • Once introduced to the concept, 44% said they would consider studying on an accelerated programme
  • 56% of respondents believe the government in their home country does recognise a UK one-year Master’s degree

Lifelong learning

In late 2017 the UK Government launched a consultation into the provision of accelerated degrees. There is little existing evidence to suggest that accelerated degrees would appeal to international students. Advocates of this new type of higher education provision suggest that studying for a UK degree in a 2 rather than 3-year period would appeal to a broad spectrum of students, including mature students who want to retrain and enter the workplace more quickly, and those who do not take a traditional A-level route into higher education.

Headline findings from key regions
The research highlighted potential demand for accelerated degrees from individuals in the following region/countries. Survey participants gave an above average positive response when asked if they would consider studying for a UK fast track degree:

  • Sub Saharan Africa (62.23%); Zimbabwe (73.7%), Nigeria (59.2%), Ghana (53.8%);
  • Brazil (65.5%)
  • The Philippines (58.3%)

‘The perceptions towards the UK higher education system highlighted through this research are a useful reminder that even those who would choose to study in the UK are not always certain that the final qualification will be recognised in their home country.

While recognition in the home country is only one of many factors in deciding where to study amongst international students, a greater focus on the international acceptance of UK qualifications would support increased student recruitment.’

Paul Norris, Deputy Managing Director, UK NARIC.

‘Our study has shown UK Higher Education qualifications are still held in high regard internationally. It has also revealed a significant opportunity for the sector to provide access to qualifications in response to contemporary education and employment needs. Communicating the benefits of accelerated degrees, including potential cost and time saving, whilst maintaining UK quality standards, is key’.

Elizabeth Shepherd, Director, IFF Research.

IFF’s Elizabeth Shepherd will be presenting findings from this report at its launch at the UKNARIC2018 annual conference in Westminster today.

Access the full report here

Investigating alternative provision – Findings from high profile IFF study published by the Department for Education

We’re delighted that the findings from IFF’s large-scale, high profile study investigating alternative provision have just been published by the Department for Education, see here.

Alternative provision (AP) is, in essence, provision for pupils who, because of exclusion, illness or other reasons, would not otherwise receive suitable education. It includes education arranged by schools for pupils on a fixed-term exclusion, and pupils who are directed by schools to off-site provision to improve their behaviour. The research investigated current practice in AP, from how schools identify pupils at risk of being referred to AP, right through to the reintegration of pupils into mainstream provision, or on to post-16 destinations. It explored how schools support pupils at risk of exclusion, how schools use alternative provision, and how AP providers support pupils placed in their settings.

This was a very large-scale qualitative project, with IFF undertaking semi-structured telephone interviews with 275 schools and 200 alternative providers, as well as 25 in-depth case study visits to AP providers (encompassing depth interviews with heads, staff, pupils and parents). IFF worked with academics at UCL and the University of Nottingham who undertook a rapid evidence assessment of the literature on alternative provision, to inform and contextualise the research.

On the same day the report was published:

  • The Education Secretary Damian Hinds spoke at a roundtable event on alternative provision and exclusions, saying that while schools will still have the right to exclude as a last resort, where pupils are excluded, the quality of education they receive should be no different than in mainstream settings, and further he would not rule out legislation to ensure more accountability for schools that permanently exclude children and place them in alternative provision, see here.
  • The government published its response to the recommendations laid out in the Education Select Committee’s report, Forgotten children: alternative provision and the scandal of ever increasing exclusions
  • The DfE published a companion report to our research, examining system-level issues in alternative provision: ‘Alternative provision market analysis’

Blog written by Mark Winterbotham and Claire Johnson

IFF works with EAIE to launch comprehensive study on the state of internationalisation in Europe

Today marks the launch of the 2nd EAIE Barometer, the research study that signposts trends in Higher Education internationalisation throughout Europe. We were delighted to work closely with the EAIE team to design and administer the research, which achieved responses from 2,317 university staff working in internationalisation across 45 countries.

The findings are of particular interest here in the UK, where the current dynamic political situation has exacerbated existing uncertainties regarding the future of international students wanting to study here. Global competition to attract the brightest and best international talent continues to grow at an unprecedented pace. Rival destinations such as the United States and Australia have witnessed a surge in international student numbers, whilst the UK is in danger of being left behind.

What do the Barometer findings tell us about internationalisation practices among UK universities, and how does this compare with their European counterparts?

In short, the EAIE Barometer highlights that this is a worrying time for UK universities developing and implementing strategies for internationalisation. One in eight (13%) UK respondents said that they felt negative about the future of internationalisation; this was more than triple the European average (4%). The reasons behind this despondency are clear, with fingers pointed firmly at Westminster:

  • Six in ten (61%) UK respondents feel that restrictive national legal barriers are a key challenge in their pursuit of internationalisation (compared to 27% European average)
  • Three in ten (31%) identified political nationalism evoking anti-international sentiments as a challenge to internationalisation (vs. 10% average)
  • Nearly a half (45%) feel that national policies have a negative impact on internationalisation (compared with just 10% average).

As a result, UK universities are focussing more and more on internationalisation practices outside of the UK: 32% reported that over the next 3 years their institution will focus more on distance or blended international learning opportunities, while 29% are prioritising branch campuses and other transnational education activities.

We Are International

Contained within this week’s publication of the UK Migration Advisory Committee’s report was the recommendation that – counter to the anticipated outcome and hope of the sector – international students should not be removed from net migration statistics, which will no doubt come as a significant blow to the sector. However, the continued proactivity of individual institutions, and the sector as a whole, to welcome international students to the UK via initiatives like #weareinternational, shows great strides are being made to illustrate and maintain the UK’s commitment to internationalisation.

The IFF Higher Education team will be presenting highlights from the UK EAIE Barometer data at the Knowledge Partnership University Market Insight Conference 27 – 28th November.

Blog co-authored by Andrew Skone James and Elizabeth Shepherd

ESS 2017: The UK Employer Skills Survey has been published by the Department for Education

We’re delighted that the report for the flagship study, the UK Employer Skills Survey, has just been published by the Department for Education. This very large-scale study (comprising over 87,000 interviews with employers across the UK) provides the definitive picture on recruitment difficulties and skills gaps experienced by employers, the extent to which they train their staff in response, and overall employer investment in training.

We’re very proud that IFF has been the lead contractor and report author on all four occasions the study has been conducted (in 2011, 2013, 2015 and 2017).

A link to the survey findings can be found here

A global comparison of national student surveys: IFF at the Advance Higher Education Training and Learning Conference

IFF’s Elizabeth Shepherd assessed the impact of teaching at this year’s HEA Training and Learning Conference with a presentation comparing various national student surveys from across the world.

Globally HEIs are increasingly concerned with understanding student experience and focusing on its measurements. This growing international trend is evidenced by the number of nationally implemented and commercially driven metrics designed to measure student satisfaction. Access to this data is beneficial across the HE piste; to students, institutions and governments.

In her session Elizabeth provided cross-national analysis of this growing global sector priority, highlighting how teaching-focused activities from the United States/Canada, National Survey of Student Engagement; the Australian Student Experience Survey; and the UK’s National Student Survey.

You can see Elizabeth’s full presentation here:

Measuring the quality of a Higher Education experience: a complex product co-produced by the consumer

Higher education is a complex experience, offering an intensive, unstructured, interactional and uncertain consumer environment (Ng and Forbes, 2009). As highlighted in the December 2017 NAO report ‘The Higher Education Market’, public sector market failure can arise for a number of reasons, all present within the HE sector; users find quality difficult to discern when exercising choice because the ‘product’ is complex and personalised, and it is unlikely the purchase will be made more than once in their lifetime; users’ knowledge of the service is only discernible during, or after, ‘consumption’; users play an important role in co-producing the value that they derive from the service.

Facilitating the effective transition of students into the labour market is one of the top government higher education policy priorities of the moment. In 2018 the sector is primed for the launch of additional regulatory and measurement frameworks and consolidation of existing metrics that have the potential to not only impact institutional funding, but most crucially expose inefficiencies in the higher education sector to function as a public service market. Data have never been so crucial.

These unique and fundamentally complex characteristics of higher education sustain the perennial challenge of constructing adequate and appropriate tools to measure quality and satisfaction outcomes from its student ‘consumers’. However, as clearly outlined in the 2016 ‘Success as a Knowledge Economy’ white paper, the UK government believe strongly that ‘information, particularly on price and quality, is critical if the higher education market is to perform properly’.

Measuring student experience

In 2018 the Graduate Outcomes record will be rolled out by HESA; ‘a new model to capture rich, robust and innovative data about graduates’, to contribute towards the governments mandate for better informed student consumers. This new model presents an innovation in higher education outcomes measurement; the inclusion of optional questions aimed at quantifying ‘subjective well-being’ of students 15 months post-graduation.

The juxtaposed interpretations of a quality student experience, in terms of value for money or intellectual enrichment, present an ongoing methodological challenge for UK HE institutions navigating the current climate of financial survival or irreversible reputational damage and exit from the market.

Student experience surveys are increasingly the research tool of choice, used throughout the UK, North America and Europe to measure satisfaction, engagement or employment outcomes. However, as described in a 2015 EHEA report presenting an international comparison of national student surveys; ‘the widespread use of student experience and engagement survey data raises questions of reliability, validity and other quality characteristics of such data to be used as evidence in higher education decision-making’[1].

Tools to measure quality, value and satisfaction

Despite the proliferation of student experience surveys, we have identified several insightful examples of innovative research methodologies used by institutions around the world that facilitate greater understanding of what determines a quality higher education experience. For example, Nottingham Trent University Business School developed a ‘Student Value Model’. Building on theories of consumer service consumption, when applied to the higher education context, characterises value as a function of ‘results’, ‘service attributes’, ‘price’ and ‘acquisition and relationship costs’. The Nottingham Trent model, whilst acknowledging the ongoing debate around categorising students as legitimate consumers, presents interesting findings from domestic and international student participants. The study revealed that for UK domestic students ‘price’ remained key, whilst for international students ‘results’ were most closely linked to a high value experience. Suggesting a substantive difference in the way these two groups of students construe quality and value.

In the United States, the University of Illinois has since 2002 developed The Ethnography of the University Initiative. This unique institution wide initiative builds upon the ethnographic premise of generating socio-culturally grounded understanding from an emic perspective, or what is sometimes described as an ‘insider’ point of view. The theoretical construct of ethnographic research is employed using qualitative research methods throughout the institution to analyse the student experience from within, creating ‘a repository of student generated data on the University’[2].

In methodological contrast, the University of Sydney, Australia, have begun harnessing digital learning analytics to provide in depth understanding of the existing student experience and how this could be improved. The design includes a suite of active learning resources, such as videos, discussion forums, student information systems, library usage and assessments. The solution aims to profile students as multifaceted individuals characterised by a sequence of digital indicators, allowing the development of new pedagogical strategies to maximise student participation, and in turn their experience[3].

Horizon Scanning; the pursuit of methodological innovation

It is our historic involvement in the design and administration of surveys measuring student destinations, experience and outcomes, that has led the IFF Higher Education team to consider how we, as a research agency, can support institutions across the UK in the pursuit of methodological innovation. Horizon scanning is a systematic, data driven, future oriented methodology that can be used to examine the factors impacting your institution, enabling you to plan the most effective, strategic response.

Want to find out more about this approach? Download our brochure or visit our Horizon Scanning web page.

Alternatively, you can contact me directly by emailing elizabeth.shepherd@iffresearch.com

 

References:

[1] Klemenčič M., Chirikov I. (2015) How Do We Know How Students Experience Higher Education? On the Use of Student Surveys. In: Curaj A., Matei L., Pricopie R., Salmi J., Scott P. (eds) The European Higher Education Area. Springer, Cham

[2] http://www.eui.illinois.edu/

[3] http://sydney.edu.au/education-portfolio/quality-and-analytics/la-roadmap.shtml

IFF expands their Higher Education team with hire of new Director

IFF Research have recruited former British Council Director Elizabeth Shepherd to their Higher Education research team. Ms. Shepherd will bring considerable public sector and international experience having spent over a decade within the British Council’s international education research team.

Ms. Shepherd will be working to grow IFF’s existing Higher Education research portfolio as they continue to make significant strides in this space. The agency is now the trusted destinations tracker for over 35 higher education institutions and works with a number of others on a range of strategic and tactical research challenges.

IFF’s Learning and Skills Director David Vivian commented:

“The HE sector has a huge impact on the country and on individual lives within it. There is a huge amount of uncertainty across the sector following recent legislative and policy changes, all magnified by the unpredictable impacts of Brexit. We want to work with institutions to help them find new and better ways to continue to fulfil their invaluable remit.

Elizabeth will be involved in the next stage of growth and development at IFF as we look to develop a dedicated Higher Education offering and consolidate our position as one of the leading players both in the UK and internationally.”