Measuring the impact of your investment
When it comes to new initiatives, like those affecting our communities and neighbourhoods, we need to understand the impact of the changes we are putting into place.
There are many interpretations of what evaluation involves and offers, but put simply, evaluation measures performance of initiatives, projects, programmes and policies. A well-designed evaluation provides evidence of a policy or programmes’ impact, helping landlords to understand how effectively it is being implemented, and helps to maximise the outcomes.
Watch our webinar
Find out how to measure the impact of new initiatives, or understand the value added by investment.
Fundamentals of evaluation
Robust and proportionate evaluation is essential for understanding not just whether something works, but also how it works, and how it could be improved. Landlords can use evaluation to understand how their policies and programmes can be made more impactful, sustainable or scalable.
Importantly, there are risks in a lack of, or low-quality, evaluation:
- Ineffective programmes and policies might continue
- Adverse or costly impacts created (and sustained)
- Opportunities for improvement might be missed
Stages of evaluation
There are 3 stages of any evaluation:
Understand: To start, evaluations should include activities to develop and deliver a fit for purpose measurement approach, drawing on a full range of techniques and sources of evidence, including stakeholder mapping.
Measure: Conducting fieldwork and collecting customer feedback will test your assumptions and ideas and fill any gaps in knowledge.
Identify and Implications: The key part of evaluation is generating insights, exploring the implications of your findings, and telling the stories evident in your evaluation to communicate your conclusions effectively.
Understanding impact evaluation
An impact evaluation is a research process made up of a theory of change; which is a visual representation of your journey, documenting where you are now and where you want to be. The evaluation takes account of all your inputs, for example, of staff time and financial resources. The cumulative value of your change at the end of your journey is your impact. You will see where you are successful and where things didn’t work as well.
An impact evaluation will help you to understand and provide evidence of the effect you are having. It measures the answers to questions you have about what has changed since your initiative started.
Such as, “What change have we measured related to our spending / staffing?” Or “What changes have we measured in satisfaction?”.
In an example where a registered provider creates a community hub for their residents, an impact evaluation will measure:
- What is the added value for the residents?
- What benefits or problems did the community hub create?
- Has anti-social behaviour increased or decreased?
- Has neighbourhood satisfaction increased or decreased?
Impact evaluations are delivered through a review of existing management information, background research, and surveys, qualitative work, community events and consultations. If you are working on a programme of change over a longer period, a monitoring programme will regularly measure your inputs and outcomes from your initiatives to make sure you stay on track.
If you see that something hasn’t had the desired impact, you can look at the journey to understand if the inputs, or activities, or other resources have affected why you haven’t achieved the desired impact. Your decision making will be evidence-based, rather than based on assumptions, saving you time and money in putting things right.
Understanding process evaluation
A process evaluation assesses the process of implementation of a programme or a policy. You want to evaluate the process, so that you know what you need to repeat or change, depending on whether you achieved the desired impact.
It’s an approach to evaluation that answers the questions related to how. Such as: “How was this initiative funded?” Or “How was the initiative staffed?”
If an initiative has been impactful, you can understand why and replicate that success elsewhere. Without this knowledge you would find it hard to scale it, sustain it, or replicate it elsewhere.
Deciding what is best for you
Evidence-based decision making in time of tighter purse strings is essential, so you can prove the value of your actions. But deciding which evaluation to use really depends on the question you need to answer.
If you want to prove that your programmes, policies or initiatives have been successful (or not) – conduct an impact evaluation.
If you want to understand how your money and resources have been spent and the value added, then you’ll want to run a process evaluation.
Getting started with impact and process evaluation – next steps
Understand your aims
It’s important for you to reflect as an organisation and determine what’s important for you. What are your priorities, and within those priorities, which are the most immediate?
Consult with stakeholders
Determine who your stakeholders are and listen to their views. Map your stakeholders including their relationships to your initiative or policy. Identify how they feed into and support the evaluation, and how they can benefit from the evaluation evidence. It’s good to tailor communications to each member of your audience, so their opinions and evidence can be taken on board and used to improve evaluation design and delivery.
Harness your resources
Consider the resources you have available to conduct an evaluation. Do you have the capacity and capability to do this work in-house? How long has your initiative been in place? Will you be able to evaluate the impact, or will you need to give it more time? You need to understand your resources in terms of your budget, timings, and the in-house skills it would take to carry out an evaluation.
Bring it all together
Once you know your aims, considering your stakeholders and resources, you can move forward with an evaluation that suits your organisation.
Looking for guidance
With the help of one of our evaluation specialists we delivered a webinar to help landlords increase their understanding of impact and process evaluations. Watch the webinar recording to learn how this tool can help you demonstrate the impact of your investment.
IFF has a number of impact and process evaluation experts who can provide advice and support for you and your team. To find out more, or to get a quote on a tailored project contact our housing team.