‘It ain’t what you do, it’s the way that you do it’!!

Aongus O’Keeffe – April 2018

Aongus_1 (2)As most people know by now, one of the few positive and progressive things to come out of the Stormont Executive in the last couple of years was a commitment to making Government here more focused on the outcomes of the work it delivers and the investments it makes for people and communities in Northern Ireland. We may chuckle or agonise at the irony of this in the current environment – and the fact that a draft Programme for Government (PfG) developed in 2016 still remains in draft stage today, lacking the political sign-off that would enable it to become airborne.

Nevertheless, life goes on. The civil service continues to run the country and administer the funding programmes and services that are required to make the place tick. In the absence of any other guidance or framework the draft PfG is the principal guiding structure that binds all departments and agencies of Government. With a political vacuum with no end in sight, arguably there has never been a better opportunity for government to transform how it goes about its work and become a more efficient, effective and outcomes-focused entity.

So how does it do that? Where do you start? How do you redesign the aeroplane while continuing to fly it?

Here at Community Evaluation NI (CENI) this takes up a lot of our thinking time. With support from the Building Change Trust we have begun to lay the foundations for establishing an Outcomes Observatory that can support, challenge and inform Government & the Voluntary/Community (V/C) sector as they embark on this journey towards outcomes.

We recently hosted a knowledge exchange event, bringing together representatives of central & local government, V/C sector and academia to deliberate on the findings of a scoping study CENI has been undertaking to inform the Outcomes Observatory in its research focus over the coming years. It also presented an opportunity to debate and discuss some of the issues and challenges of overlaying an outcomes model on systems that were designed and developed with a different purpose in mind.

To help inform the discussion we also brought Dr Ailsa Cook over to share some of the insights and learning from the Scottish Government’s experience of adopting an outcomes approach nearly 10 years ago.

The lessons that Ailsa shared very much resonated with the findings from the scoping exercise:

  1. Need for outcomes approaches to be developed through partnership and co-design – this also resonates with the language of co-design articulated in the draft PfG – that policy and funding programmes must not be designed in a vacuum or in the proverbial ivory towers, rather they must centrally involve people and communities with a deep understanding of the issues/needs. It also requires the myriad of responsible departments and agencies to work more closely together and be aware of their respective roles and how each ties in together.


  1. Need for guidance, learning and support to transform ways of thinking and working – we can’t expect the whole of government and a diverse V/C sector to change overnight without significant support and investment in people and systems. There is a real need for on the job learning, peer sharing and independent facilitation to help build trust and confidence to make this happen and genuinely shift to a new way of thinking & working. If this support is not available, the risk is nothing will really change.


  1. Need to recognise the complexity of the issues to be addressed – many of the societal challenges we face cannot be reduced down to a simple indicator or measure and any programmes aiming to address protracted social issues must factor in the multi-faceted and complex nature of the problem (including public policy and power structures that feed into them). See example below of obesity system map that conveys the complex nature of just one of these societal issues – it is clear from this that no one intervention can address or ‘cure’ obesity. A particular intervention is merely making a contribution to addressing the overall scale of the problem.

Obesity System Map

There is a real need for programmes and projects be clear on the elements of the problem they are trying to address and for which they can thus be held accountable.  Addressing any such complex problem – or achieving any complex outcome is akin to a patchwork quilt – each patch needs to have a clear purpose and then needs to be well made in order for the big picture to come together. No one programme, department or agency can produce the proverbial quilt – it is the collection of patches woven together that helps achieve the overall outcome.  This is essentially the performance v’s population accountability distinction in Outcomes Based Accountability™ speak.


  1. Outcomes thinking is hard, takes time to learn and is easily subverted – Culture and systems are pervasive and for real change to happen there needs to be investment in addressing the culture and adapting the systems so that they are aligned for an outcomes-focused way of thinking and working.


  1. In order to be able to prove outcomes at any level, we need to improve how we go about achieving them – this includes all aspects of the programme cycle but particularly in the collection and use of data. Relevant data needs to be collected in a proportionate way and if it is not being used to learn and improve then there is no purpose to asking for or collecting it. Mechanisms need to be put in place that value and make use of qualitative data as much as the quantitative.

We see each of these issues as enablers or determinants of executing an effective and genuine outcomes approach – that is to say if these things are not done then very little is likely to change. Our intention now is to establish the Outcomes Observatory to look at how well we are implementing an outcomes approach and these five areas in particular will serve as a baseline or benchmark for improving practice in the future. This will be done using a practice research approach.

Our interest is in seeing what impact moving to an outcomes approach is having at different levels – and if it is not, why not? Are we doing the things that all evidence and learning suggests are necessary for implementing an outcomes approach? We see this as a journey of learning and improving and have a lot of expertise and insights to share and support. CENI has been providing hands on support to a range of funding programmes in working through some of the issues highlighted above and moving them towards genuine outcomes focused thinking and working.

Aongus is the Programme Leader at Inspiring Impact NI, based at Community Evaluation Northern Ireland
To get in touch e-mail: aongus@ceni.org

Running on Impact

– Aongus O’Keeffe, Inspiring Impact NI Programme Leader

20 December 2016

Inspiring Impact NI is now approaching  three years of age – a turbulent time in any child’s life – but is very grateful for the insightful, caring and thoughtful parents it has in the form of the Building Change Trust and Community Evaluation NI who have brought it this far. While reflecting on achievements can be lots of fun, as with all three year olds the future is where it’s at!

ultimate-goalSo as the Trust goes about winding down its operation, the umbilical cord of funding it has provided to establish and nurture IINI will dry up and it will be left to Inspiring Impact’s adopted parent (Community Evaluation NI) to help it develop and thrive to deliver on its mission: Putting impact at the heart of the voluntary, community and social enterprise sectors and their funders’. This is a good time to look at what will be happening in the year ahead.

The focus will very much be on demonstrating the benefits of adopting good impact practice – for voluntary and community organisations, for social enterprises, for funders, for commissioners and for investors. Good impact practice is distinct from impact measurement because it encompasses all the elements an organisation needs to consider and that need to be in place before and after actual measurement takes place. The why, the what and the how is critical so that our measurement is focused to generate relevant and useful information that can be used to learn and to improve how we address and resolve social problems.

In this vein we will be continuing our support to key Voluntary, Community and Social Enterprise organisations to embed impact practice into their way of working. We will be broadening awareness of the freely available supportive resources that VCSEs can access to help them apply impact practice and fulfil their public benefit reporting requirements to the Charity Commission for Northern Ireland. We will develop and publish good practice guides, learning, tips and challenges for organisations who want to improve how they work and become more impact-focused. We are also planning a series of practice learning sessions for our Impact Network that will commence in early 2017.

This all ties in with the latest Programme for Government which advocates an ‘outcomes-based approach’ to delivering public services. Through applying impact practice and being clear about what they do and what outcomes and impact  they deliver, we are supporting organisations to be in a strong position for whatever reality the new PfG throws at them as it commences implementation in 2017.

Ongoing support from the Department for Communities to work with Government to plan and prepare for outcomes focused-funding is a critical piece of this overall jigsaw. While on the one hand having an overarching outcomes framework to get all players building the same puzzle is very useful, on the other hand if you only hold a vague, isolated piece of that jigsaw and have no guidance that helps you to see how or where it fits then it’s very hard to complete the picture. The reality in this case is even starker in that many of the jigsaw parts have not yet been designed or produced!!

Programme for Government team at our Impact Practice network event.
Programme for Government team at our Impact Practice network event.

Our Department for Communities partnership enables us to provide ongoing support to key Government programmes funding the VCSE sector to ensure they have clarity of purpose, know what they wish to achieve and can determine how to go effectively about this. This is about working with all the relevant people and organisations to agree their desired outcomes, the steps required to get there and how to define and measure progress towards those outcomes that align with the overarching PfG framework.

So, as we enter our fourth year (our pre-school year!), we are delighted with the progress so far but are also very cognisant of the need for nuanced learning that can highlight how VCSE organisations and their funders can become better at what they do and become better at addressing some of the most acute social problems of our time. We plan to share our learning and insights in 2017.


Shared Experience: Helping Small Organisations Measure Their Impact

– Judith McComb, Sported

Sported is a free membership organisation that supports thousands of community groups and clubs across the UK that use the power of sport to change young lives.  

As an impact champion for Inspiring Impact over the past two years, Sported have been supporting their members to understand and embed the principles of good impact practice – encouraging groups to sign up to Inspiring Impact’s Code of Good Impact Practice, complete the Measuring Up! online self-assessment tool and develop an ‘Impact Practice Action Plan’ for their individual organisations.

To help other organisations who may be considering doing the same but are not sure how to approach the task, Judith McComb has shared Sported’s experience below.

95% of Sported’s Northern Ireland member groups are volunteer-led, with less than five paid members of staff. The concept of impact practice, along with the time and resources associated with measuring impact, therefore often appears daunting. Yet we know these groups are able to make a significant and powerful positive difference with their local communities. Done well, Impact Practice can be invaluable: it helps these groups to demonstrate the value of their work, and work more effectively… as well as attract money.

The flexible nature of the support offered to our member groups is an important way of facilitating their ability to understand and measure impact. This includes group sessions, one-to-one support to complete Measuring Up! with specific help  around action planning, and remote support via email and telephone. This support is provided by Sported staff, our new online resources and a dedicated team of volunteers trained in our #FitforImpact programme.

With this level of assistance from Sported’s staff and volunteers, the programme of support has been hugely successful with a total of 60 member groups engaged with the support (22 in NI).

Sported’s top tips for supporting voluntary-led groups:

  1. Ensure committee buy-in
    Inspiring Impact’s Code of Good Impact Practice (the cycle and principles) is a key resource for explaining the concept of Impact Practice and ensuring groups understand the value of developing in this area. As the majority of our members are voluntary-led groups, Sported view ‘pledging’ to the Code as a crucial step in ensuring buy-in from committee members and all volunteers/staff. As our support continues, it’s really interesting to learn of the creative ways our member groups are embedding the principles. Pledging to the Code has allowed this renewed focus and shared responsibility.
  1. Offer practical support
    At Sported, we complete the Measuring Up! assessment on an annual basis and have found it really useful for identifying areas of strength and where we needed to develop our impact practice. This first-hand experience is a valuable source of learning that we are able to share with our members.

In particular, we find that using the Guidance Notes and Add Notes features of the tool is important. The notes become the basis for the automatically-generated ‘action plan’ so it’s crucial to take time to think about this.  The guidance notes are a vital resource for our groups as they provide definitions and direct links to additional resources and information.

To speed up the learning curve and help reduce the conflict between time spent on planning versus delivering activities for young people, Sported have developed a glossary of key terms for the groups and a range of supporting documents with specific examples tailored towards our membership.

Sported also offer our groups support around completing the assessment, facilitating the Plan section of discussion. Upskilling the group around this facilitation has been really beneficial in ensuring they can continue to have those occasionally-challenging conversations.
We also offer this practical support element around the action planning, reviewing their progress and actions. We support our member groups to identify the areas of impact practice where they have gaps, setting SMART targets and prioritizing areas of development.

Offering practical support in this way helps build the capacity of groups to continue developing their impact practice in the years to come.

  1. Be flexible with the range of support
    As we gather feedback and learning from our member groups, Sported continue to develop the range of resources and support we can offer in the area of Impact Practice. The practical support of the reviewing action plans has enabled us to recognise common areas of development across the groups which we capitalised on by holding workshops on theory of change, communications, and Sported’s own shared impact measurement tool, Sportworks.

We have recently launched our new online Learner Journey, packed full of resources, videos and resources for groups to continue developing in this area. This ensures we can provide support to a wider range of our member groups across the UK and provide a platform for shared learning and development in this area.

The tailored and intense level of support offered by Sported to these groups has been a key factor in the ‘success’ of our support – enabling our voluntary-led groups to understand and embed what can seem like a daunting topic. We would strongly recommend this level of support for smaller, voluntary-led groups. We would also recommend collecting feedback about the experience and passing it on to the Inspiring Impact partnership group. It all helps refine the process, and is an important part of the wider ‘sharing’ aspect of the programme’s shared measurement vision.

Our final top tip is to always remember – and remind groups – that the ‘impact’ these groups can have is essentially the reason they exist.

Developing impact practice is about understanding the change you are trying to achieve. When you know ‘what works’ in bringing about that change and can plan for this, then you’ll be most likely to fulfil your potential, deliver on your mission and have a greater impact on young lives in your local community.