Categories
Uncategorized

Making Evaluation Meaningful 

Making Evaluation Meaningful

By Tim Senior, 8th Feb, 2024

In this blog post, CTC Researcher, Tim Senior, describes the innovative approach he has taken to evaluation on the CTC project and outlines his plans for developing this model in new contexts.  

Why evaluation matters

Evaluation, when done right, can help individuals make sense of their arts and cultural life, help organisations understand the value of their work (or where they need to change), and help funders make good decisions on where funding should go. In reality, however, this is often not the case. Over the three years of the Connecting Through Culture project, we have worked with our co-researchers and partner organisations to ask how we can co-design better evaluation practices. Conventional approaches to evaluation fail in large part because they only serve some of those who should be at the centre of the process: evaluation may serve the needs of funders but run-counter to what organisations know matter to their service users; evaluation may serve the need of organisations or researchers, but feel extractive to those participating (through being tone-deaf to people’s lived experiences); evaluation may richly detail lived experiences, but frame those experiences in a way that leave funders and policy makers lost at sea. At worse, we are adopting a culture of evaluation that weakens, rather than strengths, the relationships between participants in arts/cultural life, organisations, funders and policy. The opportunity in Connecting Through Culture has been to collaboratively design (co-design) an evaluation practice for arts and cultural experiences that works for everyone involved.  

Good evaluation

To these ends, we’re working with the principles for good evaluation developed by the Centre for Cultural Value . These principles offer a detailed and wide-ranging account of what people-centred, beneficial, robust and connected (socially engaged) evaluation practice could be. Part of our work in CTC has been to identify methods and theoretical approaches that can turn these principles into concrete, practical solutions. In this process, we have worked closely with our co-researchers and project partners to disrupt conventional understanding of evaluation practice: We have, for example, asked how people engage with arts and cultural experiences in the real-world – helping us imagine an evaluation practice that ‘comes to you’ rather than feeling imposed from the outside; … we have asked how sense-making around arts and cultural experience is both an individual and shared activity – helping us imagine what is gained when you blur traditional ‘red lines’ between evaluator and those ‘evaluated’; … we have asked how co-designing ‘what matters’ together can bridge a variety of needs and values (from individuals to organisations) – helping us construct a model of evaluation that generates both qualitative and quantitative measures to serve different audiences.  

How it works

The prototype has a simple logic to it: you choose a themed envelope that best captures your motivation for attending an event (a workshops, a site or activity). There are five themes: I’m looking to Enjoy myself; I’m looking to Grow; I’m looking for a Role; I’m looking for connection; I’m looking to the Future. That envelope is then yours to keep, to be revisited later. At the end of the event, the envelope is then opened to reveal tokens describing different impacts connected to that theme, from which a selection can be made. Finally, opening-out the envelope reveals the question “What might I do next?” with prompts for reflection. In short, a connective thread is drawn between the promise of an event, it’s real-world meaning, and the potential it opens-up for the future – all in the context of someone’s own life-world and life-course. Within this logic, there is a high degree of flexibility, both in the depth of engagement that’s possible (aided through facilitation) but also the integration of these activities into arts and cultural events themselves: ‘Evaluation’ becomes a process of self-reflection that might be made part of The Experience, rather than an intrusive addition that serves solely the purposes of data collection for someone else.  

How we’re testing it

We’re currently working closely with one of Connecting Through Culture’s projects – Expressive Pockets – to test this initial prototype. Joining workshops at four locations across Bristol, we are now making sense of the strengths and weaknesses of this new evaluation practice. In parallel, we are working with CTC’s co-investigator Paul Mitchell and Isabella Floredin to trial a wellbeing evaluation approach that is increasingly popular within Health Economics (Icecap); this will help us ask how our new approach might work productively with more conventional evaluation methods. Finally, we have partnered with St George’s Bristol (an historic live music venue in the city) and the Wellspring Settlement (an important community anchor organisation in Barton Hill, Bristol) to develop our new evaluation practice further. Together we’ll ask how a new approach to evaluation might be integrated into existing organisational cultures of evaluation and ongoing conversations with funders.  

We will be publishing our findings in the months to come.