In this blog post, CTC Researcher, Tim Senior, describes the innovative approach he has taken to evaluation on the CTC project and outlines his plans for developing this model in new contexts.
Why evaluation matters
Evaluation, when done right, can help individuals make sense of their arts and cultural life, help organisations understand the value of their work (or where they need to change), and help funders make good decisions on where funding should go. In reality, however, this is often not the case. Over the three years of the Connecting Through Culture project, we have worked with our co-researchers and partner organisations to ask how we can co-design better evaluation practices. Conventional approaches to evaluation fail in large part because they only serve some of those who should be at the centre of the process: evaluation may serve the needs of funders but run-counter to what organisations know matter to their service users; evaluation may serve the need of organisations or researchers, but feel extractive to those participating (through being tone-deaf to people’s lived experiences); evaluation may richly detail lived experiences, but frame those experiences in a way that leave funders and policy makers lost at sea. At worse, we are adopting a culture of evaluation that weakens, rather than strengths, the relationships between participants in arts/cultural life, organisations, funders and policy. The opportunity in Connecting Through Culture has been to collaboratively design (co-design) an evaluation practice for arts and cultural experiences that works for everyone involved.
To these ends, we’re working with the principles for good evaluation developed by the Centre for Cultural Value . These principles offer a detailed and wide-ranging account of what people-centred, beneficial, robust and connected (socially engaged) evaluation practice could be. Part of our work in CTC has been to identify methods and theoretical approaches that can turn these principles into concrete, practical solutions. In this process, we have worked closely with our co-researchers and project partners to disrupt conventional understanding of evaluation practice: We have, for example, asked how people engage with arts and cultural experiences in the real-world – helping us imagine an evaluation practice that ‘comes to you’ rather than feeling imposed from the outside; … we have asked how sense-making around arts and cultural experience is both an individual and shared activity – helping us imagine what is gained when you blur traditional ‘red lines’ between evaluator and those ‘evaluated’; … we have asked how co-designing ‘what matters’ together can bridge a variety of needs and values (from individuals to organisations) – helping us construct a model of evaluation that generates both qualitative and quantitative measures to serve different audiences.
How it works
The prototype has a simple logic to it: you choose a themed envelope that best captures your motivation for attending an event (a workshops, a site or activity). There are five themes: I’m looking to Enjoy myself; I’m looking to Grow; I’m looking for a Role; I’m looking for connection; I’m looking to the Future. That envelope is then yours to keep, to be revisited later. At the end of the event, the envelope is then opened to reveal tokens describing different impacts connected to that theme, from which a selection can be made. Finally, opening-out the envelope reveals the question “What might I do next?” with prompts for reflection. In short, a connective thread is drawn between the promise of an event, it’s real-world meaning, and the potential it opens-up for the future – all in the context of someone’s own life-world and life-course. Within this logic, there is a high degree of flexibility, both in the depth of engagement that’s possible (aided through facilitation) but also the integration of these activities into arts and cultural events themselves: ‘Evaluation’ becomes a process of self-reflection that might be made part of The Experience, rather than an intrusive addition that serves solely the purposes of data collection for someone else.
How we’re testing it
We’re currently working closely with one of Connecting Through Culture’s projects – Expressive Pockets – to test this initial prototype. Joining workshops at four locations across Bristol, we are now making sense of the strengths and weaknesses of this new evaluation practice. In parallel, we are working with CTC’s co-investigator Paul Mitchell and Isabella Floredin to trial a wellbeing evaluation approach that is increasingly popular within Health Economics (Icecap); this will help us ask how our new approach might work productively with more conventional evaluation methods. Finally, we have partnered with St George’s Bristol (an historic live music venue in the city) and the Wellspring Settlement (an important community anchor organisation in Barton Hill, Bristol) to develop our new evaluation practice further. Together we’ll ask how a new approach to evaluation might be integrated into existing organisational cultures of evaluation and ongoing conversations with funders.
We will be publishing our findings in the months to come.
In this blog post, CTC Researchers Nick Gray and Tot Foster reflect on their participatory film work in Connecting Through Culture (CTC) and discuss how they are planning to carry this forward.
Nick Gray: I’m so happy that you and I have just been awarded some funding for a community project, making videos with older adults and then transferring the knowledge that we gain from that process into workshops with creative practitioners in the city. Can you explain the thinking behind this new project and what you think we might achieve through it?
Tot Foster: Through CTC, we’ve learned the importance of supporting stakeholders, who might be co-researchers or community organizations, to expand their creative toolbox. We’ve developed our own practice as well – it’s important to acknowledge that. Somehow it feels like we’ve established a virtuous circle of us learning together. That has really paid dividends, because we’ve now been awarded university public engagement money to work with two arts organizations in the city. We hope to learn more from them about their practice in delivering and holding creative spaces for different groups in their communities. And they can learn from us some of the skills that we have brought to and developed in CTC. In this case, it’s participatory video production skills. One of the first things we’re going to do is look at is what we bring to the project, and what our partners bring to the project and plan the best way to deliver a programme that then empowers older people. It’s a multi-stage process. So, after that we can get together with our partners and the older filmmakers and really analyse what happened in those workshops in order to learn from them. And then that will inform a further round of workshops for creative practitioners in Bristol, to spread our learning about successful practice.
NG: The approach taken in CTC was quite a slow process: a three-year project. This project is in a much shorter time frame, just six months. What strategies do you think we could use to encourage older adult co-researchers to contribute their insights into the workshop process, so that knowledge can be mobilised in a much shorter space of time.
TF: I think it’s about our partners’ experience in knowing how to inspire confidence, excite and hold – that’s a word that comes up a lot – hold a group of people. And the knowledge they have of individuals within their communities; using the familiarity of participants with those spaces and structures to help them feel at ease with sharing their ideas and their thoughts on the process – they know better than anyone what the opportunities and challenges there are when they make films in a workshop.
NG: Now two concepts that don’t often come up together in academic or cultural discourse is ‘filmmaking’ and ‘health and well-being’, yet this is central to this project – what insights have you gained from CTC into the relationship between cultural production, such as filmmaking, and health and well-being in older adults?
TF: There are multiple aspects to that. One is the idea of age being limiting. Filmmaking is often perceived as a younger person’s activity, generally because it involves digital equipment that older people might feel less confident with. If older people can claim that creative space, it can give them a sense of achievement, of empowerment. Second, there’s a power in having a voice – that’s come out over and over again in conversations with co-researchers on CTC; that people gain a sense of value and self-worth by being given the opportunity to have a voice and to speak about what matters to them, rather than speaking about what other people think matters to them.
NG: That relates to the third phase of this research – the public exhibition of these films.
TF: It’s part of having a voice. One aspect is being able to express what it is you want to express, and that has positive mental health aspects to it. And the other is being seen, being visible, and part of being visible is having a voice in the public sphere. What I’ve learned about filmmaking through CTC is that even if that public expression reaches only a small number of people, the act of being in public is important. Having what you want to say validated is important.
NG: How does that extend to the production of knowledge itself? Do the co-researchers have the same feeling of contributing something positive through the knowledge they’ve created as a collective? I suppose this all goes back to the way we consider the older people we work with in the research – not just co-producers of films, but co-producers of knowledge: a group of researchers with a shared purpose: a knowledge community.
TF: It’s harder to get to that but, when our co-researchers saw everything that we’ve done on CTC drawn together at the project Showcase event in October 2023 it really brought home how much we had achieved together, and they also saw how other people who were attending were bowled over by what they had done. I don’t think necessarily people always identify their individual contribution, but if they feel part of a collective that has achieved a lot I think that has had a really positive effect. There’s pride there.
NG: Yes, and I think that the diversity of the people involved in CTC really strengthens a knowledge community. This is one of the great things about a media text like a like a film, that it can travel, that it can reach audiences, that it can raise awareness and it can let people see very clearly the potential of individuals from a wide range of backgrounds. You talked about visibility. You talked about being heard. It also promotes the potential of these voices to actually do something, not just for themselves, but also to actually change something, to make a significant contribution to knowledge that will have a tangible effect in the real world. To offer insight, but also share knowledge.
TF: I think that’s absolutely right. As long as we can get them out to an audience, which does take some organisation, but luckily, we have already established strong relationships with local cultural organisations like Watershed where we are going to screen the films in June 2024.
NG: As well as building capacity for local organisations to deliver film workshops, being able to make films has, potentially, other benefits for our community partners. I’m thinking here about some of the practitioner work you have already done, Tot, with community organisations, which has borne fruit in terms of the rich media they have produced to advertise their projects, facilities and services.
TF: Yes, it’s not just about capacity building in terms of their work with older people and adding another string to their bow in terms of the arts and cultural activities that they can offer to people they work with; it’s also adding to their capacity in that they can use the results of that filmmaking in their communications, to promote their offering in the community, particularly through social media. So there’s professional development for the people involved that can be passed on within and between organisations.
NG: This idea of sustainability is a key feature of this project. We’ve developed our practice working on CTC, and now we’re going to take that outside of the project. We’re going to work with other community partners to develop their capacity and so it goes on, in what you have termed a virtuous circle.
TF: That’s exactly the point about enabling and empowering and knowledge exchange, as opposed to instructing or sharing our knowledge and hoping that will evolve. I think enabling and empowering has a much greater possibility of being sustainable because it makes its own momentum, and people learning together spreads. And the sheer enjoyment of being involved in something creative together, that shouldn’t be underestimated.
NG: Another aspect of the project that I would like to touch on is the social element. I mean video production is innately collaborative, it’s very difficult to make a film with just one person. You often need others, and there will be people contributing in all kinds of different ways across the group and even in families and communities between workshops. So connections are made and communities are built as a result of that. The concept of the life course was central to the CTC project, as it is to this next stage of our work. It’s the idea that people’s health and well-being, both in a physical and mental sense, are affected by a range of other factors, including their previous experiences and social and cultural participation. These workshops are specifically targeted at communities where these participatory opportunities may be limited. To what extent is this next stage of our work an attempt to bridge these participation gaps: to address the inequalities that negatively affect the life course?
TF: CTC has worked with people who have faced challenges at one time or another or even throughout their whole lives. One thing I have personally noted is that several of the co-researchers have not been afforded, and sometimes not given themselves, the space or credit for their own creativity. CTC has supported them to unlock that, opening doors to creative activities in a way that has the potential to bring a wellbeing benefit. One of the aspects of CTC, and obviously this filmmaking workshop programme, will be using certain technologies with people who may or may not be familiar or comfortable using those technologies. Using phones to make films could lead to the possibility, for example, of recording and sharing video images with friends and family, or participating more actively on social media platforms.
NG: So our approach, our offer to communities of older people, is not about using technology to solve problems or attain better health, but to use these tools to enhance and enrich their everyday practices, their everyday creativity and their relationships?
TF: Yes, what the last three years have really shown is that it’s social connection that really matters. As CTC has gone on, we have seen that the technology has been in the service of making social connection, of discovery new ways to express yourself, to participate in things going on, to communicate. Digital competencies are only relevant in that they enable people to do other things, to make those connections, to be the creative beings that they want to be.
Nick and Tot would like to extend a big thank you for this funding, which was awarded by the Public Engagement team at the University of Bristol from the Research England QR Participatory Research Fund (QR PRF) 2023-24 – Making Change Together Fund (DARO).
In this blog post, CTC Researcher, Nick Gray, invites the CTC community to reflect on our project showcase: a whole day event at the Watershed, in which we told the story of our research and our six prototype teams presented their work to the world.
On 17 October we held our Connecting Through Culture As We Age Showcase at the Watershed. In the morning session we heard from lots of different people involved in our project, including academic researchers, our older adult co-researchers/co-designers, our project partners and our creative partners. After lunch, it was all about the six prototype teams, who unveiled their creations to a large and enthusiastic audience.
The corridor connecting the two rooms housing the prototypes also became an exhibition space where we were able to show the co-researchers’ films and display their poems and our wall hanging, consisting of fabric squares embellished by CTC researchers and co-researchers and assembled by Fanny Eaton-Hall. In this post, I have attempted a similar process by requesting that various people involved in the showcase record their reflections of the event, which I have stitched (hopefully as seamlessly as Fanny!) together into a narrative that captures the experience from a range of different perspectives. We have been, and continue to be, on such an incredible journey, so it seems fitting to frame each response as a series of tales that unfold as we travel together.
The Co-facilitator’s Tale by Fanny Eaton-Hall
The Showcase was such an exciting event – so much to see and do! Recycle City were encouraging people to be creative with reclaimed `trash’; my team at Expressive Pockets were helping people make a pocket; Murmurations’ blanket was a joy of stories. Tabletop Travels’ beautifully designed boxes of flavoursome Asian sweets were tempting, but not for me. The projected book from Retirement Reloaded was fascinating with glorious illustrations. Anyone Remember the Washhouse? brought back memories from my childhood and was a poignant history lesson.
Personally, I was delighted to be in conversation with Alice Willet and Jess Linnington, talking about the early workshops at KWMC Factory, and I particularly enjoyed talking with Paul Clarke and collaborative designer Becca Rose about Expressive Pockets, the prototype where I was co-designer and co-facilitator, often needling the team on possible participant reactions to our ideas.
The Principal Investigator’s Tale by Helen Manchester
This day had been long in the planning. I was really delighted with the morning session where we told the story of the project, adopting a narrative approach, and including a lot of our co-researchers and partners, which really seemed to work for the audience. I liked that the process of discussing the showcase with partners had led to some fascinating reflections and discussion which will help us to move forward. Having so many co-researchers, partners and friends in the audience, I felt the warmth, and sheer joy and knew we’d all achieved something special.
I spent the day having conversations with practitioners, policy makers, other academics and partners about what the project had helped them to see or understand. They commented on the long term, slow co-design approach, on the prototype projects that challenged ageist approaches to technological design and on the sheer scale of what we had achieved together. I felt full of pride for the whole project community and what we have made happen.
The Newbie’s Tale by Nick Gray
As the newest member of the CTC university research team, I could not have asked for a better introduction to the work of our talented co-designers and creative partners than them telling the story of the project in the morning session. What struck me most in the afternoon was the sheer range of the projects on display at the showcase: some intimate, some celebratory, some multisensory, some immersive, some reflective, but all thought-provoking and challenging commonly held (mis)conceptions of ageing.
Being a film teacher, I loved the co-researcher movies, particularly the poetic ones and the animations. Just as the prototypes were a wonderful celebration of the work that can emerge from creative group dynamics, the films of individual co-researchers offered fascinating and engaging insights into their personal interests, concerns and stories.
The Filmmaker’s Tale by Tot Foster
I loved the positivity and warmth from a huge range of people. It felt like a community around the project. One of my old friends who came commented that the listening to the story of our work made her think about her own practice and validated her ‘slow design’ approach. It was interesting to really see, for the first time, everything in one place. I thought, “Wow! Haven’t we all done so much together and had so much fun. So many people have been involved.” It struck me how there was such a strong narrative around the project; in research terms but also a narrative that is based on care, and on respect and value for all participants. I guess it was that that really instilled pride and emotion for me. Long live the CTC community!
The Early Career Researcher’s Tale by Alice Willatt
It was the first time we have told the full story of the project from start to finish, which was exciting. It was also fantastic to see all the people invovled in different elements of the project in one space and having the chance to interact with the protypes for the first time. It was the first time I had seen some of the prototypes, and having them all in one space with the chance to move around the room and interact with each one was really magical! I also particularly enjoyed watching the prototype project teams share their protypes with such diverse audiences, in some cases for the first time. There was a real buzz and celebratory atmosphere across both the showcase rooms, which collectively offered such diverse sensory cultural and connective experiences. There were also pockets of quiet reflective spaces, such as with people interacting with the Murmurations blanket and audio stories, or pausing to look through the collection of co-researchers albums. One of the highlights for me was seeing the response to the co-researchers wall hanging, which hung in the centre of the corridor. It felt like a real centrepiece of the project, the way it came together with each co-researcher contributing a uniquely designed square, with Ruby’s fabric scraps connecting everything together, and the skill, dedication and care that Fanny took in bringing it all together.
The Co-researcher’s Tale by Carmeletta Groves
I enjoyed the day. It’s so nice just talking to everybody when people come up to me and say, “ooh I like your movie” and just to say hello. I really liked listening to the book [Retirement Reloaded prototype] that they wrote my poem into. I liked that, that sounded good. It’s everything else too, it was good watching the movie, watching the clip from everything. It was very interesting, and I learned a lot and it let me feel good that there are various things that I can do.
The Evaluator’s Tale by Karen Gray
The morning session felt like a genuine and useful account of the co-design process in the really engaging and accessible form of a narrative. There were so many sides of the research presented by a range of different people, which, I thought, really captured the involvement of all the stakeholders in this project. It was both celebratory and really useful for fellow practitioners, who could take away a host of new approaches to working with older adults as co-designers.
In the afternoon, I was involved in developing our evaluative tools with two groups of enthusiastic volunteers. It was wonderful to work with people whose responses were so insightful and thoughtful. Their input will most certainly be incorporated in future iterations of our evaluative process.
A good day out? Sensory excitement and storytelling
By Tot Foster, 1st September 2023
In this blog post, CTC Researcher, Tot Foster, describes two recent co-researcher trips – what worked well and less well – and finds that the visits underlined how experiences may be at their most powerful when they combine something for the senses with relatable storytelling.
This summer co-researchers have been on two very different trips. Firstly, to the M-Shed and a ‘behind the scenes’ tour of storage areas; a mish-mash of everything from domestic appliances from 70 years ago, ceremonial carriages, train parts and chocolate moulds. Our volunteer guides took round two separate groups and when we compared notes at the end, our tours had totally different feels. One group were all excited by the idea of seeing Bristol heritage that isn’t on show to the public and were full of questions – their tour overran by some time despite sore feet! Some members of the other group were not so engaged – they saw shelves of dusty unrelated objects piled high, with little that made sense to their lives. Some people were perhaps expecting precious items behind glass; the idea of ‘museum’ was hard to relate to this scene that was poorly lit and labelled with paper tags. The ennui continued until one co-researcher spotted an early calculating machine that her mother had used in an office, bringing memories back into vivid focus. When we saw a huge horn-type object the volunteer said that had been used in early talking cinemas – you had to line up the needle on a record with the action on the film. One of the co-researchers said her mother had done that in a Bristol cinema – timed the roar with the image of the MGM lion at the start of movies. At these moments these objects suddenly embodied personal histories, shared experiences of past work, intense nostalgia, family. They told stories beyond their physical presence, beyond the facts relating to origin and function that the guide could provide. Laundry items made me think of the Anyone remember the Washhouse’ prototype project that is currently being co-produced by co-researchers and artists, and how those memories of our mothers’ labour needs celebrating – just as their tools demand to be preserved in M-shed. On the way out, the statue of Colston – still brightly coloured with spray-paint, reclined in front of the double doors we left by. I thought to myself this would open a complex conversation but in fact everyone by this point just really wanted a cup of tea!
Our second visit, a month later, could not have been more of a contrast – a sensory overload that grabbed everyone….but with little story or moments to relate to. We went together to “Wake the Tiger” – billed as an ‘amazement park’, built inside a warehouse in the light industrial zone of St Philips. As the first visitors of the day we were greeted by two women painting a car to make it appear as if it had sunk into a huge blue puddle in the car park. On the surface a series of quirky spaces inside exuded steampunk hippy vibes – secret doors, dizzying visual effects, tiny details inside books you could take off the shelves, infinite mirrors. The experience was kicked off by a five-minute explanation about how the place was a future housing development that had been abandoned. But there the story ended – it was almost impossible to match up the disorientating walkways and the sci-fi doorways with any narrative at all. Everyone was ready to be taken on a journey, told a story, but instead this assault on the senses was just that – and only that. The experience wowed for a moment but then was gone.
Two researchers from Connecting through Culture are exploring new ways of evaluating cultural experiences. But in the meantime, and for me personally, these contrasting experiences made me think that for a cultural experience to grab and hold my attention (and from what the co-researchers said about the visits I am not alone in this), experiences need to have some powerful sensory component but also tell a story and be relatable to my life. I felt, in that moment of reflection, that our research approach which began with co-researchers reflecting on their lifecourse, is one which offers learning riches. And taking older co-researchers life experiences as a starting point to co-designing the cultural services and products that are the six prototype projects has got every chance of leading us to a good day out.