The Realist Evaluation of the Community Engagement Project

The NHS Community Mental Health Framework for Adults and Older Adults 2019 created large scale changes in expected mental health provision.  Our evaluation showed good extent of positive changes to provision in Somerset, partially created by integrated secondary-voluntary provision.  However, leaders and experts by experience in Somerset were concerned that these positive changes did not reach all equally.  To address this concern an alliance of voluntary sector organisations were funded to undertake the Community Engagement Project. It was unknown how, why, for whom and under what circumstances the Open Mental Health ‘Community Engagement Project’ would access to and provision of both statutory and non-statutory mental health services for people belonging to communities of identity that are underserved by existing services.    

Aims  

Understand how, why, for whom and under what circumstances does the Open Mental Health ‘Community Engagement Project’ improve access to and provision of both statutory and non-statutory mental health services for people belonging to communities of identity that are underserved by existing services.  

Activity   

A Realist approach was employed to better understand the CEP in terms of what works, under what circumstances, for whom it works, and why.  Data were obtained using a researcher-in-residence model, enabling an in depth understanding of system changes.  Interview and observational data were analysed alongside internal documents to address the aims of the project.    

You can find out more about PenARC’s realist work here

You can find out more about the University of Plymouth’s researcher in residence work here

We undertook a Realist evaluation in 2 phases:  

Phase 1:  

The researcher in residence undertook stakeholder engagement and evaluation of internal documents to develop an initial programme theory explaining how, why and under what circumstances the Community Engagement Project achieves its outcomes. The researcher in residence then co-designed the remainder of the project with community engagement workers, their manager, and the project leaders.  

Phase 2:  

We interviewed and observed community engagement staff and leaders.  Interview and observational data were collected and analysed retroductively to investigate the extent of change, and to test and refine the programme theory.  Following co-design with project leaders there was a particular focus on:  

1.    What small, medium and large changes to improve access and provision are made as a result of this project, and how, why and under what circumstances were these made?  Including how do people with lived experience contribute to these changes?  

2.     What might a model that can be replicated elsewhere look like, and what are the roles of contextual factors in this model?  

Outputs  

Findings were presented to project leaders via the Community Engagement Steering Group.  Interim findings were used to understand future commissioning needs. 

An internal report of the evaluation results, including the refined programme theory was completed in November 2024.
 
Methodological findings regarding the sharing of programme theory across different mental health community projects were shared at the European Network for Mental Health Service Evaluation 2024 conference.    

Next steps  

Our learnings from this project included how community organisations can contribute to improving mental health equality.  Alongside the Community Builders Realist evaluation this work has helped inform the design of our:  

• Mental health inclusion realist synthesis: How, why, for whom, and under what circumstances do what interventions improve inclusion in mental health services for marginalised groups?    

• Successful NIHR HSDR bid application: Coastal and Rural Mental Health Realist Study (CARMHRS): what works, for whom, in what circumstances, how and why?  

• Stage 1 NIHR PgFAR bid application: Prevention and Care for Common Mental Health Problems Across the Life Course.