Linking local authority and NHS data in Lambeth

ASC and SLaM operate separately, with different datasets and organisational priorities. This gap limits understanding of how residents move between systems, what support they receive, and where there may be unmet need. The Lambeth ASC team identified the need for a joined-up evidence base to better understand pathways, demand pressures, inequalities, and opportunities for earlier intervention. 

In response, HDRC Lambeth partnered with King’s College London (KCL) researchers and SLaM to create the first linked dataset of this scale in the UK, joining anonymised local authority ASC records with NHS electronic mental health records held in Clinical Records Interactive Search (CRIS). 

The challenge

The project addressed a recognised local and national gap - the absence of cross-system data to understand the experiences of residents with coexisting mental health and social care needs.  

Several challenges were clear from the outset: 

• Distinct systems and professional cultures: ASC and mental health services collect data differently, have different operational pressures, and historically have not had a mechanism for shared insight. 

• No existing precedent: There was no local model or governance precedent for linking ASC records with mental health records. 

• Sensitive subject matter: In workshops, residents raised concerns about trust, transparency, cultural and linguistic representation, and the perception that anonymised data could still feel intrusive. 

• Technical misalignment: ASC case management systems were not designed for longitudinal research, with inconsistent structures, limited metadata, and reliance on operational knowledge to interpret fields. 

• Operational constraints: ASC teams were already managing high workloads, and the multi-organisation approvals process risked long periods without visible progress. 

1. Working with practitioners and communities to define the problem 

From the outset, the project was co-designed, bringing together colleagues from adult social care and HDRC Lambeth, senior leaders, academics from Kings College London, and residents to define the problem and shape the research focus. Early workshops surfaced shared concerns around reactive care, long waits, and poor follow-up, alongside community priorities including prevention, representation, and cultural safety. 

Co-design was embedded as a continuous mechanism, with ongoing dialogue used to iteratively refine research questions as the linkage process progressed. Resident contributions highlighted lived experience, inequalities, and barriers to engagement, while practitioners grounded discussions in service realities and operational constraints. 

This iterative process continually informed research priority areas, which include: 

• predictors of institutionalisation among adults with severe mental illness and older adults with dementia 

• patterns of service use preceding crises 

• inequalities across ethnicity, gender and deprivation 

The research priorities remained aligned with frontline needs, while adapting to the opportunities and limitations of the evolving linked dataset.

2. Establishing a robust governance process 

No precedent existed for the linkage, governance documentation – including a Data Protection Impact Assessment (DPIA), a data-sharing agreement, and documentation to provide clarity on legal bases - had to be developed in parallel with the project design.  

The process involved: 

• extensive drafting and iterative review 

• alignment between NHS and local authority governance teams 

• addressing mental health data sensitivity 

• ensuring pseudonymisation and privacy protections were robust 

• These steps took time and created long periods without obvious progress. Proactive communication and reassurance helped maintain stakeholder engagement throughout. 

3. Linking ASC and mental health records 

Researchers at KCL, HDRC Lambeth and SLaM collaborated to match council mental health records to SLaM mental health records within the secure Clinical Record Interactive Search (CRIS) environment. The project was structured in phases. 

• Phase 1 used a simplified specification to establish the linkage and produce a feasible patient-level extract for those known to SLaM specialist services and matched in the Lambeth social care records. The phase 1 specification focused on core variables to test infrastructure and enable early analysis, including demographics, social care history, timing, deprivation measures, SLaM referral and diagnosis data, and summary indicators of service use (e.g. admissions, referrals, and contacts). 

• All data was pseudonymised, no identifiable information left the originating systems. This provided a practical foundation for testing linkage processes, validating outputs, and building a shared understanding of what the linked data could support analytically. 

• Phase 2 will explore an expanded specification with additional variables and greater depth, allowing the team to move beyond infrastructure testing towards targeted analyses of pathways, service use and outcomes across ASC and mental health services. 

Impact  

The project has produced the UK's first linked ASC-mental health dataset of this scale, creating joint evidence base for improving services. For the first time, teams can: 

• see full care pathways across both systems 

• identify gaps or duplication 

• explore escalation patterns 

• analyse inequalities in access and outcomes 

• design more proactive and integrated responses 

It has also strengthened cross sector collaboration by creating:  

• a defined linked cohort 

• an agreed research protocol 

• a shared analytical plan 

• clearer governance and repeatable processes 

These achievements have strengthened relationships between council analysts, NHS researchers, operational teams, and academic partners. 

The project has also created a model of working that centres transparency, representation, and resident voice. Community engagement helped shape the research questions, strengthened narrative framing, and supported public trust throughout the data linkage.  

The learning from the project and models of working could be used to link other data sets in future, including those for public health research. 

Learning 

• Governance: New linkages take time and require iteration and joint design. Governance documents and research protocols benefit from being built together. 

• Technical: Operational colleagues are essential to interpreting ASC system fields. Starting with a simplified extract accelerates testing and infrastructure validation. 

• Organisational: Differing priorities and approval processes slow timelines. Sustained communication prevents loss of stakeholder engagement. Scheduling is challenging across small teams with shared pressures. 

• Cultural: Mental health data carries intrinsic sensitivity; even anonymised data can feel intrusive. Residents value transparency, a clearly stated purpose for the work, and representation. Partners must take steps to bridge academic, local authority, and NHS cultures. 

This learning story was prepared with support from NIHR RSS Specialist Centre for Public Health delivered by Newcastle University and Partners. With thanks to HDRC Lambeth for providing the content.