Synthesis for health services and policy: case studies in the scoping of reviews

Abstract

Background: For systematic reviews to be rigorous, deliverable and useful they need a well-defined review question. Scoping for a review also requires the specification of clear inclusion criteria and planned synthesis methods. Guidance is lacking on how to develop these, especially within the context of undertaking rapid and responsive systematic reviews to inform health services and health policy. Objective: This report describes and discusses the experiences of review scoping of three commissioned research centres that conducted evidence syntheses to inform health and social care organisation, delivery and policy in the UK, between 2017 and 2020. Data sources: Researcher recollection, project meeting minutes, e-mail correspondence with stakeholders, and scoping searches; from allocation of a review topic through to review protocol agreement. Methods: We produced eight descriptive case studies of selected reviews from the three teams. From these we identified key issues that shape the processes of scoping and question formulation for evidence synthesis. The issues were then discussed and lessons drawn. Findings: Across the eight diverse case studies we identified 14 recurrent issues that were important in shaping the scoping processes and formulating a review’s questions. These were ‘consultative issues’, relating to securing input from review commissioners, policy customers, experts, patients and other stakeholders: managing and deciding priorities; reconciling different priorities/perspectives; achieving buy-in and engagement; educating the end-user about synthesis processes and products, and; managing stakeholder expectations. There were ‘interface issues’ relating to the interaction between review team and potential review users: identifying the niche/gap and optimising value; assuring and balancing rigour/reliability/relevance, and; assuring transferability/applicability of study evidence to specific policy/service user contexts. There were also ‘technical issues’ associated with the methods and conduct of the review: choosing the method(s) of synthesis; balancing fixed versus fluid review questions/components/definitions; taking stock of what research already exists; Mapping vs Scoping vs Reviewing; scoping/relevance as a continuous process not just at initial stage, and; calibrating general versus specific and broad versus deep coverage of topics. Limitations: As a retrospective joint reflection by review teams on their experiences of scoping processes, this report is not based on prospectively collected research data. Also, our evaluations were not externally validated by, for example, policy and service evidence users or patients and the public. Conclusions: We have summarised our reflections on scoping from this programme of reviews as 14 common issues and 28 practical ‘lessons learned’. Effective scoping of rapid, responsive reviews extends beyond information exchange and technical procedures for specifying a ‘gap’ in the evidence. These considerations work alongside social processes, in particular the building of relationships and shared understanding between reviewers, research commissioners and potential review users that may be reflective of consultancy, negotiation, and co-production models of research and information use.