Bridging formalisation and expert judgement in searches for studies for systematic reviews

Abstract

Systematic reviews aim to use pre-specified and explicitly described methods. This entails an element of formalisation in which methods are described according to a fixed structure. However, qualitative studies show that too much emphasis on formalisation can obscure how expert judgement is required even after clearly defined methods are established. Thus, there is a gap between how systematic review methods are formalised in guidance and reported in systematic reviews, and how they are carried out in practice using undisclosed expert judgement. The aim of this thesis is to describe and bridge the gap between formalisation and expert judgement with respect to searching for studies for systematic reviews, with a particular focus on forward citation searching and web searching. Forward citation searching and web searching are useful search methods to consider due to observed variability in both if and how they are used in systematic reviews, in contrast to searches of bibliographic databases which are routine in almost all systematic reviews. To this end, the thesis seeks to fulfil three objectives: first, to formalise the conduct and reporting of forward citation searching and web searching in systematic reviews; secondly, to describe and evaluate the conduct and reporting of forward citation searching and web searching in systematic reviews; thirdly, to explore the role of expert judgement when using forward citation searching and web searching. Both aggregative and configurative review types are considered throughout. The findings show that formalised approaches to searching are apparent in guidance to different degrees. However, systematic reviews do not always reflect formalised guidance. Qualitative investigation describes hitherto hidden practical knowledge which underpins searching decisions. The thesis draws these findings together and proposes that guidance on searching for studies should be framed in terms of the practical understanding which informs how searching is undertaken rather than limited to describing recommended processes.