Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

Raechel A. Damarell, Suzanne Lewis, Camilla Trenerry, Jennifer J. Tieman

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)
4 Downloads (Pure)


Background: Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision. Methods: An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance. Results: The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0-88.3% recall with corresponding low precision (47-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73-95% with recall reduced to between 55.9 and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care. Conclusions: The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

Original languageEnglish
Article number12
Number of pages16
JournalBMC Medical Research Methodology
Publication statusPublished - 21 Jan 2020

Bibliographical note

© The Author(s). 2020 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.


  • Information retrieval
  • Integrated care
  • Literature searching
  • Precision
  • PubMed
  • Recall
  • Search filters


Dive into the research topics of 'Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence'. Together they form a unique fingerprint.

Cite this