Evaluating the impact of online health information: A scoping review of measures

Background: High-quality, reputable online health information (OHI) offers a cost-effective and accessible means to enhance health professionals' knowledge and empower individuals to make informed health decisions. As digital health resources proliferate, there is growing pressure on OHI producers and providers to demonstrate the value and effectiveness of their offerings. However, current evaluation practices of OHI often rely on engagement and usability metrics, which fail to capture broader, meaningful outcomes such as knowledge acquisition, behaviour change, or organisational or societal benefits. Without more robust and aligned evaluation frameworks, the effectiveness of OHI may be overstated, leading to misinformed investment decisions and missed opportunities to improve health literacy and care delivery.

Objective: To systematically map how the impact of OHI is defined, assessed, and measured in the published literature, including study designs, measurement tools, and the levels of impact evaluated (micro, meso, macro).

Design: Scoping review conducted in accordance with the Arksey and O’Malley framework and reported using the PRISMA Extension for Scoping Reviews (PRISMA-ScR).

Data sources: Systematic search of Ovid Medline, CINAHL, and the full Journal of Medical Internet Research suite of journals, covering publications from 1 January 2013 to 18 May 2023.

Eligibility criteria: Primary research studies in English evaluating the impact of OHI resources— including websites, knowledge hubs, toolkits, and elearning resources—accessed independently and asynchronously by end-users. Impact was defined as demonstrable benefits to individuals, organisations, or societies causally linked to engagement with OHI.

Data extraction and charting: Two reviewers independently extracted data using a structured template, capturing study design, OHI topic, intended audience, terminology for ‘impact’, and impact evaluation measures. Findings were synthesised narratively and supported by descriptive statistics.