Development and validation of an evaluation tool of consumers’ knowledge and confidence to report patient deterioration in hospitals

Objectives
Studies have shown that many preventable hospital deaths may be reduced through early reporting of deterioration by patients and their visitors to health professionals. Engagement of patients and families for safer health care was recommended by the World Health Organization. As no validated tool was identified, the objective of the study was to develop and validate a tool to measure the impact of messages contained in self-developed educational materials on consumers’ knowledge and confidence to report patient deterioration.

Methods
A tool was created using consumer-based input. A test-retest methodology was used 2 weeks apart, using 37 participants sourced from consumer groups. Fourteen 5-point Likert scale questionnaire items were categorized into knowledge and confidence components to identify the impact of the materials in both of these areas.

Results
Test-retest reliability confirmed 10 of the 14 proposed questionnaire items (4 knowledge items and 6 confidence) had at least moderate κ agreement ranging from 0.42 to 0.69 and an intraclass correlation coefficient of 0.66. An exploratory factor analysis of the 10 remaining items indicated a 2-factor solution had the best interpretability accounting for 96.9% of variability. The Cronbach α and factor loadings for the factors, knowledge, and confidence suggested an acceptable level of reliability and validity.

Conclusions
The study confirms our tool for assessment of educational materials to be a reliable, context-specific, and validated instrument. The tool is an appropriate instrument to assess the effectiveness of educational materials in public awareness campaigns that focus on consumer reporting of patient deterioration within hospitals.