‘It's like when you play the whispering game’: exploring perceptions of conversational agents as trustworthy mediators in digital wellbeing interventions

In the absence of human healthcare professional (HCP) involvement, standalone interactions between patients and AI-powered conversational agents (CAs) have raised concerns related to trust. Positioning CAs as mediators between patients and HCPs could address these concerns. However, limited research has investigated how patients and HCPs perceive the trustworthiness of CA mediators in supporting healthcare delivery. Grounded in a real-life wellbeing intervention programme, we engaged women diagnosed with cancer and cancer nurse specialists in co-design activities to elicit their perspectives on trustworthy CA mediator roles. Our findings reveal trustworthiness barriers related to recognising boundary situations, communicating effectively and efficiently, handling escalation, and protecting privacy. These barriers pose concerns across four dimensions that patients and HCPs value in safe healthcare delivery: timeliness, efficiency, autonomy, and privacy. Based on these findings, we propose considerations to support the designs of CAs as trustworthy mediators between patients and HCPs.