PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care

Maria Alejandra Pinero de Plaza; Kristina Lambrakis; Fernando Marmolejo Ramos; Alline Beleigoli; Robyn Clark; Penelope McMillan; Erin Morton; Ehsan Khan; Renuka Visvanathan; Derek Chew; Alison Kitson; Jeroen Hendriks; Carlos Barrera-Causil

doi:10.1016/j.hlc.2023.06.833

Abstract

Objective
Bring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].

Methods
PROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.

Results
PROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).

Conclusions
PROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

Original language	English
Pages (from-to)	S364-S365
Number of pages	2
Journal	Heart, Lung and Circulation
Volume	32
Issue number	Supplement 3
DOIs	https://doi.org/10.1016/j.hlc.2023.06.833
Publication status	Published - 5 Aug 2023
Event	71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand - Adelaide Convention Centre, Adelaide, Australia Duration: 3 Aug 2023 → 6 Aug 2023 Conference number: 71st https://tcc.eventsair.com/QuickEventWebsitePortal/csanz-annual-scientific-meeting-2023-and-anzet-meeting-2023/program/Portal/Closed

Keywords

Digital Health
Evaluation
prediction models
impact assessment
Co-design
Implementation Science

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.hlc.2023.06.833Licence: In Copyright

ACvA CV Champion
Pinero de Plaza, M. A. (Recipient), 29 Aug 2022
Prize: Other distinction

File
A winner of the Big Ideas video competition to improve the Australian health system. Consumer Health Forum of Australia, 2021.
Pinero de Plaza, A. (Recipient), 23 Feb 2021
Prize: Other distinction

File
CSIRO On Prime Innovation Reward 2024
Pinero de Plaza, M. A. (Recipient), Marmolejo Ramos, F. (Recipient) & Morton, E. (Recipient), 20 Nov 2024
Prize

8 Invited talk
1 Consultancy
1 Other

Aged & Extended Care Unit Academic Meeting - Monday 17/2/25: Advancing Human-Centred AI in Emergency Care: A Multimethod Evaluation of RAPIDx AI Using the PROLIFERATE_AI Framework
Pinero de Plaza, M. A. (Speaker)
17 Feb 2025
Activity: Talk or presentation types › Invited talk

File
Research to Value | Flinders University & CSIRO | Guest Speaker
Pinero de Plaza, M. A. (Speaker)
25 Feb 2025
Activity: Talk or presentation types › Invited talk
Global Impact and Recognition of the PROLIFERATE_AI Method
Pinero de Plaza, M. A. (Participant), Lambrakis, K. (Participant), Marmolejo Ramos, F. (Participant), Beleigoli, A. (Participant), Archibald, M. (Participant), Yadav, L. (Participant), McMillan, P. (Participant), Clark, R. (Participant), Lawless, M. (Participant), Morton, E. (Participant), Hendriks, J. (Participant), Kitson, A. (Participant), Visvanathan, R. (Participant), Chew, D. (Participant) & Javier Barrera Causil, C. (Participant)
Jan 2025 → Mar 2025
Activity: Other activity types › Other

8 Article
3 Poster
2 Abstract
2 Meeting Abstract
More

A Systems-Aware Evaluation Approach for Innovation
Pinero de Plaza, D. M. A., 7 May 2025, Springer Nature.
Research output: Other contribution

Open Access
From Promise to Practice: How Health Researchers Understand and Promote Transdisciplinary Collaboration
Lawless, M. T., Tieu, M., Archibald, M. B., Pinero de Plaza, M. A. & Kitson, A. L., Jan 2025, In: Qualitative Health Research. 35, 1, p. 3-16 14 p.
Research output: Contribution to journal › Article › peer-review

Open Access
File
5 Citations (Scopus)

116 Downloads (Pure)
Human-centred AI for emergency cardiac care: Evaluating RAPIDx AI with PROLIFERATE_AI
Pinero de Plaza, M. A., Lambrakis, K., Marmolejo-Ramos, F., Beleigoli, A., Archibald, M., Yadav, L., McMillan, P., Clark, R., Lawless, M., Morton, E., Hendriks, J., Kitson, A., Visvanathan, R., Chew, D. P. & Barrera Causil, C. J., Apr 2025, In: INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS. 196, 10 p., 105810.
Research output: Contribution to journal › Article › peer-review

Open Access
File
1 Citation (Scopus)

27 Downloads (Pure)

Cite this

Pinero de Plaza, M. A., Lambrakis, K., Marmolejo Ramos, F., Beleigoli, A., Clark, R., McMillan, P., Morton, E., Khan, E., Visvanathan, R., Chew, D., Kitson, A., Hendriks, J., & Barrera-Causil, C. (2023). PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care. Heart, Lung and Circulation, 32(Supplement 3), S364-S365. https://doi.org/10.1016/j.hlc.2023.06.833

@article{a97eeb6dd8fd40fba3cb53d6341fd3ec,

title = "PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care",

abstract = "ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users{\textquoteright} acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.",

keywords = "Digital Health, Evaluation, prediction models, impact assessment, Co-design, Implementation Science",

author = "{Pinero de Plaza}, {Maria Alejandra} and Kristina Lambrakis and {Marmolejo Ramos}, Fernando and Alline Beleigoli and Robyn Clark and Penelope McMillan and Erin Morton and Ehsan Khan and Renuka Visvanathan and Derek Chew and Alison Kitson and Jeroen Hendriks and Carlos Barrera-Causil",

year = "2023",

month = aug,

day = "5",

doi = "10.1016/j.hlc.2023.06.833",

language = "English",

volume = "32",

pages = "S364--S365",

journal = "Heart, Lung and Circulation",

issn = "1443-9506",

publisher = "Elsevier Australia",

number = "Supplement 3",

note = "71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand, CSANZ 2023 ; Conference date: 03-08-2023 Through 06-08-2023",

url = "https://tcc.eventsair.com/QuickEventWebsitePortal/csanz-annual-scientific-meeting-2023-and-anzet-meeting-2023/program/Portal/Closed",

}

Pinero de Plaza, MA, Lambrakis, K, Marmolejo Ramos, F , Beleigoli, A , Clark, R, McMillan, P, Morton, E, Khan, E, Visvanathan, R, Chew, D, Kitson, A , Hendriks, J & Barrera-Causil, C 2023, 'PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care', Heart, Lung and Circulation, vol. 32, no. Supplement 3, pp. S364-S365. https://doi.org/10.1016/j.hlc.2023.06.833

PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care. / Pinero de Plaza, Maria Alejandra; Lambrakis, Kristina; Marmolejo Ramos, Fernando et al.
In: Heart, Lung and Circulation, Vol. 32, No. Supplement 3, 05.08.2023, p. S364-S365.

Research output: Contribution to journal › Meeting Abstract › peer-review

TY - JOUR

T1 - PROLIFERATE_AI

T2 - 71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand

AU - Pinero de Plaza, Maria Alejandra

AU - Lambrakis, Kristina

AU - Marmolejo Ramos, Fernando

AU - Beleigoli, Alline

AU - Clark, Robyn

AU - McMillan, Penelope

AU - Morton, Erin

AU - Khan, Ehsan

AU - Visvanathan, Renuka

AU - Chew, Derek

AU - Kitson, Alison

AU - Hendriks, Jeroen

AU - Barrera-Causil, Carlos

N1 - Conference code: 71st

PY - 2023/8/5

Y1 - 2023/8/5

N2 - ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

AB - ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

KW - Digital Health

KW - Evaluation

KW - prediction models

KW - impact assessment

KW - Co-design

KW - Implementation Science

U2 - 10.1016/j.hlc.2023.06.833

DO - 10.1016/j.hlc.2023.06.833

M3 - Meeting Abstract

SN - 1443-9506

VL - 32

SP - S364-S365

JO - Heart, Lung and Circulation

JF - Heart, Lung and Circulation

IS - Supplement 3

Y2 - 3 August 2023 through 6 August 2023

ER -

PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care

Abstract

Keywords

UN SDGs

Access to Document

Fingerprint

Prizes

Activities

Research output

Cite this