PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care

Maria Alejandra Pinero de Plaza; Kristina Lambrakis; Fernando Marmolejo-Ramos; Alline Beleigoli; Robyn Clark; Penelope McMillan; Erin Morton; Ehsan Khan; Renuka Visvanathan; Derek Chew; Alison Kitson; Jeroen Hendriks; Carlos Barrera-Causil

doi:10.1016/j.hlc.2023.06.833

Abstract

Objective
Bring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].

Methods
PROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.

Results
PROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).

Conclusions
PROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

Original language	English
Pages (from-to)	S364-S365
Number of pages	2
Journal	Heart, Lung and Circulation
Volume	32
Issue number	Supplement 3
DOIs	https://doi.org/10.1016/j.hlc.2023.06.833
Publication status	Published - 5 Aug 2023
Event	71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand - Adelaide Convention Centre, Adelaide, Australia Duration: 3 Aug 2023 → 6 Aug 2023 Conference number: 71st https://tcc.eventsair.com/QuickEventWebsitePortal/csanz-annual-scientific-meeting-2023-and-anzet-meeting-2023/program/Portal/Closed

Keywords

Digital Health
Evaluation
prediction models
impact assessment
Co-design
Implementation Science

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.hlc.2023.06.833Licence: In Copyright

ACvA CV Champion
Pinero de Plaza, M. A. (Recipient), 29 Aug 2022
Prize: Other distinction

File
A winner of the Big Ideas video competition to improve the Australian health system. Consumer Health Forum of Australia, 2021.
Pinero de Plaza, A. (Recipient), 23 Feb 2021
Prize: Other distinction

File
CSIRO On Prime Innovation Reward 2024
Pinero de Plaza, M. A. (Recipient), Marmolejo Ramos, F. (Recipient) & Morton, E. (Recipient), 20 Nov 2024
Prize

5 Invited talk
1 Consultancy

PROLIFERATE_AI: Leading the Future of Innovation with AI, Predictive Modelling, and Ethical User-Centred Design
Pinero de Plaza, M. A. (Consultant), Marmolejo Ramos, F. (Consultant), Lambrakis, K. (Advisor), Beleigoli, A. (Consultant), Morton, E. (Consultant), Archibald, M. (Advisor), Lawless, M. (Advisor), McMillan, P. (Advisor), Ambagtsheer, R. C. (Advisor), Mudd, A. (Consultant), Yadav, L. (Consultant), Clark, R. (Advisor), Visvanathan, R. (Advisor), Chipchase, L. (Advisor), Jacobs, D. (Advisor), Hendriks, J. (Consultant), Javier Barrera Causil, C. (Consultant), Chew, D. (Advisor) & Kitson, A. (Consultant)
2024 → 2045
Activity: Consultancy types › Consultancy
A Human-Machine Evaluation of AI in Cardiac Emergencies
Pinero de Plaza, M. A. (Invited speaker), Lambrakis, K. (Speaker), Marmolejo-Ramos, F. (Speaker), Beleigoli, A. (Speaker), Kitson, A. (Speaker), McMillan, P. (Speaker), Chew, D. (Speaker), Clark, R. (Speaker), Lawless, M. (Speaker), Morton, E. (Speaker), Archibald, M. (Speaker), Visvanathan, R. (Speaker), Yadav, L. (Speaker), Hendriks, J. (Speaker) & Javier Barrera Causil, C. (Speaker)
11 Jun 2024
Activity: Talk or presentation types › Invited talk
HTSA's Consumer Engagement Summit Panel Session
Pinero de Plaza, M. A. (Speaker)
24 Oct 2023
Activity: Talk or presentation types › Invited talk

7 Article
3 Poster
2 Abstract
2 Meeting Abstract
More
- 2 Review article
- 1 Chapter

From Promise to Practice: How Health Researchers Understand and Promote Transdisciplinary Collaboration
Lawless, M. T., Tieu, M., Archibald, M. B., Pinero de Plaza, M. A. & Kitson, A. L., Jan 2025, In: Qualitative Health Research. 35, 1, p. 3-16 14 p.
Research output: Contribution to journal › Article › peer-review

Open Access
File
3 Citations (Scopus)

95 Downloads (Pure)
A human-centered approach to measuring the impact of evidence-based online resources
Pinero de Plaza, M. A., Archibald, M., Lawless, M., Ambagtsheer, R. C., McMillan, P., Mudd, A., Freeling, M. & Kitson, A., 2024, Studies in Health Technology and Informatics: MEDINFO 2023 — The Future Is Accessible (Proceedings of the 19th World Congress on Medical and Health Informatics).. Bichel-Findlay, J., Otero, P., Scott, P. & Huesing, E. (eds.). IOS Press, Vol. 310. p. 389-393 5 p. (Studies in health technology and informatics).
Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Open Access
File
1 Citation (Scopus)

424 Downloads (Pure)
Clinical Effectiveness and Implementation Evaluation of a Large-Scale Translation Co-Designed Model on Cardiac Rehabilitation Attendance and Completion in Rural Australia Authors
Beleigoli, A., Gebremichael, L., Bulamu, N., Gulyani, A., Chew, D., Nicholls, S., Ramos, J., Maher, C., Beltrame, J., Kaambwa, B., Versace, V., Hendriks, J., Tavella, R., Foote, J., Nesbitt, K., Powell, S., Pinero de Plaza, M., Keech, W., Ludlow, M. & Maeder, A. & 7 others, Wanguhu, K., Meyer, H., Prichard, I., Suebkinorn, O., Zeitz, C., Brown, A. & Clark, R., Aug 2024, In: Heart Lung and Circulation. 33, Supplement 4, p. S226-S227 2 p.
Research output: Contribution to journal › Meeting Abstract › peer-review

Cite this

Pinero de Plaza, M. A., Lambrakis, K., Marmolejo-Ramos, F., Beleigoli, A., Clark, R., McMillan, P., Morton, E., Khan, E., Visvanathan, R., Chew, D., Kitson, A., Hendriks, J., & Barrera-Causil, C. (2023). PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care. Heart, Lung and Circulation, 32(Supplement 3), S364-S365. https://doi.org/10.1016/j.hlc.2023.06.833

@article{a97eeb6dd8fd40fba3cb53d6341fd3ec,

title = "PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care",

abstract = "ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users{\textquoteright} acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.",

keywords = "Digital Health, Evaluation, prediction models, impact assessment, Co-design, Implementation Science",

author = "{Pinero de Plaza}, {Maria Alejandra} and Kristina Lambrakis and Fernando Marmolejo-Ramos and Alline Beleigoli and Robyn Clark and Penelope McMillan and Erin Morton and Ehsan Khan and Renuka Visvanathan and Derek Chew and Alison Kitson and Jeroen Hendriks and Carlos Barrera-Causil",

year = "2023",

month = aug,

day = "5",

doi = "10.1016/j.hlc.2023.06.833",

language = "English",

volume = "32",

pages = "S364--S365",

journal = "Heart, Lung and Circulation",

issn = "1443-9506",

publisher = "Elsevier Australia",

number = "Supplement 3",

note = "71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand, CSANZ 2023 ; Conference date: 03-08-2023 Through 06-08-2023",

url = "https://tcc.eventsair.com/QuickEventWebsitePortal/csanz-annual-scientific-meeting-2023-and-anzet-meeting-2023/program/Portal/Closed",

}

Pinero de Plaza, MA, Lambrakis, K, Marmolejo-Ramos, F, Beleigoli, A , Clark, R, McMillan, P, Morton, E, Khan, E, Visvanathan, R, Chew, D, Kitson, A , Hendriks, J & Barrera-Causil, C 2023, 'PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care', Heart, Lung and Circulation, vol. 32, no. Supplement 3, pp. S364-S365. https://doi.org/10.1016/j.hlc.2023.06.833

PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care. / Pinero de Plaza, Maria Alejandra; Lambrakis, Kristina; Marmolejo-Ramos, Fernando et al.
In: Heart, Lung and Circulation, Vol. 32, No. Supplement 3, 05.08.2023, p. S364-S365.

Research output: Contribution to journal › Meeting Abstract › peer-review

TY - JOUR

T1 - PROLIFERATE_AI

T2 - 71st Annual Scientific Meeting of the Cardiac Society of Australia and New Zealand

AU - Pinero de Plaza, Maria Alejandra

AU - Lambrakis, Kristina

AU - Marmolejo-Ramos, Fernando

AU - Beleigoli, Alline

AU - Clark, Robyn

AU - McMillan, Penelope

AU - Morton, Erin

AU - Khan, Ehsan

AU - Visvanathan, Renuka

AU - Chew, Derek

AU - Kitson, Alison

AU - Hendriks, Jeroen

AU - Barrera-Causil, Carlos

N1 - Conference code: 71st

PY - 2023/8/5

Y1 - 2023/8/5

N2 - ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

AB - ObjectiveBring a method to predict the implementation impact of an Artificial Intelligence (AI) tool by modelling its end-users’ acceptance and effect (pre- and post-training about RAPIDx AI: (A set of computer algorithms in hospital emergency departments designed to help doctors provide better care for patients with symptoms that may relate to their heart) [1].MethodsPROLIFERATE (An adaptable framework with tools to evaluate different processes, outputs, and products via participatory research) [2-4] is combined with Bayesian Statistics (Expert Knowledge Elicitation) [5] to estimate the parameters of the probability model that best fit the data via maximum likelihood, using the fitdist function of the fitdistrplus package of the R software.ResultsPROLIFERATE_AI, formulates a quantifiable framework (and an algorithm) for informing the implementation of RAPIDx_AI and its impact, providing 95% probability prediction via point and interval estimates about the proportion of people who would use, understand, enjoy, keep using, and prefer RAPIDx AI. This simulation compares trained versus untrained end-users (Figure 1).ConclusionsPROLIFERATE_AI, considers the non-linear characteristics of complex and adaptive acute care environments from an end-user perspective. This predictive procedure will monitor real-world clinical settings to address end-user-centric goals around better cardiac care through AI utilisation within 12 emergency departments in Australia. It will measure AI fitness via person-centred parameters. The forthcoming non-simulated evaluation has received Ethics approval (SACHREC): OfR no.272.20.

KW - Digital Health

KW - Evaluation

KW - prediction models

KW - impact assessment

KW - Co-design

KW - Implementation Science

U2 - 10.1016/j.hlc.2023.06.833

DO - 10.1016/j.hlc.2023.06.833

M3 - Meeting Abstract

SN - 1443-9506

VL - 32

SP - S364-S365

JO - Heart, Lung and Circulation

JF - Heart, Lung and Circulation

IS - Supplement 3

Y2 - 3 August 2023 through 6 August 2023

ER -

PROLIFERATE_AI: A prediction modeling method to evaluate Artificial Intelligence in meeting end-user-centric goals around better cardiac care

Abstract

Keywords

UN SDGs

Access to Document

Fingerprint

Prizes

Activities

Research output

Cite this