Model Validation: Has it's Time Come?

There is broad agreement that effort should be made to validate cost-effectiveness models. The International Society for Pharmacoeconomics and Outcomes Research–Society for Medical Decision Making (ISPOR–SMDM) Modeling Good Research Practices Task Force considered model validation to be ‘vital’, while recognising that it is “not possible to specify criteria that a model must meet to be declared “valid”” [1] (page 736).

Guidelines for submissions to reimbursement agencies make common references to model validation but provide limited guidance on expectations regarding the application of alternative validation approaches. National Institute for Health and Care Excellence (NICE) guidelines request that sponsors provide the rationale for the chosen validation methods but provide no further guidance other than to note that sponsors should consider if and why presented results differ from the published literature [2]. Canadian guidelines describe alternative validation approaches, that the validation process should be documented, and ideally undertaken by ‘someone impartial’ [3]. In Australia, current Pharmaceutical Benefits Advisory Committee (PBAC) guidelines note that sponsors should “Consider developing and presenting any approaches to validate the results of a modelled economic evaluation” [4]. More specifically, the PBAC guidelines request that sponsors compare “model traces that correspond with observed or empirical data (e.g. overall survival or partitioned survival) as a means of validating the model”.