TY - JOUR
T1 - Groundwater modelling reports fail to comply with guideline recommendations for model reproducibility
AU - Kamali Maskooni, Ehsan
AU - Werner, Adrian D.
AU - Solórzano-Rivas, S. Cristina
PY - 2024/3
Y1 - 2024/3
N2 - Computer models are routinely used to underpin critical decision-making for projects that impact groundwater systems. Modelling results are communicated through technical reports, which advise regulators and other stakeholders of groundwater impacts, thereby informing approvals, project restrictions and monitoring requirements. Several guidelines and texts are available to instruct groundwater model development and reporting. In seven of the eight guidelines/texts reviewed, it is recommended that modelling reports (or a model archive) contain sufficient information for an external party to rebuild the model. This study examined that expectation (assumed to be “best practice”) by reviewing 25 groundwater modelling reports from eight countries and assessing whether the information contained therein was sufficient (or an archive was provided) to rebuild the model on which the report was based. The reports were characterised based on 18 model components (e.g., aquifer properties, boundary conditions, etc.), and the availability of sufficient information in the report to rebuild each one. The “rebuildability” of model components was classified as: (a) reproducible (from the report), (b) reproducible but assumptions needed, and (c) not reproducible. The Analytical Hierarchical Process was employed to rank the reports based on the reproducibility of the models they describe. Only one of the 25 reports provided adequate information to rebuild the model, while one other report was accompanied by a model archive, resulting in two cases of model reproducibility, contrary to guideline recommendations. This outcome reflects problems with reproducibility in the wider scientific community. We conclude that modelling reports need to provide more detailed information to be compliant with best practice or model archives ought to be made available. Addressing this issue will ensure that stakeholders have access to the information needed to properly assess whether future groundwater impacts have been reliably evaluated.
AB - Computer models are routinely used to underpin critical decision-making for projects that impact groundwater systems. Modelling results are communicated through technical reports, which advise regulators and other stakeholders of groundwater impacts, thereby informing approvals, project restrictions and monitoring requirements. Several guidelines and texts are available to instruct groundwater model development and reporting. In seven of the eight guidelines/texts reviewed, it is recommended that modelling reports (or a model archive) contain sufficient information for an external party to rebuild the model. This study examined that expectation (assumed to be “best practice”) by reviewing 25 groundwater modelling reports from eight countries and assessing whether the information contained therein was sufficient (or an archive was provided) to rebuild the model on which the report was based. The reports were characterised based on 18 model components (e.g., aquifer properties, boundary conditions, etc.), and the availability of sufficient information in the report to rebuild each one. The “rebuildability” of model components was classified as: (a) reproducible (from the report), (b) reproducible but assumptions needed, and (c) not reproducible. The Analytical Hierarchical Process was employed to rank the reports based on the reproducibility of the models they describe. Only one of the 25 reports provided adequate information to rebuild the model, while one other report was accompanied by a model archive, resulting in two cases of model reproducibility, contrary to guideline recommendations. This outcome reflects problems with reproducibility in the wider scientific community. We conclude that modelling reports need to provide more detailed information to be compliant with best practice or model archives ought to be made available. Addressing this issue will ensure that stakeholders have access to the information needed to properly assess whether future groundwater impacts have been reliably evaluated.
KW - Computer model
KW - Hydrogeology
KW - Industry best practice
KW - Water management
KW - Water policy
UR - http://www.scopus.com/inward/record.url?scp=85186495017&partnerID=8YFLogxK
UR - http://purl.org/au-research/grants/ARC/LP190100713
U2 - 10.1016/j.jenvman.2024.120292
DO - 10.1016/j.jenvman.2024.120292
M3 - Article
C2 - 38422845
AN - SCOPUS:85186495017
SN - 0301-4797
VL - 355
JO - Journal of Environmental Management
JF - Journal of Environmental Management
M1 - 120292
ER -