On Benchmarking and Validation in Wargames

Adam J. Wilden, Mehwish Nasim, Peter Williams, Tim Legrand, Benjamin Peter Turnbull, Patricia A.H. Williams

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

30 Downloads (Pure)


There are multiple arguments for and against wargames. Many scientists do not recognise the science in wargames. It is suggested that there is not enough literature relating to wargaming, for there to be any large-scale research into wargames. This is primarily because scientists often refuse to publish results, thus creating a vicious cycle where research is not published because there is not enough research being published. This ultimately deters researchers from studying wargaming in any serious fashion. Owing to this limitation, published work on the results, and protocols of wargames are scarce in scholarly research. Wargaming has considerably less academic focus with a fragmented and practical focus on design and benchmarking. This is surprising given the long history of wargaming (dating back to the early 1600 s), when compared to the relatively recent history of other domains such as software engineering. To better understand the current state of research into wargaming in reference to benchmarking and validation, a scoping review (SR) was conducted. The scholarly research into wargaming reveals papers on general modelling, conflict modelling, influence modelling, evaluation of wargames, analytical tools, use of AI in wargame design, evaluation of predictive modelling in wargames, improving command and control through wargaming, and cost-benefit analysis for decision making. The initial analysis of the coverage of wargaming research, together with the limited number of papers found, indicate that there is a distinct lack of academic research into wargaming. Additionally, there is a wide variety of areas that are interested in the wargaming field, however, with no universal method of analysis or benchmarking, this limits the reproducibility of results, and the ability to judge the overall effectiveness of wargaming efforts. Wargame designers need to be able to assess wargame components to validate, compare, and predict the effects on gameplay and for decision-makers to draw conclusions with more confidence.

Original languageEnglish
Title of host publicationProceedings of the 22nd European Conference on Cyber Warfare and Security, ECCWS 2023
EditorsAntonios Andreatos, Christos Douligeris
Place of PublicationReading, UK
PublisherCurran Associates Inc
Number of pages11
ISBN (Electronic)978-1-914587-70-2
ISBN (Print)978-1-914587-69-6
Publication statusPublished - 19 Jun 2023
Event22nd European Conference on Cyber Warfare and Security, ECCWS 2023 - Athens, Greece
Duration: 22 Jun 202323 Jun 2023

Publication series

NameEuropean Conference on Information Warfare and Security, ECCWS
ISSN (Print)2048-8602
ISSN (Electronic)2048-8610


Conference22nd European Conference on Cyber Warfare and Security, ECCWS 2023


  • Benchmarking
  • Information Operations
  • Reproducibility
  • Validation
  • Wargaming


Dive into the research topics of 'On Benchmarking and Validation in Wargames'. Together they form a unique fingerprint.

Cite this