The Difficulty of Novelty Detection and Adaptation in Physical Environments

Vimukthini Pinto, Chathura Gamage, Matthew Stephenson, Jochen Renz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Detecting and adapting to novel situations is a major challenge for AI systems that operate in open-world environments. One reason for this challenge is due to the diverse range of forms that novelties can take. To accurately evaluate an AI system’s ability to detect and adapt to novelties, it is crucial to investigate and formalize the difficulty of different novelty types. In this paper, we propose a method for quantifying the difficulty of novelty detection and novelty adaptation in open-world physical environments, considering factors such as the appearance and location of objects, as well as the actions required by the agent. We implement several difficulty measures using a combination of qualitative spatial relations, learning algorithms, and statistical distance measures. To demonstrate an application of our approach, we apply our difficulty measures to novelties in the popular physics simulation game Angry Birds. We invite researchers to incorporate the proposed novelty difficulty measures when evaluating AI systems to gain a better understanding of their limitations and identify areas for future improvement.

Original languageEnglish
Title of host publicationAI 2023
Subtitle of host publicationAdvances in Artificial Intelligence - 36th Australasian Joint Conference on Artificial Intelligence, AI 2023, Proceedings
EditorsTongliang Liu, Geoff Webb, Lin Yue, Dadong Wang
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages13
ISBN (Electronic)978-981-99-8391-9
ISBN (Print)978-981-99-8390-2
Publication statusPublished - 2024
Event36th Australasian Joint Conference on Artificial Intelligence, AJCAI 2023 - Brisbane, Australia
Duration: 28 Nov 20231 Dec 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14472 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference36th Australasian Joint Conference on Artificial Intelligence, AJCAI 2023


  • AI Evaluation
  • Difficulty
  • Novelty
  • Open-world Learning


Dive into the research topics of 'The Difficulty of Novelty Detection and Adaptation in Physical Environments'. Together they form a unique fingerprint.

Cite this