The Difficulty of Novelty Detection and Adaptation in Physical Environments

Vimukthini Pinto, Chathura Gamage, Matthew Stephenson, Jochen Renz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Detecting and adapting to novel situations is a major challenge for AI systems that operate in open-world environments. One reason for this challenge is due to the diverse range of forms that novelties can take. To accurately evaluate an AI system’s ability to detect and adapt to novelties, it is crucial to investigate and formalize the difficulty of different novelty types. In this paper, we propose a method for quantifying the difficulty of novelty detection and novelty adaptation in open-world physical environments, considering factors such as the appearance and location of objects, as well as the actions required by the agent. We implement several difficulty measures using a combination of qualitative spatial relations, learning algorithms, and statistical distance measures. To demonstrate an application of our approach, we apply our difficulty measures to novelties in the popular physics simulation game Angry Birds. We invite researchers to incorporate the proposed novelty difficulty measures when evaluating AI systems to gain a better understanding of their limitations and identify areas for future improvement.

Original languageEnglish
Title of host publicationAI 2023
Subtitle of host publicationAdvances in Artificial Intelligence - 36th Australasian Joint Conference on Artificial Intelligence, AI 2023, Proceedings
EditorsTongliang Liu, Geoff Webb, Lin Yue, Dadong Wang
PublisherSpringer Science and Business Media Deutschland GmbH
Pages28-40
Number of pages13
ISBN (Electronic)978-981-99-8391-9
ISBN (Print)978-981-99-8390-2
DOIs
Publication statusPublished - 2024
Event36th Australasian Joint Conference on Artificial Intelligence, AJCAI 2023 - Brisbane, Australia
Duration: 28 Nov 20231 Dec 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14472 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference36th Australasian Joint Conference on Artificial Intelligence, AJCAI 2023
Country/TerritoryAustralia
CityBrisbane
Period28/11/231/12/23

Keywords

  • AI Evaluation
  • Difficulty
  • Novelty
  • Open-world Learning

Fingerprint

Dive into the research topics of 'The Difficulty of Novelty Detection and Adaptation in Physical Environments'. Together they form a unique fingerprint.

Cite this