Adaptive Shielding under Uncertainty.

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

This paper targets control problems that exhibit specific safety and performance requirements. In particular, the aim is to ensure that an agent, operating under uncertainty, will at runtime strictly adhere to such requirements. Previous works create so-called shields that correct an existing controller for the agent if it is about to take unbearable safety risks. However, so far, shields do not consider that an environment may not be fully known in advance and may evolve for complex control and learning tasks. We propose a new method for the efficient computation of a shield that is adaptive to a changing environment. In particular, we base our method on problems that are sufficiently captured by potentially infinite Markov decision processes (MDP) and quantitative specifications such as mean payoff objectives. The shield is independent of the controller, which may, for instance, take the form of a high-performing reinforcement learning agent. At runtime, our method builds an internal abstract representation of the MDP and constantly adapts this abstraction and the shield based on observations from the environment. We showcase the applicability of our method via an urban traffic control problem.

Original languageEnglish
Title of host publication2021 American Control Conference, ACC 2021
Pages3467-3474
Number of pages8
ISBN (Electronic)9781665441971
DOIs
Publication statusPublished - 25 May 2021
Event2021 American Control Conference: ACC 2021 - Virtual, New Orleans, United States
Duration: 25 May 202128 May 2021

Publication series

NameProceedings of the American Control Conference
Volume2021-May
ISSN (Print)0743-1619

Conference

Conference2021 American Control Conference
Country/TerritoryUnited States
CityVirtual, New Orleans
Period25/05/2128/05/21

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Adaptive Shielding under Uncertainty.'. Together they form a unique fingerprint.

Cite this