Interrogating Virtual Agents: In Quest of Security Vulnerabilities

Josip Bozic, Franz Wotawa*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


Chatbots, i.e., systems that communicate in natural language, have been of increasing importance over the last few years. These virtual agents provide specific services or products to clients on a 24/7 basis. Chatbots provide a simple and intuitive interface, i.e., natural language processing, which makes them increasingly attractive for various applications. In fact, chatbots are used as substitutes for repetitive tasks or user inquiries that can be automated. However, these advantages always are accompanied with concerns, e.g., whether security and privacy can be assured. These concerns become more and more important, because in contrast to simple requests, more sophisticated chatbots are able to utilize personalized services to users. In such cases, sensitive user data are processed and exchanged. Hence, such systems become natural targets for cyber-attacks with unforeseen consequences. For this reason, assuring information security of chatbots is an important challenge in practice. In this paper, we contribute to this challenge and introduce an automated security testing approach for chatbots. The presented framework is able to generate and run tests in order to detect intrinsic software weaknesses leading to the XSS vulnerability. We assume a vulnerability to be triggered when obtaining critical information from or crashing the virtual agent, regardless of its purpose. We discuss the underlying basic foundations and demonstrate the testing approach using several real-world chatbots.

Original languageEnglish
Title of host publicationTesting Software and Systems - 32nd IFIP WG 6.1 International Conference, ICTSS 2020, Proceedings
EditorsValentina Casola, Alessandra De Benedictis, Massimiliano Rak
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages15
ISBN (Print)9783030648800
Publication statusPublished - 1 Jan 2020
Event32nd IFIP WG 6.1 International Conference on Testing Software and Systems, ICTSS 2020 - Naples, Italy
Duration: 9 Dec 202011 Dec 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12543 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference32nd IFIP WG 6.1 International Conference on Testing Software and Systems, ICTSS 2020


  • Chatbots
  • Model-based testing
  • Security testing
  • Web applications

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Interrogating Virtual Agents: In Quest of Security Vulnerabilities'. Together they form a unique fingerprint.

Cite this