Projects per year
Abstract
Large Language Models (LLMs) like GPT-4o are of growing interest. Interfaces such as ChatGPT invite an ever-growing number of people to ask questions, including health advice, which brings in additional risks for harm. It is well known that tools based on LLMs tend to hallucinate or deliver different answers for the same or similar questions. In both cases, the outcome might be wrong or incomplete, possibly leading to safety issues. In this paper, we investigate the outcome of ChatGPT when we ask similar questions in the medical domain. In particular, we suggest using combinatorial testing to generate variants of questions aimed at identifying wrong or misleading answers. In detail, we discuss the general framework and its parts and present a proof-of-concept utilizing a medical query and ChatGPT.
Original language | English |
---|---|
Title of host publication | Proceedings of the 27th International Multiconference Information Society – IS 2024 |
Chapter | K |
Pages | 930-935 |
Number of pages | 6 |
DOIs | |
Publication status | Published - Oct 2024 |
Event | 27th International Multiconference Information Society – IS 2024 - Ljubljana, Slovenia Duration: 7 Oct 2024 → 11 Oct 2024 |
Conference
Conference | 27th International Multiconference Information Society – IS 2024 |
---|---|
Abbreviated title | IS 2024 |
Country/Territory | Slovenia |
City | Ljubljana |
Period | 7/10/24 → 11/10/24 |
Keywords
- Large Language Models
- ChatGPT
- Prompt Engineering
- Combinatorial Testing
- Validation
ASJC Scopus subject areas
- Artificial Intelligence
Fields of Expertise
- Information, Communication & Computing
Projects
- 1 Active
-
EU - ChatMED - Bridging Research Institutions to Catalyze Generative AI Adoption by the Health Sector in the Widening Countries
Wotawa, F. (Co-Investigator (CoI))
1/09/24 → 31/08/27
Project: Research project
Activities
- 1 Talk at conference or symposium
-
Using Combinatorial Testing for Prompt Engineering of LLMs in Medicine
Perko, A. (Speaker)
9 Oct 2024Activity: Talk or presentation › Talk at conference or symposium › Science to science
Prizes
-
Best Paper Award
Perko, A. (Recipient), Nica, I.-D. (Recipient) & Wotawa, F. (Recipient), Oct 2024
Prize: Prizes / Medals / Awards