Abstract
Conversational user interfaces open up new opportunities for reflection guidance. This paper presents a computer-mediated dialogue structure for reflecting on learning tasks, Rebo Junior, and its evaluation in the context of apprenticeship training. We answer three research questions. Firstly, how apprentices react to Rebo Junior; secondly, whether Rebo Junior’s dialogue structure is
apt to lead apprentices in reflective conversations; and thirdly, how user engagement with Rebo Junior develops over time. During three months, 17 apprentices led 153 reflective conversations with Rebo Junior in the context of a training workshop, 117 in phase one and 36 in phase two of the study (five to thirteen
interactions per apprentice). We coded interactions manually for coherence, level of reflectivity, and user engagement. Our results show that apprentices react well to the intervention and that the dialogue structure is successful in leading apprentices through different levels of reflection (114 out of 153 showed observable reflection on the learning experience; 133 out of 153 expressed learning or planned behaviour change for future tasks). Furthermore, the interactions between the apprentices and Rebo Junior result in coherent conversations (149 out of 153 were coherent). Contrary to expectations, engagement did not decrease
over time in either phase. With the present paper, we therefore publish a dialogue structure for reflecting on learning tasks that has worked extremely well despite no adaptivity in the conversational interface. Overall, we interpret the results of
our work as underscoring the importance of dialogue structure quality in conversational agents.
apt to lead apprentices in reflective conversations; and thirdly, how user engagement with Rebo Junior develops over time. During three months, 17 apprentices led 153 reflective conversations with Rebo Junior in the context of a training workshop, 117 in phase one and 36 in phase two of the study (five to thirteen
interactions per apprentice). We coded interactions manually for coherence, level of reflectivity, and user engagement. Our results show that apprentices react well to the intervention and that the dialogue structure is successful in leading apprentices through different levels of reflection (114 out of 153 showed observable reflection on the learning experience; 133 out of 153 expressed learning or planned behaviour change for future tasks). Furthermore, the interactions between the apprentices and Rebo Junior result in coherent conversations (149 out of 153 were coherent). Contrary to expectations, engagement did not decrease
over time in either phase. With the present paper, we therefore publish a dialogue structure for reflecting on learning tasks that has worked extremely well despite no adaptivity in the conversational interface. Overall, we interpret the results of
our work as underscoring the importance of dialogue structure quality in conversational agents.
Original language | English |
---|---|
Title of host publication | ECTEL 2020 Impact Paper Proceedings |
Publisher | CEUR Workshop Proceedings |
Number of pages | 14 |
Volume | 2676 |
Publication status | Published - 1 Jan 2020 |
Event | 15th European Conference on Technology-Enhanced Learning: EC-TEL-Impact 2020 - Virtuell, Heidelberg, Germany Duration: 14 Sept 2020 → 18 Sept 2020 Conference number: 15 http://www.ec-tel.eu/index.php?id=957 |
Publication series
Name | CEUR Workshop Proceedings |
---|---|
Publisher | RWTH Aachen |
ISSN (Print) | 1613-0073 |
Conference
Conference | 15th European Conference on Technology-Enhanced Learning |
---|---|
Abbreviated title | ECTEL 2020 |
Country/Territory | Germany |
City | Virtuell, Heidelberg |
Period | 14/09/20 → 18/09/20 |
Internet address |
Keywords
- reflection guidance
- conversational agent
- dialogue design
- field study
- Reflection guidance chatbot
- Dialogue structure
- Reflection guidance
- Learning technology
- Levels of reflection
- Proof of concept evaluation
ASJC Scopus subject areas
- Computer Science(all)