TY - JOUR
T1 - Spreadsheet debugging
T2 - The perils of tool over-reliance
AU - Mukhtar, Adil
AU - Hofer, Birgit
AU - Jannach, Dietmar
AU - Wotawa, Franz
N1 - Funding Information:
The work described in this paper has been funded by the Austrian Science Fund (FWF) project Interactive Spreadsheet Debugging under contract number P 32445 .
Publisher Copyright:
© 2021 The Author(s)
PY - 2022/2
Y1 - 2022/2
N2 - Spreadsheets are widely used in organizations for various purposes such as data aggregation, reporting and decision-making. Since spreadsheets, like other types of software, can contain faulty formulas, it is important to provide developers with appropriate methods to find and fix such faults. Recently, various heuristic and statistics-based fault identification methods were proposed, which point developers to potentially faulty parts of the spreadsheets. Due to their heuristic nature, these methods might, however, miss some faults. As a result, if spreadsheet developers rely too strongly on these methods, they might not pay sufficient attention to problems that are not pinpointed by the methods. In this research, we are the first to study this potential problem of over-reliance in spreadsheet debugging, which may lead to limited debugging effectiveness. We report the outcome of a controlled experiment where 59 participants were tasked to find faulty formulas in a given spreadsheet with and without support of a novel spreadsheet debugging tool. Our results indicate that tool over-reliance can indeed result as a phenomenon of using heuristic debugging techniques. However, the study also provides evidence that making users aware of potential tool limitations within the debugging environment may help to address this problem. Open Science material was validated by the Journal of Systems and Software Open Science Board.
AB - Spreadsheets are widely used in organizations for various purposes such as data aggregation, reporting and decision-making. Since spreadsheets, like other types of software, can contain faulty formulas, it is important to provide developers with appropriate methods to find and fix such faults. Recently, various heuristic and statistics-based fault identification methods were proposed, which point developers to potentially faulty parts of the spreadsheets. Due to their heuristic nature, these methods might, however, miss some faults. As a result, if spreadsheet developers rely too strongly on these methods, they might not pay sufficient attention to problems that are not pinpointed by the methods. In this research, we are the first to study this potential problem of over-reliance in spreadsheet debugging, which may lead to limited debugging effectiveness. We report the outcome of a controlled experiment where 59 participants were tasked to find faulty formulas in a given spreadsheet with and without support of a novel spreadsheet debugging tool. Our results indicate that tool over-reliance can indeed result as a phenomenon of using heuristic debugging techniques. However, the study also provides evidence that making users aware of potential tool limitations within the debugging environment may help to address this problem. Open Science material was validated by the Journal of Systems and Software Open Science Board.
KW - Fault identification
KW - Over-reliance
KW - Spreadsheets
KW - User study
UR - http://www.scopus.com/inward/record.url?scp=85118112155&partnerID=8YFLogxK
U2 - 10.1016/j.jss.2021.111119
DO - 10.1016/j.jss.2021.111119
M3 - Article
AN - SCOPUS:85118112155
SN - 0164-1212
VL - 184
JO - Journal of Systems and Software
JF - Journal of Systems and Software
M1 - 111119
ER -