On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic

Patrick Kasper; Philipp Koncar; Tiago Filipe Teixeira dos Santos; Christian Gütl

doi:10.1109/SNAMS.2019.8931866

On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic

Patrick Kasper, Philipp Koncar, Tiago Filipe Teixeira dos Santos, Christian Gütl

Institute of Interactive Systems and Data Science (7060)

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

Evaluating others' reviews of products via helpfulness votes is a crucial aspect of purchase decision-making in online marketplaces. Previous work studied key determinant factors of review helpfulness, such as product metadata and review text. However, understanding the extent to which review helpfulness depends on product context, rather than the inherent textual value of a review, remains an open question. In this work, we study how genre, score and review text relate to the helpfulness of 319 017 video game reviews on Metacritic via correlational analyses and prediction experiments. We find significant, genre-dependent correlations between score and helpfulness. This underlies the strength of text vs. score features in our helpfulness prediction models, which achieve up to 0.64 F 1 . This work is thus a stepping stone towards causal disentanglement of product context from text in review helpfulness estimation.

Original language	English
Title of host publication	2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS)
Publisher	IEEE Xplore
Pages	75 - 82
ISBN (Electronic)	978-1-7281-2946-4
DOIs	https://doi.org/10.1109/SNAMS.2019.8931866
Publication status	Published - 16 Dec 2019
Event	2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS) - Granada, Spain Duration: 22 Oct 2018 → 25 Oct 2019 http://emergingtechnet.org/SNAMS2019/

Conference

Conference	2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS)
Country/Territory	Spain
City	Granada
Period	22/10/18 → 25/10/19
Internet address	http://emergingtechnet.org/SNAMS2019/

Access to Document

10.1109/SNAMS.2019.8931866

DiSpecs - Distant Spectators: Distant Reading for periodicals of the Enlightenmen
Helic, D.
1/06/19 → 30/11/20
Project: Research project

Cite this

On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic. / Kasper, Patrick; Koncar, Philipp; Teixeira dos Santos, Tiago Filipe et al.
2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS). IEEE Xplore, 2019. p. 75 - 82.

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Kasper, P, Koncar, P, Teixeira dos Santos, TF & Gütl, C 2019, On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic. in 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS). IEEE Xplore, pp. 75 - 82, 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), Granada, Spain, 22/10/18. https://doi.org/10.1109/SNAMS.2019.8931866

@inproceedings{7715ec6b58b3455dbf199e51064522a4,

title = "On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic",

abstract = "Evaluating others' reviews of products via helpfulness votes is a crucial aspect of purchase decision-making in online marketplaces. Previous work studied key determinant factors of review helpfulness, such as product metadata and review text. However, understanding the extent to which review helpfulness depends on product context, rather than the inherent textual value of a review, remains an open question. In this work, we study how genre, score and review text relate to the helpfulness of 319 017 video game reviews on Metacritic via correlational analyses and prediction experiments. We find significant, genre-dependent correlations between score and helpfulness. This underlies the strength of text vs. score features in our helpfulness prediction models, which achieve up to 0.64 F 1 . This work is thus a stepping stone towards causal disentanglement of product context from text in review helpfulness estimation.",

author = "Patrick Kasper and Philipp Koncar and {Teixeira dos Santos}, {Tiago Filipe} and Christian G{\"u}tl",

year = "2019",

month = dec,

day = "16",

doi = "10.1109/SNAMS.2019.8931866",

language = "English",

pages = "75 -- 82",

booktitle = "2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS)",

publisher = "IEEE Xplore",

note = "2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS) ; Conference date: 22-10-2018 Through 25-10-2019",

url = "http://emergingtechnet.org/SNAMS2019/",

}

TY - GEN

T1 - On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic

AU - Kasper, Patrick

AU - Koncar, Philipp

AU - Teixeira dos Santos, Tiago Filipe

AU - Gütl, Christian

PY - 2019/12/16

Y1 - 2019/12/16

N2 - Evaluating others' reviews of products via helpfulness votes is a crucial aspect of purchase decision-making in online marketplaces. Previous work studied key determinant factors of review helpfulness, such as product metadata and review text. However, understanding the extent to which review helpfulness depends on product context, rather than the inherent textual value of a review, remains an open question. In this work, we study how genre, score and review text relate to the helpfulness of 319 017 video game reviews on Metacritic via correlational analyses and prediction experiments. We find significant, genre-dependent correlations between score and helpfulness. This underlies the strength of text vs. score features in our helpfulness prediction models, which achieve up to 0.64 F 1 . This work is thus a stepping stone towards causal disentanglement of product context from text in review helpfulness estimation.

AB - Evaluating others' reviews of products via helpfulness votes is a crucial aspect of purchase decision-making in online marketplaces. Previous work studied key determinant factors of review helpfulness, such as product metadata and review text. However, understanding the extent to which review helpfulness depends on product context, rather than the inherent textual value of a review, remains an open question. In this work, we study how genre, score and review text relate to the helpfulness of 319 017 video game reviews on Metacritic via correlational analyses and prediction experiments. We find significant, genre-dependent correlations between score and helpfulness. This underlies the strength of text vs. score features in our helpfulness prediction models, which achieve up to 0.64 F 1 . This work is thus a stepping stone towards causal disentanglement of product context from text in review helpfulness estimation.

U2 - 10.1109/SNAMS.2019.8931866

DO - 10.1109/SNAMS.2019.8931866

M3 - Conference paper

SP - 75

EP - 82

BT - 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS)

PB - IEEE Xplore

T2 - 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS)

Y2 - 22 October 2018 through 25 October 2019

ER -

On the Role of Score, Genre and Text in Helpfulness of Video Game Reviews on Metacritic

Abstract

Conference

Access to Document

Fingerprint

Projects

DiSpecs - Distant Spectators: Distant Reading for periodicals of the Enlightenmen

Cite this