Analyzing item popularity bias of music recommender systems: Are different genders equally affected?

Oleg Lesota, Alessandro Melchiorre, Navid Rekabsaz, Stefan Brandl, Dominik Kowald, Elisabeth Lex, Markus Schedl*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


Several studies have identified discrepancies between the popularity of items in user profiles and the corresponding recommendation lists. Such behavior, which concerns a variety of recommendation algorithms, is referred to as popularity bias. Existing work predominantly adopts simple statistical measures, such as the difference of mean or median popularity, to quantify popularity bias. Moreover, it does so irrespective of user characteristics other than the inclination to popular content. In this work, in contrast, we propose to investigate popularity differences (between the user profile and recommendation list) in terms of median, a variety of statistical moments, as well as similarity measures that consider the entire popularity distributions (Kullback-Leibler divergence and Kendall's t rank-order correlation). This results in a more detailed picture of the characteristics of popularity bias. Furthermore, we investigate whether such algorithmic popularity bias affects users of different genders in the same way. We focus on music recommendation and conduct experiments on the recently released standardized LFM-2b dataset, containing listening profiles of users. We investigate the algorithmic popularity bias of seven common recommendation algorithms (five collaborative filtering and two baselines). Our experiments show that (1) the studied metrics provide novel insights into popularity bias in comparison with only using average differences, (2) algorithms less inclined towards popularity bias amplification do not necessarily perform worse in terms of utility (NDCG), (3) the majority of the investigated recommenders intensify the popularity bias of the female users.

Original languageEnglish
Title of host publicationRecSys 2021 - 15th ACM Conference on Recommender Systems
PublisherAssociation of Computing Machinery
Number of pages6
ISBN (Electronic)9781450384582
Publication statusPublished - 13 Sept 2021
Event15th ACM Conference on Recommender Systems: RECSYS 2021 - Amsterdam, Netherlands
Duration: 27 Sept 20211 Oct 2021


Conference15th ACM Conference on Recommender Systems
Abbreviated titleRECSYS 2021


  • Fairness
  • Gender
  • Music recommendation
  • Popularity bias

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Networks and Communications
  • Hardware and Architecture

Cite this