Extracting semantic knowledge from Web context for multimedia IR: A taxonomy, survey and challenges

T. Bracamonte, B. Bustos, B. Poblete, T. Schreck

Research output: Contribution to journalArticlepeer-review


Since its invention, the Web has evolved into the largest multimedia repository that has ever existed. This evolution is a direct result of the explosion of user-generated content, explained by the wide adoption of social network platforms. The vast amount of multimedia content requires effective management and retrieval techniques. Nevertheless, Web multimedia retrieval is a complex task because users commonly express their information needs in semantic terms, but expect multimedia content in return. This dissociation between semantics and content of multimedia is known as the semantic gap. To solve this, researchers are looking beyond content-based or text-based approaches, integrating novel data sources. New data sources can consist of any type of data extracted from the context of multimedia documents, defined as the data that is not part of the raw content of a multimedia file. The Web is an extraordinary source of context data, which can be found in explicit or implicit relation to multimedia objects, such as surrounding text, tags, hyperlinks, and even in relevance-feedback. Recent advances in Web multimedia retrieval have shown that context data has great potential to bridge the semantic gap. In this article, we present the first comprehensive survey of context-based approaches for multimedia information retrieval on the Web. We introduce a data-driven taxonomy, which we then use in our literature review of the most emblematic and important approaches that use context-based data. In addition, we identify important challenges and opportunities, which had not been previously addressed in this area.
Original languageEnglish
Pages (from-to)13853 - 13889
Number of pages37
JournalMultimedia Tools and Applications
Issue number11
Early online date2017
Publication statusPublished - Jun 2018

Fields of Expertise

  • Information, Communication & Computing


Dive into the research topics of 'Extracting semantic knowledge from Web context for multimedia IR: A taxonomy, survey and challenges'. Together they form a unique fingerprint.

Cite this