Volltext-Downloads (blau) und Frontdoor-Views (grau)

Predicting Visible Terms from Image Captions using Concreteness and Distributional Semantics

  • Image captions in scientific papers usually are complementary to the images. Consequently, the captions contain many terms that do not refer to concepts visible in the image. We conjecture that it is possible to distinguish between these two types of terms in an image caption by analysing the text only. To examine this, we evaluated different features. The dataset we used to compute tf.idf values, word embeddings and concreteness values contains over 700 000 scientific papers with over 4,6 million images. The evaluation was done with a manually annotated subset of 329 images. Additionally, we trained a support vector machine to predict whether a term is a likely visible or not. We show that concreteness of terms is a very important feature to identify terms in captions and context that refer to concepts visible in images.

Download full text files

Export metadata

Statistics

frontdoor_oas
Metadaten
Author:Jean CharbonnierORCiD, Christian WartenaORCiDGND
URN:urn:nbn:de:bsz:960-opus4-23758
DOI:https://doi.org/10.25968/opus-2375
DOI original:https://doi.org/10.5220/0011351400003335
ISBN:978-989-758-614-9
ISSN:2184-3228
Parent Title (English):Proceedings of the 14th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, Volume 1: KDIR
Publisher:SciTePress
Editor:Frans Coenen, Ana Fred, Joaquim Filipe
Document Type:Conference Proceeding
Language:English
Year of Completion:2022
Publishing Institution:Hochschule Hannover
Release Date:2022/11/21
Tag:Concreteness; Distributional Semantics
GND Keyword:Information Retrieval; Legende <Bild>
First Page:161
Last Page:169
Institutes:Fakult├Ąt III - Medien, Information und Design
DDC classes:000 Informatik, Informationswissenschaft, allgemeine Werke
Licence (German):License LogoCreative Commons - CC BY-NC-ND - Namensnennung - Nicht kommerziell - Keine Bearbeitungen 4.0 International