Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Vector Semiotic Model for Visual Question Answering
Artificial Intelligence Research Institute FRC CSC RAS, Moscow, Russia; HSE University, Moscow, Russia.ORCID iD: 0000-0003-2180-0990
HSE University, Moscow, Russia.
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Computer Science.ORCID iD: 0000-0003-0069-640x
Artificial Intelligence Research Institute FRC CSC RAS, Moscow, Russia; Moscow Institute of Physics and Technology, Moscow, Russia.ORCID iD: 0000-0002-9747-3837
2022 (English)In: Cognitive Systems Research, ISSN 2214-4366, E-ISSN 1389-0417, Vol. 71, p. 52-63Article in journal (Refereed) Published
Abstract [en]

In this paper, we propose a Vector Semiotic Model as a possible solution to the symbol grounding problem in the context of Visual Question Answering. The Vector Semiotic Model combines the advantages of a Semiotic Approach implemented in the Sign-Based World Model and Vector Symbolic Architectures. The Sign-Based World Model represents information about a scene depicted on an input image in a structured way and grounds abstract objects in an agent’s sensory input. We use the Vector Symbolic Architecture to represent the elements of the Sign-Based World Model on a computational level. Properties of a high-dimensional space and operations defined for high-dimensional vectors allow encoding the whole scene into a high-dimensional vector with the preservation of the structure. That leads to the ability to apply explainable reasoning to answer an input question. We conducted experiments are on a CLEVR dataset and show results comparable to the state of the art. The proposed combination of approaches, first, leads to the possible solution of the symbol-grounding problem and, second, allows expanding current results to other intelligent tasks (collaborative robotics, embodied intellectual assistance, etc.).

Place, publisher, year, edition, pages
Elsevier, 2022. Vol. 71, p. 52-63
Keywords [en]
Vector-symbolic architecture, Semiotic approach, Symbol grounding problem, Causal network, Visual Question Answering
National Category
Computer Sciences
Research subject
Dependable Communication and Computation Systems
Identifiers
URN: urn:nbn:se:ltu:diva-87680DOI: 10.1016/j.cogsys.2021.09.001ISI: 000721354800002Scopus ID: 2-s2.0-85118894608OAI: oai:DiVA.org:ltu-87680DiVA, id: diva2:1606771
Note

Validerad;2021;Nivå 2;2021-11-10 (beamah);

Forskningsfinansiär: Russian Science Foundation (20-71-10116)

Available from: 2021-10-28 Created: 2021-10-28 Last updated: 2021-11-29Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Osipov, Evgeny

Search in DiVA

By author/editor
Kovalev, Alexey K.Osipov, EvgenyPanov, Aleksandr I.
By organisation
Computer Science
In the same journal
Cognitive Systems Research
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 129 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf