How abstract knowledge is stored in the brain is being studied by psychologists and computer scientists in the new DFG research group ARENA. The findings should contribute to making artificially intelligent (AI) systems more efficient and flexible.

Since artificially intelligent systems have been able to reliably recognise objects and language, AI research has experienced a boom. However, the systems still have to be trained with a great deal of work and energy - and still store their knowledge about objects and words differently than the human brain: modern AI systems are usually neural network models. They consist of several layers of artificial nerve cells that are interconnected. That is why they are also called deep neural networks. An AI system developed for image recognition and speech recognition can link an image of an orange (input) with the word "orange" (output). However, such an AI system cannot generalise to other sensory impressions - which our brain, on the other hand, manages effortlessly.

This is because one of the most important properties of the human brain is the ability to abstract: our knowledge about an orange can be activated when we see, feel, taste or smell it. The brain thus abstractly represents our semantic knowledge about oranges - regardless of how we perceive oranges through the senses.

AI could learn this kind of abstract knowledge representation from the human brain. However, the 'format' in which our semantic knowledge is stored in the human brain is not yet well understood. Here again, brain research can benefit from the powerful AI models. The interdisciplinary research group ARENA (Abstract Representations in Neural Architectures) at the Frankfurt Institute for Advanced Studies (FIAS), Goethe University, and the Max Planck Institute for Software Systems in Saarbrücken, funded by the German Research Foundation (DFG), builds a bridge between computer science, psychology, and neuroscience to explore these questions. It will receive a total of around 3.7 million Euros over the next four years.

An important goal of the ARENA research group is to investigate whether AI systems that are trained with data of different formats - images, language or videos, i.e. multimodal data - develop more abstract forms of knowledge or at least forms of knowledge that are more similar to the human brain. 

As part of the project, Prof. Dr. Jochen Triesch from FIAS will investigate how AI can independently learn abstract representations of objects. "In contrast to common AI approaches, it should achieve this autonomously and without outside help," says Triesch, "like a child learns to recognise objects and group them into abstract categories." His FIAS colleague Prof. Dr. Matthias Kaschube will investigate how both the brain and AIs can use "cognitive maps" to efficiently process abstract semantic information. Prof. Gemma Roig acts as a bridge professor between computer science and psychology in the research group.

The psychologists and neuroscientists involved are interested in how well AI systems can explain how the brain works when processing abstract meanings. To do this, they want to compare how an AI system and the human brain work when they solve the same tasks. To answer this question, AI models are used as a statistical tool to analyse brain activity measured with functional magnetic resonance imaging and magnetoencephalography methods at the Brain Imaging Center at Goethe University during the processing of language and object recognition tasks. The researchers expect that the same representations in the brain will be addressed at the highest level of abstraction.

A core part of this work will be the collection of a very large data set of test subjects who will work on a whole series of corresponding tasks in several study sessions while their brain activity is measured. "The planned data set is unique and will be shared with other scientists in the future, also in the spirit of open science," explains Prof. Christian Fiebach, the spokesperson of the ARENA research group.

But for the time being, the data collected will be used by the modellers in the ARENA research group to explore whether they can make AI systems more flexible and efficient, based on the biological model of the human brain. To this end, they are also incorporating findings from developmental psychology. Conversely, the experimenters would like to learn new analytical techniques from the modellers in order to make their models of the brain more precise. In other words, how can the neuronal image of the orange in the brain be better decoded, and how can this knowledge contribute to providing AI models with more human-like knowledge about the orange in the future?