There are now AI systems that give the impression of understanding language and even do better than humans in a number of comprehension tasks. However, it is precisely these natural language processing (NLP) models that do not seem to notice when words are mixed up in a sentence. The problem lies in the way AI systems are trained – but there are initial indications of how to overcome it.

Researchers at Auburn University in Alabama, together with Adobe Research, discovered something interesting when they tried to get an NLP system to generate explanations for its own behavior. For example, why did it claim that different sentences mean the same thing? When the scientists tested their approach, they noticed that rearranged words in a sentence made no difference, at least not in how the AI ​​explained them. “This is a general problem with all NLP models,” says Anh Nguyen, researcher at Auburn University who led the study.

“Man bites dog” The team looked at some of the most cutting-edge NLP systems based on BERT, a language model developed by Google on which many of the latest systems were based, including GPT-3. All of these systems performed better than previous systems on the General Language Understanding Evaluation (GLUE) test. GLUE is a standardized set of tasks designed to test language comprehension. It tests, for example, the recognition of paraphrases, the ability to judge whether a sentence expresses positive or negative feelings and the ability to argue verbally.

The tests showed that the systems could not tell whether words were mixed up in a sentence were – not even if the new order changed the meaning. For example, the systems correctly recognized that the phrases “Does marijuana cause cancer?” and “How can smoking marijuana lead to lung cancer?” Paraphrases are.

But they were even more certain that “You smoke cancer like marijuana can give you lung?” (“You smoking cancer how marijuana lung can give?”) And “Lungs can give marijuana smoke like you cancer?” (“Lung can give marijuana smoking how you cancer?”) Also mean the same thing. The systems also believed that sentences with opposing statements – such as “Does marijuana cause cancer?” and “Does cancer cause marijuana?” – asked the same question.

The AI ​​focuses on the essential The only task in What made the word order different for the models was the checking of the (correct) grammatical order of a sentence. Otherwise 75 to 90 percent of the systems tested did not change theirs Answer after mixing up words.

The models seem to include some keywords in a sentence, regardless of the order in which they appear. They don’t understand language like humans do, and GLUE doesn’t measure real language use. In many cases, a model does not need to worry about word order or syntax in general during training tasks. In other words, GLUE teaches NLP models to jump through certain hoops – the systems only do what is expected of them in the task at hand. Many researchers have started to work with a more difficult series of tasks called SuperGLUE, but Nguyen suspects that the problems will be similar there.

Yoshua Bengio and his colleagues at the University of Montreal also recognized the difficulties. They found that switching the words in a conversation sometimes didn’t change the responses from chatbots. And a team from Facebook AI Research found examples of this in Chinese. Nguyen’s team now shows how widespread the problem is.

A repair is possible How much that matters depends on the application. On the one hand, it would be useful to have an AI system that understands when you make a mistake or tell nonsense – just as a human would notice. In general, word order is ultimately elementary in figuring out the meaning of a sentence.

The good news is that it may not be too difficult to fix. The researchers found that if you force a model to focus on word order – by choosing workouts where it has meaning – a model does better at such tasks.

You can therefore assume that optimizing the training tasks will improve the models overall. Nguyen’s findings are another example of how Artificial Intelligence capabilities often lag far behind what people believe. He finds that the results highlight how difficult it is to teach AI to understand and understand in a human way. “Nobody really has a clue,” he says. (bsc)