“Two dogs are throwing Frisbees at each other” sounds good to the AI, but not to us.
Despite advances in natural language processing, an AI still doesn’t have the common sense to understand human language, according to a new study.
A team of computer scientists from the University of Southern California (USC), the University of Washington, and the Allen Institute for Artificial Intelligence created a test to examine verbal reasoning skills in machine learning (ML) systems.
In the test, the researchers asked the natural language processing (NLP) models of an AI to create sentences using words from a list of simple nouns and verbs. The sentences were supposed to describe a common scenario.
For example, when asked to use the words “dog”, “frisbee”, “throw”, and “catch” in a sentence, the AI responded with: “Two dogs are throwing frisbees at each other.” Although the syntax and structure of the phrase is valid, it’s not something that no human would ever utter.
So it seems that, while natural language processing (NLP) has taken great strides recently, an AI actually understands a lot less of what it reads than we had thought.
Assistant Professor Xiang Ren and PhD student Yuchen Lin at USC’s Department of Computer Science have published a paper that found that despite advances, AI still doesn’t have the common sense needed to create sentences that make sense to a human.
Trying to get beyond simple mimicry
“Current machine text-generation models can write an article that may be convincing to many humans, but they’re basically mimicking what they have seen in the training phase,” said Lin. “Our goal in this paper is to study the problem of whether current state-of-the-art text-generation models can write sentences to describe natural scenarios in our everyday lives.”
Ren and Lin hope the results of their test project will serve as a new benchmark to benefit future research. Their ultimate goal is to introduce common sense to natural language generation.
“Robots need to understand natural scenarios in our daily life before they make reasonable actions to interact with people,” said Lin.
“By introducing common sense and other domain-specific knowledge to machines, I believe that one day we can see AI agents such as Samantha in the movie Her that generate natural responses and interact with our lives.”
9 hours ago
