Google’s Advanced AI Struggles with Wording of Questions

While some people fear artificial intelligence will take over the world, it may sound like a joke that a math-learning algorithm designed as part of Google’s DeepMind project flunked a basic high school math test.

Trained on algebra, calculus, and other types of math questions, the AI failed a math test, due largely to its inability to translate abstract questions involving words, symbols and functions, and determine the actual operations needed to solve them, according to a story on Futurism.com.

“It turns out, according to the research, that even a simple math problem involves a great deal of brainpower, as people learn to automatically learn to make sense of mathematical operations, memorize the order in which to perform them, and know how to turn word problems into equations,” the Futurism story said.

DeepMind developers published a paper entitled “Analyzing Mathematical Reasoning Abilities of Neural Models,” explaining how the algorithm works and why it failed the test. The blog site Medium published a less scholarly explanation of the study, breaking down the analysis into layman’s terms.

·       Parsing the characters into entities such as numbers, arithmetic operators, variables (which together form functions) and words (determining the question).

·       Planning (for example, identifying the functions in the correct order to compose).

·       Using sub-algorithms for function composition (addition, multiplication).

·       Exploiting working memory to store intermediate values (such as the composition h(f(x))).

·       Generally applying acquired knowledge of rules, transformations, processes, and axioms.

The algorithm scored a 14 out of 40, which didn’t even come close to a human’s passing grade.