Artificial intelligence has already beaten us at chess and bested us in IQ tests.
Now, scientists say they have created an AI that can solve SAT geometry questions as well as the average American 11th-grade student.
Researchers at the Allen Institute for Artificial Intelligence in Seattle created a program called GeoS, which uses computer vision to see, uses language algorithms to read and understand problems, and uses a mathematical algorithm to solve them.
The AI correctly answered 49% of geometry questions from official SAT tests, and 61% of practice questions, the researchers showed. If you extrapolate its performance on the official questions to the entire SAT math section, the AI would have an approximate SAT score of 500 out of 800 — the same as the average high schooler's math score in 2015.
The findings were presented Monday presented at the Conference on Empirical Methods in Natural Language Processing (EMNLP) in Lisbon, Portugal.
The best-known test of a machine's intelligence is the Turing test, invented by mathematician Allen Turing in 1950. But this test, which involves fooling a human in a blind conversation, is no longer considered a good measure of articifial intelligence by many scientists today.
"Unlike the Turing Test, standardized tests such as the SAT provide us today with a way to measure a machine’s ability to reason and to compare its abilities with that of a human," Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence, said in a statement.
A big part of how humans understand text and graphics is not explicitly stated, but requires more subtle knowledge. The team's biggest challenge was converting the SAT questions into language the AI could understand. In order to solve geometry questions, the AI had to have an in-depth understanding of text, diagrams, and reasoning.
The Allen Institute researchers say they are also working on systems that can take science tests, which require a knowledge of unstated facts and common sense that humans develop over the course of their lives.