Scientists recently unveiled a new AI system that can outfox average high school students on the geometry portion of the SAT.
The program answered 49% of the official SAT geometry questions correctly, and got 61% of the practice test questions right. If you extrapolate the it’s performance to the entire SAT maths section, it would have gotten a 500 out of 800.
That’s about the score of an average high school student, according to the press release from the Allen Institute for Artificial Intelligence and the University of Washington.
Scroll down for some of the SAT geometry practice test questions the program, called GeoS, solved to see how well you stack up.
When GeoS encounters a new SAT question it first looks at the diagram and reads the question. It then interprets the diagram and the question as equations.
Then the system scores and ranks the accuracy of the equations and chooses the answer that best reflects those equations. The GIF below shows the equations that the program found in the question and used to solved it.
Oren Etzioni, AI2's CEO, told Tech Insider by email 'that SATs are more useful than the Turing Test' for testing how smart AI systems are. Take a stab at the question below:
Etzioni said they focused on geometry because it requires different aspects of intelligence -- vision, reading, and reasoning. Below the AI chose the answer that best fit its list of formulas.
Like students taking the real SAT test, GeoS was tested on questions it had never encountered before -- how well AI programs can handle novelty is a gold standard for intelligence -- like this question about triangles.
To solve this problem, GeoS had to understand that the triangle's sides are all equal, though it's not explicitly stated in the question.
'Much of what we understand from text and graphics is not explicitly stated, and requires far more knowledge than we appreciate,' Etzioni said in a press release. 'Creating a system to be able to successfully take these tests is challenging, and we are proud to achieve these unprecedented results.'
Etzioni said they're just getting started. Eventually they will build an AI that can tackle the whole maths portion of the SAT.
Business Insider Emails & Alerts
Site highlights each day to your inbox.