AI Language Model ChatGPT Solves Problems as Well as College Students, Study Shows
A recent study has uncovered groundbreaking findings about the problem-solving abilities of ChatGPT, an AI language model. The research indicates that ChatGPT, specifically GPT-3, can perform at par or even better than college students when it comes to reasoning and problem-solving tasks. The study, conducted by a team of researchers, involved testing GPT-3 on intelligence problems found in exams like the SAT.
The SAT, a standardized test widely used for college admissions in the United States, assesses a student’s reading, writing, and math skills. It is scored on a scale of 400 to 1600, with an average score of 1060. Notably, some of the most prestigious universities, including Harvard, Yale, and Princeton, require SAT scores for admission. However, numerous other universities also consider SAT scores, regardless of their level of selectivity.
To assess GPT-3’s abilities, the researchers presented the AI model with reasoning problems, including predicting the next image in a complex shape arrangement. The questions were converted into text to enable the model to comprehend them. Importantly, GPT-3 had never encountered these questions before.
Remarkably, GPT-3 performed exceptionally well, correctly solving 80% of the problems presented. This outperformed the average score of the human participants, who were 40 college students from UCLA.
Furthermore, the researchers challenged GPT-3 with SAT analogy questions, where the model had to create links between pairs of words. They believed that these questions were not available on the internet, making it a tougher test. However, GPT-3 surpassed the average score of human college applicants.
Despite these impressive results, there were certain areas where GPT-3 struggled. For instance, when asked to match a passage of prose with a different short story conveying the same meaning, GPT-3 did not perform as well as the students. However, the improved version of the model, GPT-4, showed better results.
Taylor Webb, the lead author of the study, clarified that ChatGPT’s AI did not exhibit human-level intelligence or artificial general intelligence. GPT-3 faced challenges when it came to social interactions, math reasoning, and problems requiring an understanding of physical space.
While GPT-3 showcased a strong capacity for recognizing patterns and making inferences, the researchers couldn’t fully comprehend its reasoning abilities. They were unable to determine whether GPT-3 was thinking like a human or presenting a distinct form of intelligence.
Notably, the training method for GPT-3 differed from how humans learn, making its reasoning process intriguing. The researchers aim to investigate whether GPT-3 represents a genuine form of artificial intelligence, which would be a groundbreaking discovery.
Overall, the study highlights the exceptional problem-solving abilities of GPT-3, shedding light on its unique reasoning capabilities. However, further exploration is necessary to comprehend the inner workings of this AI model fully. The researchers’ pursuit is to unravel if GPT-3 signifies a genuine form of artificial intelligence, marking a significant advancement in the field.
In conclusion, GPT-3 has demonstrated its potential in solving problems, rivalling or surpassing college students’ performance. As researchers continue to delve deeper, a clearer understanding of AI models like ChatGPT will likely emerge, paving the way for more remarkable advancements in the realm of artificial intelligence.
