Artificial intelligence (AI) chatbot ChatGPT, renowned for its ability to generate fluent Japanese texts, has been found to have distinct stylistic features that differentiate its writing from that of human authors, according to a recent study. Researchers from Mejiro University and Kyoto University of Advanced Science compared 144 texts generated by ChatGPT with 72 existing Japanese-language academic papers on psychology. The study focused on stylistic elements such as comma placement and the sequence of parts-of-speech. By analyzing these features, an AI classifier could accurately distinguish between ChatGPT-generated and human-written Japanese texts with 100 percent accuracy. The results held true even with the latest version of the chatbot, GPT-4, which is known for its enhanced text-generating capabilities.
The researchers drew inspiration from criminal investigations, specifically techniques used to identify the authorship of writings. They discovered that ChatGPT-generated texts more frequently used a comma after the postpositional particle wa and had a higher frequency of the prefix hon, meaning the present. Utilizing these distinctive features, an AI classifier was able to easily differentiate between ChatGPT-generated and human-written Japanese texts.
While ChatGPT may appear natural to human eyes, the implementation of data analysis techniques allows for the identification of its machine-generated nature with relative ease. However, it is worth noting that other text-generative AI chatbots produce Japanese texts that are closer in resemblance to those written by humans. As a result, the researchers hope to further explore methods for distinguishing between such chatbots.
The study, published in the scientific journal Plos One, raises concerns about potential dishonesty in academic settings. Previous reports have highlighted the accuracy with which ChatGPT-generated English-language research papers can be distinguished from those written by humans. This study marks an important step in assessing the capabilities of AI chatbots when it comes to generating Japanese texts. As the development of AI continues to progress, ensuring the integrity of written materials becomes increasingly crucial.
In conclusion, ChatGPT, an AI chatbot famous for its Japanese language fluency, has been found to have stylistic features that set its writing apart from human authors. By applying techniques used in criminal investigations, researchers were able to differentiate between ChatGPT-generated and human-written Japanese texts with 100 percent accuracy. This study sheds light on the potential misuse of ChatGPT in writing academic papers and calls for further exploration into distinguishing between different AI chatbots.
