In a groundbreaking development, a team of researchers from the GrapheneX-UTS Human-centric Artificial Intelligence Centre at the University of Technology Sydney (UTS) has created a portable, non-invasive system that has the ability to translate silent thoughts into written text. This revolutionary technology not only has the potential to benefit individuals who are unable to speak due to illness or injury, such as stroke or paralysis, but also opens up doors for seamless communication between humans and machines.
The discovery has garnered significant attention, with the study being selected as the spotlight paper at the NeurIPS conference, a prestigious annual event focusing on artificial intelligence and machine learning. The researchers, led by Distinguished Professor CT Lin, collaborated with first author Yiqun Duan and fellow Ph.D. candidate Jinzhou Zhou from the UTS Faculty of Engineering and IT.
During the study, participants wore a cap that recorded electrical brain activity through their scalp using an electroencephalogram (EEG). Subsequently, an AI model called DeWave, developed by the researchers, analyzed the EEG signals and translated them into words and sentences. By training the AI model with extensive EEG data, it was able to capture distinct characteristics and patterns from the human brain, allowing for the decoding of silent thoughts.
Professor CT Lin describes this research as a pioneering effort in the field, marking a significant breakthrough. What sets this technology apart is its ability to incorporate discrete encoding techniques in the brain-to-text translation process, providing an innovative approach to neural decoding. Additionally, the integration of large language models reveals new possibilities in the realms of neuroscience and AI.
Unlike previous technologies that required invasive procedures such as surgical implantation of electrodes or the use of MRI machines, the non-invasive nature of this system makes it more accessible for everyday use. Additionally, it can be utilized both with and without eye-tracking, making it more practical for a wider range of applications.
The study involved 29 participants, making it more robust and adaptable compared to previous decoding technology that had only been tested on a small number of individuals. However, due to the noisier nature of EEG signals received through a cap instead of implanted electrodes, the translation accuracy score currently stands at around 40% on BLEU-1. The researchers aim to improve this score to a level that matches traditional language translation or speech recognition programs, which typically reach 90%.
Although the model shows greater proficiency in matching verbs than nouns, it still demonstrates meaningful results by aligning keywords and forming similar sentence structures. Yiqun Duan, the first author of the study, believes that this tendency towards synonymous pairs instead of precise translations may stem from semantically similar words generating similar brainwave patterns.
The research builds upon previous brain-computer interface technology developed by UTS in partnership with the Australian Defence Force, which utilized brainwaves to control a quadruped robot. This latest breakthrough not only enhances the potential of human-machine interaction but also holds promise for individuals seeking alternative communication methods.
As researchers continue to refine and improve the system, there is hope that this technology can revolutionize communication for people affected by speech impairment and facilitate seamless interaction between humans and machines. The possibilities offered by such mind-reading AI are vast, and they have the potential to positively impact the lives of many individuals in the future.
