AI to Read Human Thoughts and Covert Them into Texts

Imagine an age where you can read the thoughts of a person via telepathy like Professor X of X-Men comics and uncover their anarchical plans or a tech that reads the thoughts of a mute person or your pets and helps you have better communication.

Well, a team at the University of California, San Francisco, performed this experiment and put us a step closer to the dream. Joseph Makin, co-author of the research team says, "We are not there yet, but we think this could be the basis of a speech prosthesis." The university developed the AI to decipher up to 250 words in real-time from a set of between 30 and 50 sentences.

The university recruited four women participants with a history of epilepsy and already had electrode arrays implanted in their brain to monitor epileptic seizures. These participants were asked to read aloud from 50 set sentences multiple times as the team tracked their neural using electrodes while they were speaking.  The sample included "Tina Turner is a pop singer", "the oasis was a mirage", "part of the cake was eaten by the dog", "Those thieves stole 30 jewels", "how did the man get stuck in the tree" and "the ladder was used to rescue the cat and the man." The largest group of sentences contained 250 unique words.

Then each woman repeated the sentences at least twice, with the final repetition to be used by the researchers to test the system. The data was then fed into a machine-learning algorithm that converted it into a string of numbers. It was necessary as these numbers would represent specific words and help future algorithms understand the order of words that occur in grammatically correct sentences

To make sure the numbers related only to aspects of speech, the system compared sounds predicted from small chunks of the brain activity data with actual recorded audio. This was done to ensure the creation of a proper, brain-mapped model, thus allow closer imitation of the functioning of the human brain.

The string of numbers was fed into a second part of the system which converted it into a sequence of words. Next, a recurrent neural network decoded them into sentences. The initial sentences made by the system had a lesser yet funny resemblance to actual ones. For instance, sentences like "Those musicians harmonize marvelously" was decoded as "The spinach was a famous singer", and "A roll of wire lay near the wall" was misinterpreted as "Will robin wear a yellow lily".

However, when the system compared the output with actual sentences that were read, it improved by understanding which numbers were related to words and which words were most likely to follow each other in a sequence. As a result, the system had an average word error rate of 3% compared to the previous versions.

The team further believes that introducing too many words can confuse the system as it will increase the number of possible sentences, thereby reducing the overall accuracy.

"Although we should like the decoder to learn and to exploit the regularities of the language, it remains to show how much data would be required to expand from our tiny languages to a more general form of English," their published paper on journal Nature Neuroscience stated.

Despite the wait for these Brain-Machine Interfaces to become reliable and mainstream someday, one must applaud this incredible feat achieved by Makin and his colleagues. Once the day dawns, it can be used to understand paralyzed patients, offer voice to verbally-challenged and many more.