How does this AI read your mind to create images?

Japanese researchers at Osaka University have succeeded in training an AI to recreate the images a person thinks about from MRI scan data. Find out all you need to know about this scientific feat, which opens up many possibilities for the future…

Following on from DALL-E and MidJourney’s ability to create images from text prompts, researchers at Osaka University have now reached a new milestone: an AI that can read your mind to represent what’s on your mind.

In a study published on biorxiv.orgYu Takagi and Shinji Nishimoto explain that they taught the Stable Diffusion model to reconstruct high-definition images from MRI scan data.

Thanks to this training, the AI is able to visually represent what a person is thinking about based on brain activity. The results are simply astonishing.

ia brain images

AI guesses what you’re thinking from an MRI scan

In this experiment, images were presented to the participants by the researchers. They were then placed on an MRI scanner, and the AI took care of reconstructing the visuals they still had in their heads.

More precisely, this feat is based on the use of functional Magnetic Resonance Imaging (fMRI) to map brain activity. It can analyze small changes in blood flow, indicating that specific parts of the brain are functioning.

By presenting different images to people during an fMRI scan, it is possible to identify the different parts of their brains that “activate in response. The responses are then mapped in a format similar to existing generative models, and can be used to generate new images.

In order to train AI, each participant viewed 10,000 images inside an fMRI scanner. The operation was repeated three times, and the MRI data fed into a computer so it could learn how each participant’s brain processed the images.

Researchers stunned by results

Surprisingly, it turns out that artificial intelligence is better at reading the brain activity of some people better than others.

However, in most cases, we find clear similarities between objects, color schemes and compositions of the images presented to the participants and the reconstructions created by the AI.

According to Takagi, co-author of the study and assistant professor at Osaka University, the researchers were very surprised by the results of this experiment.

According to him, the most interesting part of this experiment is that the diffusion model (the image generation AI) manages to predict the brain’s activity and can be used to reconstruct its visual experiences. Yet, it was not designed to understand the brain !

ia image brain process

Next step: AI that can represent your imagination

During the study, the AI was able to see what the participants were seeing by analyzing their brain activity. However, Takagi claims that this technique could theoretically be used to assemble images directly from the imagination a person’s imagination.

As he explains, ” when we see things, the visual information captured by the retina is processed in the brain. a region of the brain called the visual cortex located in the occipital lobe “.

So, when we imagine an image, similar brain regions are activated. It is therefore possible to apply the same technique to brain activity during imagination. However, the researchers do not yet know with what degree of precision they could decode such activity…

According to Tagaki, this technology could potentially be used for the development of brain-machine interfaces for medical or creative purposes.

One of the strong points of this new technique is that it does not require the physical manipulation of a device. As a result, it offers exciting possibilities for artists and designers with physical disabilities that prevent them from using traditional creative tools.

Will our innermost thoughts soon be out in the open?

Unfortunately, such an innovation also entails dangers. Access to a person’s fMRI data introduces major risks to security and confidentiality.

By making it possible to reconstruct internal and private thoughts from data, this technology could enable an unprecedented level of surveillance. Malicious actors could exploit it to spy on our innermost ideas

However, deploying such a system outside a laboratory environment seems very difficult. An fMRI scanner is essential for measuring brain activity, and such a machine costs over a million dollars.

Moreover, as things stand at present, AI must first learn to map an individual’s brain activity. Everyone reacts differently to the same image, so the method must be personalized for each user.

Generating images from brain activity is a complex task. both costly and time-consuming. Work in this field is therefore focused on research, and could demonstrate the differences in perception and interpretation of the world between AI and humans.

Researchers are convinced that they can demonstrate the potential for integration between artificial intelligence and neuroscience researchand how these two fields might interact in the future.

A priori, you can still think freely without having to worry that the person sitting next to you is listening to everything.