New AI Tool Reconstructs What People See From Brain Scans, Raising Mental Privacy Questions

Imagine someone could look at a scan of your brain and recreate, almost exactly, the picture you were staring at when the scan was taken. Not a vague sketch. A detailed, recognizable image, down to the position of objects and the colors in the frame. That's no longer hypothetical.

Researchers at the Weizmann Institute of Science in Rehovot, Israel, led by Michal Irani, have built an AI tool that does exactly this. Feed it a brain scan taken while someone looked at a photo, and it reconstructs that photo with a level of precision that outperforms previous attempts by a significant margin. It also works in reverse: give it an image, and it can predict what a person's brain activity would look like while viewing it.

Irani hopes the tool will eventually help explain how the brain processes the world around it. She also sees a path toward something more immediately human: helping people who are "locked in," fully paralyzed but mentally aware, communicate again. Judy Illes, a neuroethicist at the University of British Columbia who wasn't involved in the work, calls it "magnificent" and says the therapeutic potential is "tremendously exciting."

But not everyone is only celebrating. Some scientists see a darker possibility lurking behind the same technology: a future where someone's private thoughts, memories, or mental imagery could be extracted without their permission. "If there's a way to surreptitiously extract information about what you're thinking about, then... 150 years of sci-fi can come true anytime, and that's worrisome in a lot of ways," says Tommy Sprague, a neuroscientist at the University of California, Santa Barbara.

How the Brain Decoder Works

To understand why this tool is a leap forward, it helps to know how these scans actually work. Functional MRI, or fMRI, tracks blood flow in the brain using a powerful magnet. Areas that light up are thought to be more active in that moment. Each unit of activity the scanner measures, called a voxel, typically covers about three cubic millimeters of brain tissue, which works out to roughly 16,000 neurons bundled together. That's a blurry picture, like trying to read a street sign through a foggy window.

Irani's team used newer, higher-resolution scanners where each voxel covers closer to one cubic millimeter. That's a sharper window. They trained their system on existing data from eight people who had each viewed around 9,000 images inside high-resolution scanners.

The real innovation is architectural. Their "brain decoder" splits the task into two parts: one branch predicts the structure of an image, meaning where colors and shapes fall, while the other predicts its content, meaning what the image actually depicts, like a bunch of bananas on a plate. Those two predictions then feed into a diffusion model, the same category of AI that generates video and images by gradually refining random noise into a coherent picture. The combination produces reconstructions that are far more faithful to the original than earlier efforts, which tended to generate a banana that didn't match the real one's shape or position.

The team also faced a classic AI bottleneck: not enough data. Their solution was to build a second model, an "encoder," that predicts brain activity from an image rather than the other way around. By pairing the encoder and decoder together, they could train on images that were never actually shown to anyone in a scanner. Remarkably, about 70 percent of their training data came from such unpaired images.

That trick paid off in another way, too. By combining data across multiple studies, the researchers found brain regions that seem to respond consistently across different people, one area reacting to images of food, another to images of sports. The resulting "universal brain encoder" needs only about one hour of calibration data from a new person, compared with the roughly 40 hours typically required by earlier tools. At $600 to $1,000 per hour of scanner time, that difference could make large-scale brain research dramatically more affordable, says Sprague.

The system isn't flawless. Irani has shown colleagues a reconstructed image of a cake that came out looking like a stack of sandwiches, and a dog in a bathtub that emerged as a similarly colored goat. Still, in head-to-head comparisons, the tool beat previously published approaches by a wide margin. Irani, somewhat wryly, calls "mind reading" a "cute, jazzy name" for what her team is actually doing. The work was presented last month at the Cognitive Computational Neuroscience conference in New York.

Why It Matters

Irani isn't stopping at still images. She wants to extend the approach to video and audio, eventually reconstructing what people imagine or dream about, something she admits "we don't have yet" but is actively pursuing. If it works, the implications for medicine are real: scientists could get a window into what a PTSD flashback actually looks like inside someone's mind, or give locked-in patients a new way to express themselves.

The same capability, though, is exactly what worries people like Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich. He points to electroencephalography, or EEG, which reads electrical brain activity through a cap of electrodes, or increasingly through consumer devices as unobtrusive as headphones. Unlike fMRI, EEG doesn't require a giant immobile scanner. Once an EEG device is calibrated to a person's brain, Ienca warns, it could become relatively easy for a company to extract additional information beyond what the user intended to share, potentially without clear consent. He can even imagine courts someday weighing reconstructed mental images as evidence.

"I have no doubt that this is, you know, well-intentioned research," Ienca says, "but I think it's also pretty obvious that it could be co-opted for... ethically and societally problematic commercial uses."

Sprague, who ten years ago says he would have laughed off concerns about involuntary mind reading, now takes the risk seriously. Getting a willing volunteer to lie still and cooperate in a scanner is hard enough, he notes, let alone decoding someone against their will. But as EEG-based decoding improves, he believes Irani's general approach would likely work "quite well" even for predicting images a person is merely thinking about, not actively looking at. "We have to be a little more serious about the ethical considerations," he says.

Irani acknowledges the risk of misuse but isn't dwelling on it. "I'm trying to think only of good things," she says. That optimism is understandable from someone building tools meant to help paralyzed patients speak again. But the history of powerful technology suggests good intentions rarely stay in control of how a tool gets used once it leaves the lab. The gap between a research breakthrough and a consumer EEG headset that quietly profiles what you're looking at, or thinking about, may be smaller than anyone wants to admit.