At first glance, the two pictures show an ordinary baseball scene: a batter preparing to swing, with a catcher and umpire behind home plate. One is a photograph; the other, an imperfect but recognizable reconstruction. What makes the second remarkable is its source. The computer that generated it was working from recordings of someone’s brain activity as the subject viewed the original.
The reconstruction was produced by Brain-IT, an artificial-intelligence system developed in Professor Michal Irani’s laboratory at the Weizmann Institute of Science in Rehovot. Using recordings from functional magnetic resonance imaging, or fMRI, the system translates patterns associated with visual processing into an estimate of the picture a person was looking at. At the reconstruction stage, it receives the brain measurements without being given the original photograph. The study was published at the 2026 International Conference on Learning Representations Rio de Janeiro, Brazil, and highlighted by the institute in September.
Researchers have reconstructed recognizable images from brain scans before. The Weizmann team’s advance brings those reconstructions closer to what the person actually saw. “The new model we developed outperforms them in reconstructing both the content of the image and its details,” Irani said in the institute’s announcement.
Brain-IT also draws on patterns shared across different people’s brain responses, allowing it to adapt to a new participant with relatively little additional scanning. In the researchers’ tests, one hour of that person’s recordings yielded results comparable to some earlier systems trained on roughly 40 hours.
Behind the recognizable pictures lies a difficult scientific problem: How do you translate the brain’s response to a scene back into something another person can see?
Decoding the brain
When we look at a photograph, our eyes do not simply pass a finished picture to the brain. Cells at the back of the eye turn light into electrical signals. Different parts of the brain then work together to process the scene, its shapes and colors, where objects are positioned, and what those objects are. To reconstruct the photograph, a computer must learn how those patterns of activity relate to what the person saw.
The scanner provides the clues, but even these require interpretation. When nerve cells become active, the blood supply around them changes. An fMRI scanner tracks changes in the blood’s oxygen levels, allowing researchers to estimate activity in small areas of brain tissue. It is an indirect measurement, rather than a recording of the electrical signals themselves.
The computer learns to interpret these measurements by comparing many photographs with the scans recorded while people viewed them. Over time, it learns connections between features in the pictures and patterns in the scans. The difficulty is that those patterns vary from person to person. A system that has learned one person’s responses cannot simply assume that another person’s brain will respond in exactly the same way.
Gathering enough examples also takes considerable time. The collection used by the Weizmann team, known as the Natural Scenes Dataset, contains recordings from eight volunteers, each shown thousands of photographs across dozens of scanning sessions. Even that substantial effort provides a limited sample of the many things people see and the different ways their brains respond.
Without sufficiently clear clues, AI can generate a realistic-looking picture while placing an object in the wrong location or giving it the wrong color. The challenge was to make better use of the available recordings. For the Weizmann researchers, that began with looking for common patterns beneath the differences between individual brains.
A common language for the brain
The team’s starting point was to compare what different parts of the brain do when someone looks at a picture. The researchers examined scanner readings from roughly 40,000 small areas of brain tissue per person, then organized them into 128 groups. Readings went into the same group when they responded to similar visual information, even if they came from different people or different locations in their brains. These shared groups allowed the computer to apply lessons learned from one person’s scans to another’s.
This work drew on a model the team had developed called an encoder, which learns to predict how brain tissue will respond to a photograph. “The encoder’s strength lies in its ability to identify regions that perform similar functions in different brains, even when their anatomical locations differ,” Irani explained.
That ability to predict responses also offered a way around the shortage of real scans. The researchers gave the encoder additional photographs that had no brain recordings attached to them. It generated estimates of the responses those pictures would produce.
Those estimates became teaching material for a second model, the decoder, which works in reverse: learning to turn brain responses back into pictures. Because the researchers knew which photograph each estimated response belonged to, they could train the decoder to work back toward that image. This expanded the available training examples without requiring volunteers to spend more time inside a scanner.
Rebuilding the picture required two kinds of information. Knowing that someone saw a dog, for example, does not tell the computer where it stood or what surrounded it. Brain-IT therefore combines information from the shared groups to determine what a picture contains and how its contents are arranged.
One part of the system creates a rough image showing the scene’s layout, shapes and colors. Another supplies clues about the objects and other content. An AI image generator then develops that rough starting image into a detailed reconstruction, guided by the content clues. Together, these two sources of information help it preserve the subject of the photograph and the way the scene actually looked.
Putting Brain-IT to the test
To test Brain-IT, the researchers asked it to reconstruct 1,000 photographs that had been excluded from its training. The system received the brain recordings associated with those photographs. The researchers then compared the pictures it generated with the originals.
They assessed the results using eight standard tests that measure different aspects of image accuracy, including whether shapes, positions and recognizable content match the original. When compared to other leading reconstruction systems, Brain-IT achieved the best average score on seven of those tests.
Another question was how much scanning the system needed to learn a new person’s brain responses. MindEye2, a competing model published in 2024, first learned from several people’s recordings, then adapted to another person using just one hour of their personalized data. Given one hour of recordings from each new participant, Brain-IT scored higher than MindEye2 and another competing system on all eight tests. The advance was therefore greater accuracy with the same limited amount of personal scanning data.
The researchers also tested whether Brain-IT could adapt with even less data. For one participant, 15 minutes of training recordings were enough to produce recognizable reconstructions. That finding is promising, but testing more people will be necessary to establish how widely it holds.
Beyond the mind’s eye
The same tools that reconstruct a picture could also help scientists understand how the brain makes sense of it. In earlier work on the encoder, the team identified patterns suggesting that different parts of a brain region involved in recognizing places respond preferentially to indoor and outdoor scenes. Such findings give researchers specific questions to investigate about how visual information is organized. “In the future, the encoder could also help us understand how different regions work together and how perception takes shape,” Irani said.
Over time, more accurate decoding could contribute to communication tools for people unable to express themselves, or help researchers investigate what patients perceive when they cannot describe it. In their paper, Irani and her colleagues suggest that reconstructing visual experiences could “assist in assessing disorders of consciousness,” offering another way to explore what such patients perceive. These remain research possibilities, with substantial work needed before clinical use.
The immediate achievement is already striking: Brain measurements can guide a computer toward a recognizable picture of what someone saw. The next scientific opportunity is to use those connections to explain how the brain turns incoming signals into the visual world we experience.