Our brain has a 'filing system' to sort visual memories

What if you could watch your brain at work while it sorts your memories? Not just broad recollections, but the moment it identifies an image. Thanks to a breakthrough study at the University of Southern California, that scenario is now a reality.

The new study reveals how the human brain organizes visual memories using category-based codes in the hippocampus.

The researchers combined recordings from 24 epilepsy patients with a powerful machine learning model. The goal was to uncover how the brain stores and retrieves different types of visual memories.

Brain sorts image memories

The hippocampus cannot store every image as a unique file. Instead, it sorts images into categories like “animal” or “tool.” This helps the brain manage the vast flood of visual information it receives daily.

Dong Song and Charles Liu from the University of Southern California conducted the research. Their team recorded brain activity while patients viewed images.

“We let the patients see five categories of images: ‘animal,’ ‘plant,’ ‘building,’ ‘vehicle,’ and ‘small tools.’ Then we recorded the hippocampal signal,” Song said.

“Then, based on the signal, we asked ourselves a question, using our machine learning technique. Can we decode what category image they are recalling purely based on their brain signal?”

Brain signals show memory categories

The model accurately identified which category a person was remembering. This proved that brain signals during memory tasks carry category-specific information.

The results also confirmed that neurons in the hippocampus use spike timing, not just firing rate, to store memory.

“It’s like reading your hippocampus to see what kind of memory you are trying to form,” said Song.

How memory is decoded

The researchers used a two-layer memory decoding model. The first layer extracted spike features across many time scales.

The second layer combined these to predict the image category. This setup allowed the system to find the best time window for decoding memory.

Unlike black-box models, this one is fully interpretable – it reveals exactly when and where in the brain the memory signal occurs.

Recording setup and patient details

Each patient had electrodes surgically implanted in their brain as part of their treatment for epilepsy.

In some cases, the devices were specifically positioned to target the CA3 and CA1 regions of the hippocampus – areas known to play key roles in memory processing.

The team recorded about 30 neurons per patient. Subjects performed 100 to 150 trials using a touchscreen task involving short memory delays.

According to the researchers, the spatial and temporal sparseness metrics showed that only 20 to 30 percent of each neuron’s activity window contributed to memory coding. Yet across the population, 70 to 80 percent of neurons participated.

Timing of spikes matters

The experts found that the timing of spikes carries more weight than how often neurons fire. Even brief bursts of activity, lasting just milliseconds, were enough to encode category-level memories.

Rate-coding models using fixed bin sizes (like 100 ms or 2000 ms) performed worse. Fine-grained timing (20 to 50 ms bins) worked better, but not as well as the multi-resolution model.

The study supports developing brain-computer tools to aid memory in patients with cognitive decline. With this spatio-temporal coding model, future devices could “read” or even “write” memories in the brain.

“With this knowledge, we can begin to develop clinical tools to restore memory loss and improve lives,” Liu said.

Study limitations and future research

“Several limitations of this study should be acknowledged, along with opportunities for future research,” wrote the researchers.

The study used only five image categories and focused on short-term memory. Long-term memory processes like forgetting and memory replay were not addressed. Future research might explore more complex categories and naturalistic settings.

The authors also suggest studying how memory representations evolve over time. For now, this work lays the groundwork for decoding and potentially repairing human memory.

This research shifts how we understand memory. The brain doesn’t just record what we see. It builds organized, efficient maps. With the right tools, scientists can now read and one day restore those maps, one memory at a time.

The study is published in the journal Advanced Science.

—–

Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates. 

Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.

—–