Researchers at Western University and the Salk Institute for Biological Studies in California have discovered yet another remarkable feature in the ever-complex brain: its ability to extrapolate information from an individual action and forecast subsequent related actions—essentially predicting the immediate future. When we see an object for instance, that information is received by the brain in waves that travel across interconnected neurons in a highly structured way, causing the brain to create a fuller picture of what’s occurring and what’s about to occur. They recently published their findings in Nature Communications.

“We find that the connections between neurons in a single region of visual cortex can help to generate short-term predictions of incoming sensory input,” says senior author Lyle Muller, Assistant Professor at Western University’s Department of Applied Mathematics and Project Manager on the NeuroNex Project, The Fabric of the Primate Neocortex and the Origin of Mental Representations. “These connections generate complex activity patterns, layered on top of the input image, that appear to enable these short-term predictions.”

By introducing a new network architecture and adapting a learning rule recently used to predict chaotic systems in physics, researchers trained networks to make short-term predictions of natural movie inputs. In the video above, we see a woman walking in “ground truth” (above, left panel). Based on the ground truth, the network can use its internal “recurrent” connections to predict the next course of action—the woman continuing to walk (second from left). Intriguingly, multiple versions of the walking legs are apparent in the internal dynamics of the network. These “ghost legs” may represent multiple potential predictions of future input, generated by the internal connections in the network, from which the network can then learn to select the correct prediction. The fourth panel on the right shows a static prediction that occurs when no internal recurrent connections are present, demonstrating that, in this model, the internal connections play an important role in creating dynamic short-term predictions of incoming inputs.

While this is an early version of what’s to come, researchers can use this understanding of the brain to build similar predictive technologies and systems. “Understanding how the recurrent circuits of the visual cortex create short-term predictions can help us understand how our visual system processes the continuous stream of input from our eyes in real time,” says Muller. “These results could lead to new algorithms for processing and predicting movies in the future.”

This research was featured in the August 2023 issue of the NeuroNex Newsletter.