Brain-inspired algorithms may make for optimized computational networks

Quantifying the rates of synapse pruning in the mammalian neocortex led to new algorithms for constructing adaptive and robust computational networks across several domains
July 19, 2015

Salk and Carnegie Mellon researchers developed a new model for building efficient networks by studying the rate at which the brain prunes back some of its connections during development. In this model, nodes (such as neurons or sensors) make too many connections (left) before pruning back to connections that are most relevant (right). The team applied their synaptic pruning-based algorithm to air flight patterns and found it was able to create routes to allow passengers to reach their destinations efficiently. (credit: Salk Institute and Carnegie Mellon University)

The developing brain prunes (eliminates) unneeded connections between neurons during early childhood. Now researchers from the Salk Institute for Biological Studies and Carnegie Mellon University have determined the rate at which that happens, and the implications of that finding for computational networks.

Neurons create networks through a process called pruning. At birth and throughout early childhood, the brain’s neurons make a vast number of connections — many more than the brain actually needs to function. So as the brain matures and learns, it begins to quickly cut away connections that aren’t being used. When the brain reaches adulthood, it has about 50 to 60 percent less synaptic connections than it had at its peak in childhood. Understanding how the network of neurons in the brain organizes to form its adult structure is key to understanding how the brain learns and functions.

“By thinking computationally about how the brain develops, we questioned how rates of synapse pruning may affect network topology and function,” says Saket Navlakha, assistant professor at the Salk Institute’s Center for Integrative Biology and a former postdoctoral researcher in Carnegie Mellon’s Machine Learning Department. “We have used the resulting insights to develop new algorithms for constructing adaptive and robust networks in other domains.” The findings were recently published in an open-access paper in PLOS Computational Biology,

But the processes the brain and network engineers conventionally use to learn the optimal network structure are very different. Computer science and engineering networks initially contain a small number of connections and then add more connections as needed.

An improved computer-network algorithm based on brain pruning

“Engineered networks are built by adding connections rather than removing them. You would think that developing a network using a pruning process would be wasteful,” says Ziv Bar-Joseph, associate professor in Carnegie Mellon’s Machine Learning and Computational Biology departments. “But as we showed, there are cases where such a process can prove beneficial for engineering as well.”

The researchers first determined key aspects of the pruning process by counting the number of synapses present in a mouse model’s somatosensory cortex over time. After counting synapses in more than 10,000 electron microscopy images, they found that synapses were rapidly pruned early in development, and then as time progressed, the pruning rate slowed.

The results of these experiments allowed the team to develop an algorithm for designing computational networks based on the brain pruning approach. Using simulations and theoretical analysis they found that the neuroscience-based algorithm produced computer networks that were much more efficient than the current engineering methods. The flow of information was more direct, and provided multiple paths for information to reach the same endpoint, minimizing the risk of network failure.

Optimizing airline routes as a test case

Delta U.S. routes (not the focus of this study) (credit: David Galvin/University of Notre Dame)

“We took this high-level algorithm that explains how neural structures are built during development and used that to inspire an algorithm for an engineered network,” says Alison Barth, professor in Carnegie Mellon’s Department of Biological Sciences and member of the university’s BrainHubSM initiative. “It turns out that this neuroscience-based approach could offer something new for computer scientists and engineers to think about as they build networks.”

Improving airline efficiency and robustness using pruning algorithms. Based on actual data of travel frequency among 122 popular cities from the 3rd quarter of 2013, researchers derived a comparison of efficiency (travel time in terms of number of hops) and robustness (number of alternative routes with the same number of hops) using different algorithms. Decreasing-rate pruning produced more efficient networks with similar robustness. (credit: Saket Navlakha1et al. PLOS Computational Biology)

As a test of how the algorithm could be used outside of neuroscience, Navlakha applied the algorithm to flight data from the U.S. Department of Transportation. He found that the synaptic pruning-based algorithm created the most effective routes to allow passengers to reach their destinations.

“We realize that it wouldn’t be cost effective to apply this to networks that require significant infrastructure, like railways or pipelines,” Navlakha said. “But for those that don’t, like wireless networks and sensor networks, this could be a valuable adaptive method to guide the formation of networks.”

Abstract of Decreasing-rate Pruning Optimizes the Construction of Efficient and Robust Distributed Networks

Robust, efficient, and low-cost networks are advantageous in both biological and engineered systems. During neural network development in the brain, synapses are massively over-produced and then pruned-back over time. This strategy is not commonly used when designing engineered networks, since adding connections that will soon be removed is considered wasteful. Here, we use this process as inspiration for a new network design algorithm, which also led to a new experimental hypothesis. In particular, we show that for large distributed routing networks, network function is markedly enhanced by hyper-connectivity followed by aggressive pruning and that the global rate of pruning, a developmental parameter not previously studied by experimentalists, plays a critical role in optimizing network structure. We first used high-throughput image analysis techniques to quantify the rate of pruning in the mammalian neocortex across a broad developmental time window and found that the rate is decreasing over time. Based on these results, we analyzed a model of computational routing networks and show using both theoretical analysis and simulations that decreasing rates lead to more robust and efficient networks compared to other rates. We also present an application of this strategy to improve the distributed design of airline networks. This inspiration from neural network formation suggests effective ways to design distributed networks across several domains.