Thursday, March 28, 2024

A Little Rest Can Work Best For AI

By Jay Soni

- Advertisement -

Researchers mimic sleep patterns of the human brain in artificial neural networks and record a boost in their utility.

In this representation of memories, sleep represents a period when the human brain can consolidate old memories with new memories, without loss of learning. In artificial neural networks, new information can overwrite old information, known as catastrophic forgetting. Photo Credit: Golden, R. et al. 2022, PLOS Computational Biology

Human body needs 7-13 hours of sleep based on their age. Many biological processes happen during sleep: The brain stores new information and gets rid of toxic waste. Nerve cells communicate and reorganize, which supports healthy brain function. The body repairs cells, restores energy, and releases molecules like hormones and proteins.

Artificial neural networks leverage the architecture of the human brain to improve numerous technologies and systems, from basic science and medicine to finance and social media. In some ways, they have achieved superhuman performance, such as computational speed, but unlike the human brain the artificial neural network overwrites previous information as it learns something new, this is called catastrophic forgetting.

- Advertisement -

But this recent study shows how biological models may help mitigate the threat of catastrophic forgetting in artificial neural networks, boosting their utility across a spectrum of research interests.

The scientists used spiking neural networks that artificially mimic natural neural systems: Instead of information being communicated continuously, it is transmitted as discrete events (spikes) at certain time points.

Results showed that when these spiking networks were trained on a new task, and were provided with occasional off-line periods that mimicked sleep, catastrophic forgetting was mitigated. Like the human brain, “sleep” for the networks allowed them to replay old memories without explicitly using old training data.

“When we learn new information,” said Bazhenov, “neurons fire in specific order and this increases synapses between them. During sleep, the spiking patterns learned during our awake state are repeated spontaneously. It’s called reactivation or replay.

The capacity of synapses to be altered or molded–synaptic plasticity, is still in place during sleep and it can further enhance synaptic weight patterns that represent the memory, helping to prevent forgetting or to enable transfer of knowledge from old to new tasks.”


SHARE YOUR THOUGHTS & COMMENTS

Electronics News

Truly Innovative Tech

MOst Popular Videos

Electronics Components

Calculators