05 March 2025
Physical implementation of recurrent neural networks is hindered by the fact that hidden units need to be trained and are often fully-connected. In this work, EMERGE partners from the University of Pisa propose to relieve both these constraints by adopting and improving on an oscillators-based reservoir computing model called Random Oscillators Network (RON). RON is a recurrent neural network composed by damped oscillatory units that showed excellent performance in many sequence processing tasks. RON does not require training of its hidden parameters, since it leverages on a random heterogeneous reservoir.
However, the reservoir of RON depends on a fully connected set of oscillators. They propose 6 sparse topologies for RON and study the performance of the model across different levels of sparsity and different numbers of hidden units. Their experiments highlight that RON can tolerate large levels of sparsity without harming its expressive power, and in most cases even outperforming its fully connected counterpart. Also, RON clearly surpasses Leaky ESN (sparse and fully connected) and LSTM in all benchmarks. The authors believe RON to be an ideal candidate for the realization and the study of physical neural networks in the real world.
Read the paper in the link below.
More Information
