31 May 2024
In this work, EMERGE partners from the University of Pisa introduce the Random Oscillators Network (RON), a physically-inspired recurrent model derived from a network of heterogeneous oscillators. Unlike traditional recurrent neural networks, RON keeps the connections between oscillators untrained by leveraging on smart random initialisations, leading to exceptional computational efficiency. A rigorous theoretical analysis finds the necessary and sufficient conditions for the stability of RON, highlighting the natural tendency of RON to lie at the edge of stability, a regime of configurations offering particularly powerful and expressive models.
Through an extensive empirical evaluation on several benchmarks, they show that the RON: 1) shows excellent long-term memory and sequence classification ability, outperforming other randomised approaches; 2) outperforms fully-trained recurrent models and state-of-the-art randomised models in chaotic time series forecasting; 3) provides expressive internal representations even in a small parametrisation regime making it amenable to be deployed on low-powered devices and at the edge; and 4) is up to two orders of magnitude faster than fully-trained models.
Read the paper in the link below.
More Information
