Learning and Representation Capabilities of Echo State Networks

We present results on the capability of echo state networks (ESNs) to generate and learn sinusoidal oscillations, and on the sufficiency condition for ESNs to exhibit the echo state property. In particular, we show analytically and verify numerically that, provided , the adjustable weights of a linear ESN with N reservoir nodes can be chosen so that the ESN generates a linear superposition of K sinusoidal oscillations having arbitrary prescribed periods, including periods long compared with the network dynamics, and multiple incommensurate periods. These weights can be learned using the usual MSE minimization training procedure; however, the accuracy with which they are learned is limited by numerical round-off error. When the network activities (after training) are temporarily perturbed by random noise, the periods of the individual oscillations comprising the signal are recovered correctly by the ESN, but their amplitudes and relative phases are not. We also present numerical results on the ability of nonlinear ESNs to learn individual sinusoidal oscillations and their superpositions. Finally, we demonstrate through explicit example that not all ESNs having reservoir connection matrices with spectral radii less than unity possess the echo state property. The probability of encountering such counterexamples decreases with increasing N, consistent with a conjecture by Jaeger.

By: Ralph Linsker; Geoffrey Grinstein

Published in: RC24112 in 2006


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to reports@us.ibm.com .