Gallicchio, Claudio and Micheli, Alessio (2009) On the Predictive Effects of Markovian and Architectural Factors of Echo State Networks. Technical Report del Dipartimento di Informatica . Università di Pisa, Pisa, IT.
Other (GZip) Available under License Creative Commons Attribution No Derivatives. Download (326Kb) |
Abstract
Echo State Networks (ESNs) represent an emerging paradigm for modeling Recurrent Neural Networks (RNNs).In this report we try to identify and investigate some of the main aspects that can be accounted for the success and limitations of this class of models.Independently of the architectural design, we first show the effect on ESNs behavior due to the contractivity of the state transition function and the related Markovian bias.The purpose of our study is also to give an insight on how and why a larger reservoir may improve the predictive performance. We identify four key factors which can influence the performance of ESNs: input variability, multiple time-scales dynamics, non-linear interactions among units and regression in a high dimensional state space. Several variants of the basic ESN model are introduced in order to study these main factors. The proposed variants are tested on four datasets: the Mackey-Glass chaotic time series, the 10th order NARMA system, and two predictive tasks on a symbolic sequence domain with Markovian/anti-Markovian flavor. Experimental evidence shows that all the key identified factors have a major role in determining ESN performances.
Item Type: | Book |
---|---|
Uncontrolled Keywords: | Recurrent Neural Networks, Echo State Networks, Markovianity, Architectural Design Analysis, Sequence Processing |
Subjects: | Area01 - Scienze matematiche e informatiche > INF/01 - Informatica |
Divisions: | Dipartimenti (until 2012) > DIPARTIMENTO DI INFORMATICA |
Depositing User: | dott.ssa Sandra Faita |
Date Deposited: | 04 Dec 2014 14:33 |
Last Modified: | 04 Dec 2014 14:33 |
URI: | http://eprints.adm.unipi.it/id/eprint/2242 |
Repository staff only actions
View Item |