UnipiEprints
Università di Pisa
Sistema bibliotecario di ateneo

On the Predictive Effects of Markovian and Architectural Factors of Echo State Networks

Gallicchio, Claudio and Micheli, Alessio (2009) On the Predictive Effects of Markovian and Architectural Factors of Echo State Networks. Technical Report del Dipartimento di Informatica . Università di Pisa, Pisa, IT.

[img] Other (GZip)
Available under License Creative Commons Attribution No Derivatives.

Download (326Kb)

    Abstract

    Echo State Networks (ESNs) represent an emerging paradigm for modeling Recurrent Neural Networks (RNNs).In this report we try to identify and investigate some of the main aspects that can be accounted for the success and limitations of this class of models.Independently of the architectural design, we first show the effect on ESNs behavior due to the contractivity of the state transition function and the related Markovian bias.The purpose of our study is also to give an insight on how and why a larger reservoir may improve the predictive performance. We identify four key factors which can influence the performance of ESNs: input variability, multiple time-scales dynamics, non-linear interactions among units and regression in a high dimensional state space. Several variants of the basic ESN model are introduced in order to study these main factors. The proposed variants are tested on four datasets: the Mackey-Glass chaotic time series, the 10th order NARMA system, and two predictive tasks on a symbolic sequence domain with Markovian/anti-Markovian flavor. Experimental evidence shows that all the key identified factors have a major role in determining ESN performances.

    Item Type: Book
    Uncontrolled Keywords: Recurrent Neural Networks, Echo State Networks, Markovianity, Architectural Design Analysis, Sequence Processing
    Subjects: Area01 - Scienze matematiche e informatiche > INF/01 - Informatica
    Divisions: Dipartimenti (until 2012) > DIPARTIMENTO DI INFORMATICA
    Depositing User: dott.ssa Sandra Faita
    Date Deposited: 04 Dec 2014 14:33
    Last Modified: 04 Dec 2014 14:33
    URI: http://eprints.adm.unipi.it/id/eprint/2242

    Repository staff only actions

    View Item