Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis
IEEE Transactions on Neural Networks and Learning Systems
Institute of Electrical and Electronics Engineers (IEEE)
In this paper, we elaborate over the well-known interpretability issue in echo-state networks (ESNs). The idea is to investigate the dynamics of reservoir neurons with time-series analysis techniques developed in complex systems research. Notably, we analyze time series of neuron activations with recurrence plots (RPs) and recurrence quantification analysis (RQA), which permit to visualize and characterize high-dimensional dynamical systems. We show that this approach is useful in a number of ways. First, the 2-D representation offered by RPs provides a visualization of the high-dimensional reservoir dynamics. Our results suggest that, if the network is stable, reservoir and input generate similar line patterns in the respective RPs. Conversely, as the ESN becomes unstable, the patterns in the RP of the reservoir change. As a second result, we show that an RQA measure, called Lmax, is highly correlated with the well-established maximal local Lyapunov exponent. This suggests that complexity measures based on RP diagonal lines distribution can quantify network stability. Finally, our analysis shows that all RQA measures fluctuate on the proximity of the so-called edge of stability, where an ESN typically achieves maximum computational capability. We leverage on this property to determine the edge of stability and show that our criterion is more accurate than two well-known counterparts, both based on the Jacobian matrix of the reservoir. Therefore, we claim that RPs and RQA-based analyses are valuable tools to design an ESN, given a specific problem.
This is the author accepted manuscript. The final version is available from IEEE via the DOI in this record.
Published online 02 December 2016
Place of publication
Showing items related by title, author, creator and subject.
Phoenix, Ann; Brannen, Julia; Elliott, Heather; Smithson, Janet; Morris, Paulette; Smart, Cordet; Barlow, Anne; Bauer, Elaine (Institut für Qualitative Forschung, 2016-05)Working in groups is increasingly regarded as fruitful for the process of analyzing qualitative data. It has been reported to build research skills, make the analytic process visible, reduce inequalities and social distance ...
Bias and sensitivity analysis when estimating treatment effects from the cox model with omitted covariates Lin, NX; Logan, S; Henley, WE (Wiley for International Biometric Society, 2013-12)Omission of relevant covariates can lead to bias when estimating treatment or exposure effects from survival data in both randomized controlled trials and observational studies. This paper presents a general approach to ...
Phoenix, Cassandra; Smith, B; Sparkes, A (Elsevier, 2010-01)This article seeks to expand our understanding on narrative and the analysis of stories researchers invite and collect in the domain of aging studies. To do so, we first offer an understanding of what narrative inquiry can ...