oliver.obst.eu

Our new paper describes a mathematical model for generic neural microcircuits, with potential engineering applications, as well as implications to understand how networks in biology are shaped to be optimally adapted to requirements of their environment.

Reservoir computing (RC) is a recent paradigm in the field of recurrent neural networks. Networks in RC have a sparsely and randomly connected fixed hidden layer, and only output connections are trained. RC networks have recently received increased attention as a mathematical model for generic neural microcircuits to investigate and explain computations in neocortical columns. Applied to specific tasks, their fixed random connectivity, however, leads to significant variation in performance. Few problem-specific optimization procedures are known, which would be important for engineering applications, but also in order to understand how networks in biology are shaped to be optimally adapted to requirements of their environment. We study a general network initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP). The IP-based learning uses only local learning, and its aim is to improve network performance in a self-organized way. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much more persistent memory than the other methods but are also able to perform highly nonlinear mappings. We also show that IP-based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

neurophysiology, optimisation, physiological models, recurrent neural nets, unsupervised learning, reservoir computing, echo state networks

In a paper that was recently accepted at the European Symposium on Artificial Neural Networks (ESANN 2009), we look at different ways to influence the performance of echo state networks. Traditionally, echo state networks and other reservoir computing approaches use a fixed random connected reservoir, which leads to significant variation in performance. Only few problem specific optimisation procedures are known to date. We study a general initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP) for echo state networks. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much longer memory than the other methods, but are also able to perform highly non-linear mappings. We also show that IP based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

Studies on Reservoir Initialization and Dynamics Shaping in Echo State Networks,
J. Boedecker, O. Obst, N.M. Mayer, M. Asada. The full paper will be available after the conference (April) is now available.

neural networks, intrinsic plasticity, echo state networks, reservoir computing

ICDL is a multidisciplinary conference pertaining to all subjects related to
the development and learning processes of natural and artificial systems,
including perceptual, cognitive, behavioral, emotional and all other mental
capabilities that are exhibited by humans, higher animals, and robots. Its
visionary goal is to understand autonomous development in humans and higher
animals in biological, functional, and computational terms, and to enable such
development in artificial systems. ICDL strives to bring together researchers
in neuroscience, psychology, artificial intelligence, robotics and other
related areas to encourage understanding and cross-fertilization of latest
ideas. ICDL2009 is held in Shanghai, June 5-7, 2009.
For a list of topics of see the CfP at http://www.icdl09.org/.

icdl, conference, cfp, learning, development, robotics

There is an international summer school on Functional Genomics at the Baia Samuele Conference Centre, Scicli, Sicily, Italy, July 5th-19th 2008. The webpage is http://www.functional-genomics.it/school, registration deadline May 20th.

Also in Italy, there is the Bertinoro International Summer School of Natural Computation – BNC 2008. It is to be held at the University Residential Center – Bertinoro (Forlì-Cesena), Italy, September 20-27, 2008. See the webpage at http://www.dmi.unict.it/~bnc/index.html for details.

Finally, in Porto, there is NN2008, the 2008 summer school on neural networks in classification, regression and data mining. July 7-11, Porto, Portugal. http://www.nn.isep.ipp.pt.

Technorati Tags: summer school, italy, portugal, neural networks, genomics, call for participation

Special issue of the Elsevier Journal of Algorithms in Cognition, Informatics and Logic.

Submissions connected to the following non-exhaustive list of topics are particularly encouraged:

• new learning paradigms of RNNs such as unsupervised learning or reservoire learning
• biologically plausible methods
• integration of RNNs and symbolic reasoning
• universal approaches for general data structures such as sets or graphs
• methods which address the generalization ability of RNNs
• challenging applications which have the potential to be benchmark problems
• visionary papers concerning the future of RNNs

Deadline for submissions is 18th of July, 2008.

Technorati Tags: neural networks, rnn, journal, cfp, special issue, recurrent neural networks

At IPSN 2008, I’m going to present our work “Using Echo State Networks for Anomaly Detection in Underground Coal Mines”. In this work, we investigate the problem of identifying anomalies in monitoring critical gas concentrations using a sensor network in an underground coal mine. In this domain, one of the main problems is a provision of mine specific anomaly detection, with cyclical (moving) instead of flatline (static) alarm threshold levels. An additional practical difficulty in modelling a specific mine is the lack of fully labelled data of normal and abnormal situations. We present an approach addressing these difficulties based on echo state networks learning mine specific anomalies when only normal data is available. Echo state networks utilize incremental updates driven by new sensor readings, thus enabling a detection of anomalies at any time during the sensor network operation. We evaluate this approach against a benchmark — Bayes Network based anomaly detection, and observe that the quality of the overall predictions is comparable to the benchmark. However, the echo state networks maintain the same level of predictive accuracy for data from multiple sources. Therefore, the ability of echo state networks to model dynamical systems make this approach more suitable for anomaly detection and predictions in sensor networks. Check out the details here.

Technorati Tags: Sensor networks, Echo State Networks, Neural Networks, Anomaly detection, IPSN, 2008, Paper, Computer Science