Dr. Oliver Obst
Neurocomputing & Distributed Systems
 
Autonomous Systems Lab, CSIRO ICT Centre, Sydney, Australia

Initialization and self-organized optimization of recurrent neural network connectivity

Our new paper describes a mathematical model for generic neural microcircuits, with potential engineering applications, as well as implications to understand how networks in biology are shaped to be optimally adapted to requirements of their environment.

Reservoir computing (RC) is a recent paradigm in the field of recurrent neural networks. Networks in RC have a sparsely and randomly connected fixed hidden layer, and only output connections are trained. RC networks have recently received increased attention as a mathematical model for generic neural microcircuits to investigate and explain computations in neocortical columns. Applied to specific tasks, their fixed random connectivity, however, leads to significant variation in performance. Few problem-specific optimization procedures are known, which would be important for engineering applications, but also in order to understand how networks in biology are shaped to be optimally adapted to requirements of their environment. We study a general network initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP). The IP-based learning uses only local learning, and its aim is to improve network performance in a self-organized way. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much more persistent memory than the other methods but are also able to perform highly nonlinear mappings. We also show that IP-based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

, , , , , ,

CFP: Special Issue on Perspectives and Challenges for Recurrent Neural Networks

Special issue of the Elsevier Journal of Algorithms in Cognition, Informatics and Logic.

Submissions connected to the following non-exhaustive list of topics are particularly encouraged:

  • new learning paradigms of RNNs such as unsupervised learning or reservoire learning
  • biologically plausible methods
  • integration of RNNs and symbolic reasoning
  • universal approaches for general data structures such as sets or graphs
  • methods which address the generalization ability of RNNs
  • challenging applications which have the potential to be benchmark problems
  • visionary papers concerning the future of RNNs

Deadline for submissions is 18th of July, 2008.

Technorati Tags: , , , , ,

CFP: Inaugural Issue for International Journal of Social Robotics

There is a new international Springer journal on social robotics, covering quite a range of topics in this field. Authors are invited to submit scientific, technological and philosophical advances in social robots, and their interactions and communications with humans, especially innovative ideas and concepts, new discoveries and improvements, as well as novel applications on the latest fundamental advances in the core technologies that form the backbone of Social Robotics, distinguished developmental projects, as well as seminal works in aesthetic design, ethics and philosophy, studies on social impact and influence pertaining to, and its interaction and communication with human beings and its social impact on our society.

The submission deadline is the 1st July, 2008. For details, check http://www.editorialmanager.com/soro/.

Technorati Tags: , , ,