oliver.obst.eu

We are organising a special session on Interactive Data Analysis and Visualization at the 2012 International Joint Conference on Neural Networks! The special session web site contains all the details.

By offering automated information extraction tools from data, machine learning has revolutionized the way in which humans can cope with electronic data volumes. The ever increasing complexity of the settings continues to pose challenges to the field: often, it is no longer possible to specify a priori a formal learning task; complex parameter choices can severely influence the outcome; and an appropriate encoding of data is not clear at all. More and more often, the human constitutes an important step in the loop to interactively decide about an appropriate learning model, model parameters, and data representation. Because of this fact, intuitive models and model parameters, and human understandable interfaces to the model and data are needed. In this frame, interesting new technologies have been developed such as high quality data visualization tools, sparse interpretable data representation and models, informed priors, active learning, and similar.

This special session aims to foster research in neural learning paradigms which offer an intuitive interface to data or models and thus have the potential as parts of an interactive pipeline. Go to the special session web site more information.

We are organising a special session “Information Processing, Inference and Learning” at the Fifth Australian Conference on Artificial Life (ACAL11).

Organisers: Oliver Obst, Mikhail Prokopenko (CSIRO ICT Centre)

Keynote speaker (tentative): Prof. Martin Riedmiller, Albert-Ludwigs University Freiburg, Germany

Learning is one of the most important capabilities of living systems, and processes involved in inference and learning are ingrained in many levels of organisation scale, e.g., from single neurons to organisations or societies. Much of the information processing involved in these processes is consequently distributed, and relies on local interactions among individual entities. On the other hand, many successful approaches from machine learning require some centralised processing, while in living systems no such requirement exists.

In this special session, we are looking for contributions that will address issues that involve information processing, inference, and learning, with a perspective on their application to Alife scenarios. These include for example methods to help characterising relevant information, self-organised encoding of information, decentralised approaches to learning, and inference mechanisms. Possible topics include, but are not limited to,

• information theoretic methods for analysing/describing Alife scenarios
• interactions between genetic evolution, and individual learning
• self-organised information processing
• intrinsically motivated learning
• guided self-organisation

To submit papers to this special session, please use the ACAL 2011 submission system and use Information Processing, Inference and / or Learning as a keyword.

(more…)

DMMD 2011 Symposium: Distributed machine learning and sparse representation with massive data sets
Web page: http://research.ict.csiro.au/conferences/machine-learning/

The symposium will take place at the CSIRO Campus in Sydney (Marsfield), Australia.

The exponentially increasing demand for computing power as well as physical and economic limitations has contributed to a proliferation of distributed and parallel computer architectures. To make better use of current and future high-performance computing, and to fully benefit from these massive amounts of data, we must discover, understand and exploit the available parallelism in machine learning. Simultaneously, we have to model data in an adequate manner while keeping the models as simple as possible, by making use of a sparse representation of the data or sparse modelling of the respective underlying problem.

The invited speakers are:

Samy Bengio (Google Research, CA, USA)
Barbara Hammer (University of Bielefeld, Germany)
Yann LeCun (New York University, NY, USA)
Michael Mahoney (Stanford University, CA, USA)

Call for Papers / Extended Abstracts

(more…)

Our new paper, Improving Recurrent Neural Network Performance using Transfer Entropy, has been accepted at the 17th International Conference on Neural Information Processing (ICONIP) in Sydney.

In this paper, we present an approach to improve the hidden layer of recurrent neural networks, guided by the learning goal of the system, and apply this new method to reservoir computing approaches. Reservoir computing uses, in general, a fixed, randomly initialised hidden layer. A consequence of this is that performance is usually quite good, but not necessarily optimal for the task at hand. There exist self-organised approaches – like intrinsic plasticity – that are able to improve performance of reservoir computing approaches, but usually, they just consider the input to the system, and don’t take the actual task of the system into account.

Our reservoir adaptation optimises the information transfer at each individual unit, dependent on properties of the information transfer between input and (desired) output of the system. Using synthetic data, we show that this reservoir adaptation improves the performance of offline echo state learning and Recursive Least Squares Online Learning.

ICONIP 2010 takes place from the 22nd–25th November 2010 in Sydney, Australia.

The Third International Workshop on Guided Self-Organisation (GSO-2010) will be held at Indiana University in Bloomington, Indiana, USA, 4-6 September 2010.

The workshop is comprised of a group of researchers with diverse yet related interests, overlapping in the area of self-organizing systems and methods for characterizing those systems in ways that may ultimately allow them to be guided toward prespecified goals. Information theory and graph theory are core to many of these methods; quantifying complexity and its sources a common theme.

If interested in participating, send an extended abstract to the email addresses on the workshop web site.  Selected works from the workshop will likely be published in a special journal issue (as has been the case in the past).  More information on the GSO-2010 web site.

guided self-organisation, workshop, artificial life, ai

To get a quick overview on what is happening this year at NIPS, I have taken the titles of accepted papers and used the resulting text to produce a word cloud (a screen shot created using http://www.wordle.net/). The word cloud shows the hottest topics as the largest words (unsurprisingly, ‘learning’ is the most prominent word in all the titles). But see for yourselves…

(more…)

Our new paper describes a mathematical model for generic neural microcircuits, with potential engineering applications, as well as implications to understand how networks in biology are shaped to be optimally adapted to requirements of their environment.

Reservoir computing (RC) is a recent paradigm in the field of recurrent neural networks. Networks in RC have a sparsely and randomly connected fixed hidden layer, and only output connections are trained. RC networks have recently received increased attention as a mathematical model for generic neural microcircuits to investigate and explain computations in neocortical columns. Applied to specific tasks, their fixed random connectivity, however, leads to significant variation in performance. Few problem-specific optimization procedures are known, which would be important for engineering applications, but also in order to understand how networks in biology are shaped to be optimally adapted to requirements of their environment. We study a general network initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP). The IP-based learning uses only local learning, and its aim is to improve network performance in a self-organized way. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much more persistent memory than the other methods but are also able to perform highly nonlinear mappings. We also show that IP-based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

neurophysiology, optimisation, physiological models, recurrent neural nets, unsupervised learning, reservoir computing, echo state networks

I was excited to find one of my approaches being used in a commercial product for emergency egress simulation, sold by a company in the US: Back in 2006, Heni, Jan and I published an approach we called Inverse Steering Behaviors in the chapter “Fast, Neat, and Under Control: Arbitrating Between Steering Behaviors” of AI Game Programming Wisdom 3. The technique builds on Steering Behaviors by Craig Reynolds – reactive procedures for physical agents (like robots or simulated creatures) to move in a lifelike way within dynamic environments. Developed in the late 80s, steering behaviors found applications for example in movies like Lord of the Rings. Our Inverse Steering Behaviors improve the arbitration between individual behaviors, which results in less collisions. Back when we did the work, we used the approach in our robotic soccer team for navigation and to dribble around opponents. The agent-based emergency evacuation simulation system sold by  Thunderhead Engineering, is called Pathfinder.
(more…)

In a paper that was recently accepted at the European Symposium on Artificial Neural Networks (ESANN 2009), we look at different ways to influence the performance of echo state networks. Traditionally, echo state networks and other reservoir computing approaches use a fixed random connected reservoir, which leads to significant variation in performance. Only few problem specific optimisation procedures are known to date. We study a general initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP) for echo state networks. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much longer memory than the other methods, but are also able to perform highly non-linear mappings. We also show that IP based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

Studies on Reservoir Initialization and Dynamics Shaping in Echo State Networks,
J. Boedecker, O. Obst, N.M. Mayer, M. Asada. The full paper will be available after the conference (April) is now available.

neural networks, intrinsic plasticity, echo state networks, reservoir computing

ICDL is a multidisciplinary conference pertaining to all subjects related to
the development and learning processes of natural and artificial systems,
including perceptual, cognitive, behavioral, emotional and all other mental
capabilities that are exhibited by humans, higher animals, and robots. Its
visionary goal is to understand autonomous development in humans and higher
animals in biological, functional, and computational terms, and to enable such
development in artificial systems. ICDL strives to bring together researchers
in neuroscience, psychology, artificial intelligence, robotics and other
related areas to encourage understanding and cross-fertilization of latest
ideas. ICDL2009 is held in Shanghai, June 5-7, 2009.
For a list of topics of see the CfP at http://www.icdl09.org/.

icdl, conference, cfp, learning, development, robotics