Neural Computation, Complexity, and Computational Intelligence (NC³I)

Friday, April 11, 10:30-04:00
CSIRO Marsfield Lecture Theatre
Address: CSIRO, Pembroke Rd, Marsfield NSW

Mikhail Prokopenko, Oliver Obst


10:30-11:15 Thomas Nowotny, University of Sussex:
GeNN: Accelerating spiking neuronal networks on GPU hardware (45min)
11:15-12:00 Joseph Lizier, CSIRO:
Local active information storage as a tool to understand distributed neural information processing (30min)
12:00-01:30 lunch break
01:30-02:15 Andreea Lazar, Ernst-Strüngmann Institute for Neuroscience, MPI Frankfurt:
Self-Organising Recurrent Networks (45min)
02:15-02:45 Denis Bauer, CSIRO:
Population-scale high-thoughput sequencing data analysis (30min)
02:45-03:00 coffee break
03:00-03:30 Maryam M. Khan, University of Newcastle:
Cartesian Genetic Programming Artificial Neural Network (30min)
03:30-04:00 David Howard, CSIRO:
Evolving Spiking Networks for Quadrotor Hover in Turbulent Wind (30min)


GeNN: Accelerating spiking neuronal networks on GPU hardware

Dr Thomas Nowotny, University of Sussex

In this talk I will briefly introduce the concepts of how graphical processing units (GPUs) can be used for general purpose computing using the NVIDIA CUDA API and then present how we are using a simple domain specific language and code generation to build a versatile simulator for neural networks on GPU hardware. This approach has decisive advantages over dedicated specialised hardware solutions and other simulators for GPUs in that: (i) it relies on commercial, reliable hardware that continues to improve (the GPUs), (ii) code can be optimized both for individual scientific models and specific GPUs detected at compile time and (iii) a practically unlimited number of models can be supported in a source library while generated code remains compact and efficient.
GPU enhanced neuronal networks (GeNN, is our prototype for such a code-generation based simulator of neural network on GPU accelerators. With GeNN we observe competitive speedups of up to almost 2 orders of magnitude as compared to a single core of a modern CPU. At the same time we are developing convenient interfaces to the spineML model description standard and the Brian 2 simulator, which will make GeNN very easy to use.

Local active information storage as a tool to understand distributed neural information processing

Dr Joseph Lizier, CSIRO Computational Informatics

Every act of information processing can in principle be decomposed into the component operations of information storage, transfer, and modification. Yet, while this is easily done for today's digital computers, the application of these concepts to neural information processing was hampered by the lack of proper mathematical definitions of these operations on information. Recently, definitions were given for the dynamics of these information processing operations on a local scale in space and time in a distributed system, and the specific concept of local active information storage was successfully applied to the analysis and optimization of artificial neural systems. However, no attempt to measure the space-time dynamics of local active information storage in neural data has been made to date. Here we measure local active information storage on a local scale in time and space in voltage sensitive dye imaging data from area 18 of the cat. We show that storage reflects neural properties such as stimulus preferences and surprise upon unexpected stimulus change, and in area 18 reflects the abstract concept of an ongoing stimulus despite the locally random nature of this stimulus. We suggest that LAIS will be a useful quantity to test theories of cortical function, such as predictive coding.

Self-Organizing Recurrent Networks

Dr Andreea Lazar, Ernst-Strüngmann Institute for Neuroscience, MPI Frankfurt

The dynamics of cortical circuits are shaped by a range of plasticity mechanisms which affect both their synaptic and neuronal properties. These mechanisms allow the recurrent networks in the cortex to learn appropriate representations of complex spatio-temporal stimuli.

Here we introduce a brain-inspired self-organizing recurrent network (SORN) in which the recurrent connections are trained via biologically plausible local plasticity mechanisms in an unsupervised fashion. The network receives input sequences composed of different symbols and learns the structure embedded in these sequences via a simple spike-timing-dependent plasticity rule, while synaptic normalization and intrinsic plasticity keep the level of network activity in a healthy regime. We show that SORNs develop internal representations which capture the spatio-temporal aspects of their inputs. As a result, SORNs outperform classical reservoir networks with no plasticity on memory and prediction tasks.

To further explore the interaction between network dynamics and computational power, we employ an information theoretical framework and characterize the effects of local plasticity mechanisms in SORNs in terms of information storage and transfer.

Evolving Spiking Networks for Quadrotor Hover in Turbulent Wind

Dr David Howard, CSIRO Computational Informatics

We investigate the automatic development of robust quadrotor neurocontrollers based on spiking neural networks. A self-adaptive evolutionary algorithm is used to generate high-utility topology/weight combinations in the networks, and a simple synaptic plasticity mechanism provides some degree of in-trial adaptation.

Cartesian Genetic Programming Artificial Neural Network

Ms Maryam M. Khan, University of Newcastle

Cartesian Genetic Programming (CGP) is a form of genetic programming that has been proved to be flexible and adaptable to a range of problems. We propose a fast learning neuroevolutionary algorithm that is inspired by the method of CGP known as CGP based artificial neural network (CGPANN). The basic idea is the replacement of each computational node with an artificial neuron, thus producing an artificial neural network.The algorithm has the ability to produce feed forward and recurrent networks.
The performance of CGPANN was tested on two diverse problems; a) Benchmark control problem –Inverted Pendulum and b) Classification of breast cancer from Finite Needle Aspiration data samples. In both cases the evolved networks showed improved accuracy, generalization and robustness in comparison with other techniques.
The power of CGP based ANN is its representation which leads to an efficient evolutionary search of suitable topologies. This opens new avenues for applying the proposed technique to other control and pattern recognition problems.

Population-scale high-thoughput sequencing data analysis

Dr. Denis Bauer, CSIRO Computational Informatics

Unprecedented computational capabilities and high-throughput data collection methods promise a new era of personalised, evidence-based healthcare, utilising individual genomic profiles to tailor health management as demonstrated by recent successes in rare genetic disorders or stratified cancer treatments. However, processing genomic information at a scale relevant for the health-system remains challenging due to high demands on data reproducibility and data provenance. Furthermore, the necessary computational requirements requires a large investment associated with compute hardware and IT personnel, which is a barrier to entry for small laboratories and difficult to maintain at peak times for larger institutes. This hampers the creation of time-reliable production informatics environments for clinical genomics. Commercial cloud computing frameworks, like Amazon Web Services (AWS) provide an economical alternative to in-house compute clusters as they allow outsourcing of computation to third-party providers, while retaining the software and compute flexibility. To cater for this resource-hungry, fast pace yet sensitive environment of personalized medicine, we developed NGSANE, a Linux-based, HPC-enabled framework that minimises overhead for set up and processing of new projects yet maintains full flexibility of custom scripting and data provenance when processing raw sequencing data either on a local cluster or Amazon's Elastic Compute Cloud (EC2). Building on these methods, we develop in-memory MapReduce-style machine learning tools to identify biological features in large volumes of diverse genomics data from cancer patients.