Self-Organising Neural Networks

1. Introduction.- 1.1 Self-Organisation and Blind Signal Processing.- 1.2 Outline of Book Chapters.- 2. Background to Blind Source Separation.- 2.1 Problem Formulation.- 2.2 Entropy and Information.- 2.2.1 Entropy.- 2.2.2 Kullback-Leibler Entropy and Mutual Information.- 2.2.3 Invertible Probability Density Transformations.- 2.3 A Contrast Function for ICA.- 2.4 Cumulant Expansions of Probability Densities and Higher Order Statistics.- 2.4.1 Moment Generating and Cumulant Generating Functions.- 2.4.2 Properties of Moments and Cumulants.- 2.5 Gradient Based Function Optimisation.- 2.5.1 The Natural Gradient and Covariant Algorithms.- 3. Fourth Order Cumulant Based Blind Source Separation.- 3.1 Early Algorithms and Techniques.- 3.2 The Method of Contrast Minimisation.- 3.3 Adaptive Source Separation Methods.- 3.4 Conclusions.- 4. Self-Organising Neural Networks.- 4.1 Linear Self-Organising Neural Networks.- 4.1.1 Linear Hebbian Learning.- 4.1.2 Principal Component Analysis.- 4.1.3 Linear Anti-Hebbian Learning.- 4.2 Non-Linear Self-Organising Neural Networks.- 4.2.1. Non-Linear Anti-Hebbian Learning: The Herrault-Jutten Network.- 4.2.2 Information Theoretic Algorithms.- 4.2.3 Non-Linear Hebbian Learning Algorithms.- 4.2.3.1 Signal Representation Error Minimisation.- 4.2.3.2 Non-Linear Criterion Maximisation.- 4.3 Conclusions.- 5. The Non-Linear PCA Algorithm and Blind Source Separation.- 5.1 Introduction.- 5.2 Non-Linear PCA Algorithm and Source Separation.- 5.3 Non-Linear PCA Algorithm Cost Function.- 5.4 Non-Linear PCA Algorithm Activation Function.- 5.4.1 Asymptotic Stability Requirements.- 5.4.2 Stability Properties of the Compound Activation Function.- 5.4.3 Stability of Solution with Sub-Gaussian Sources.- 5.4.4 Simulation: Separation of Mixtures of Sub-Gaussian Sources.- 5.4.5 Stability of Solution with Super-Gaussian Sources.- 5.4.6 Simulation: Separation of Mixtures of Super-Gaussian Sources.- 5.4.7 Separation of Mixtures of Both Sub-and Super-Gaussian Sources.- 5.5 Conclusions.- 6. Non-Linear Feature Extraction and Blind Source Separation.- 6.1 Introduction.- 6.2 Structure Identification in Multivariate Data.- 6.3 Neural Network Implementation of Exploratory Projection Pursuit.- 6.4 Neural Exploratory Projection Pursuit and Blind Source Separation.- 6.5 Kurtosis Extrema.- 6.6 Finding Interesting and Independent Directions.- 6.7 Finding Multiple Interesting and Independent Directions Using Symmetric Feedback and Adaptive Whitening.- 6.7.1 Adaptive Spatial Whitening.- 6.7.2 Simulations.- 6.7.3 An Extended EPP Network with Non-Linear Output Connections.- 6.8 Finding Multiple Interesting and Independent Directions Using Hierarchic Feedback and Adaptive Whitening.- 6.9 Simulations.- 6.10 Adaptive BSS Using a Deflationary EPP Network.- 6.11 Conclusions.- 7. Information Theoretic Non-Linear Feature Extraction And Blind Source Separation.- 7.1 Introduction.- 7.2 Information Theoretic Indices for EPP.- 7.3 Maximum Negentropy Learning.- 7.3.1 Single Neuron Maximum Negentropy Learning.- 7.3.2 Multiple Output Neuron Maximum Negentropy Learning.- 7.3.3 Maximum Negentropy Learning and Infomax Equivalence.- 7.3.4 The Natural Gradient and Covariant Learning.- 7.4 General Maximum Negentropy Learning.- 7.5 Stability Analysis of Generalised Algorithm.- 7.6 Simulation Results.- 7.7 Conclusions.- 8. Temporal Anti-Hebbian Learning.- 8.1 Introduction.- 8.2 Blind Source Separation of Convolutive Mixtures.- 8.3 Temporal Linear Anti-Hebbian Model.- 8.4 Comparative Simulation.- 8.5 Review of Existing Work on Adaptive Separation of Convolutive Mixtures.- 8.6 Maximum Likelihood Estimation and Source Separation.- 8.7 Temporal Anti-Hebbian Learning Based on Maximum Likelihood Estimation.- 8.8 Comparative Simulations Using Varying PDF Models.- 8.9 Conclusions.- 9. Applications.- 9.1 Introduction.- 9.2 Industrial Applications.- 9.2.1 Rotating Machine Vibration Analysis.- 9.2.2 A Multi-Tag Frequency Identification System.- 9.3 Biomedical Applications.- 9.3.1 Detection of Sleep Spindles in EEG.- 9.4 ICA: A Data Mining Tool.- 9.5 Experimental Results.- 9.5.1 The Oil Pipeline Data.- 9.5.2 The Swiss Banknote Data.- 9.6 Conclusions.- References.