Tipping, M.E. (1996). Topographic mappings and feed-forward neural networks. Ph.D thesis, Aston University, Aston Street, Birmingham B4 7ET, UK.
Abstract
This thesis is a study of the generation of topographic mappings --- dimension reducing transformations of data that preserve some element of geometric structure --- with feed-forward neural networks.
As an alternative to established methods, a transformational variant of Sammon's method is proposed, where the projection is effected by a radial basis function neural network. This approach is related to the statistical field of multidimensional scaling, and from that the concept of a `subjective metric' is defined, which permits the exploitation of additional prior knowledge concerning the data in the mapping process. This then enables the generation of more appropriate feature spaces for the purposes of enhanced visualisation or subsequent classification.
A comparison with established methods for feature extraction is given for data taken from the 1992 Research Assessment Exercise for higher educational institutions in the United Kingdom. This is a difficult high-dimensional dataset, and illustrates well the benefit of the new topographic technique.
A generalisation of the proposed model is considered for implementation of the classical multidimensional scaling (CMDS) routine. This is related to Oja's principal subspace neural network, whose learning rule is shown to descend the error surface of the proposed CMDS model.
Some of the technical issues concerning the design and training of topographic neural networks are investigated. It is shown that neural network models can be less sensitive to entrapment in the sub-optimal global minima that badly affect the standard Sammon algorithm, and tend to exhibit good generalisation as a result of implicit weight decay in the training process. It is further argued that for ideal structure retention, the network transformation should be perfectly smooth for all inter-data directions in input space.
Finally, there is a critique of optimisation techniques for topographic mappings, and a new training algorithm is proposed. A convergence proof is given, and the method is shown to produce lower-error mappings more rapidly than previous algorithms.
The entire thesis is available in gzip-ed [PostScript]
format, but beware, the file is 1.7 MB long.
Interested parties may
therefore wish to download only single chapters, thus skipping the
boring bits. Downloading the contents first will therefore be helpful
in this respect.
- Title, Abstract, Contents and Introduction (210 KB)
- Established Techniques for Topographic Mapping (430 KB)
- NeuroScale (353 KB)
- Feature Extraction and the 1992 Research Assessment Exercise (139 KB)
- Classical Multidimensional Scaling and the Principal Subspace Network (63 KB)
- The Form of Topographic Transformations (281 KB)
- Optimising Topographic Transformations (68 KB)
- Conclusions (28 KB)
- Bibliography (31 KB)
Return to my scientific publications page.