A Variable Metric Probabilistic k-Nearest-Neighbours Classifier
Everson, Richard M.
Fieldsend, Jonathan E.
Lecture Notes in Computer Science
k-nearest neighbour (k-nn) model is a simple, popular classifier. Probabilistic k-nn is a more powerful variant in which the model is cast in a Bayesian framework using (reversible jump) Markov chain Monte Carlo methods to average out the uncertainy over the model parameters.The k-nn classifier depends crucially on the metric used to determine distances between data points. However, scalings between features, and indeed whether some subset of features is redundant, are seldom known a priori. Here we introduce a variable metric extension to the probabilistic k-nn classifier, which permits averaging over all rotations and scalings of the data. In addition, the method permits automatic rejection of irrelevant features. Examples are provided on synthetic data, illustrating how the method can deform feature space and select salient features, and also on real-world data.
Copyright © 2004 Springer Verlag. The final publication is available at link.springer.com
5th International Conference, Exeter, UK. August 25-27, 2004. Proceedings
Book title: Intelligent Data Engineering and Automated Learning – IDEAL 2004
Vol. 3177, pp. 654-659