On the efficient use of uncertainty when performing expensive ROC optimisation.
Fieldsend, Jonathan E.
Everson, Richard M.
Institute of Electrical and Electronics Engineers (IEEE)
When optimising receiver operating characteristic (ROC) curves there is an inherent degree of uncertainty associated with the operating point evaluation of a model parameterisation x. This is due to the finite amount of training data used to evaluate the true and false positive rates of x. The uncertainty associated with any particular x can be reduced, but only at the computation cost of evaluating more data. Here we explicitly represent this uncertainty through the use of probabilistically non-dominated archives, and show how expensive ROC optimisation problems may be tackled by only evaluating a small subset of the available data at each generation of an optimisation algorithm. Illustrative results are given on data sets from the well known UCI machine learning repository.
Copyright © 2008 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
IEEE Congress on Evolutionary Computation 2008 (CEC 2008). (IEEE World Congress on Computational Intelligence), Hong Kong, 1-6 June 2008
Proceedings of the IEEE Congress on Evolutionary Computation 2008 (CEC 2008). (IEEE World Congress on Computational Intelligence), pp. 3984 - 3991