Optimisation of Short Term Conflict Alert Safety Related Systems

Abstract

Short Term Conflict Alert (STCA) is an automated warning system designed to alert air traffic controllers to possible loss of separation between aircraft. STCA systems are complex, with many parameters that must be adjusted to achieve best performance. Current procedure is to manually ‘tune’ the governing parameters in order to finely balance the trade-off between wanted alerts and nuisance alerts. We present an incremental approach to automatically optimising STCA systems, using a simple evolutionary algorithm. By dividing the parameter space into regional subsets, we investigate methods of reducing the number of evaluations required to generate the Pareto optimal Receiver Operating Characteristic (ROC) curve. Multi-archive techniques are devised and are shown to cut the necessary number of iterations by half. A method of estimating the fitness of recombined regional parameter subsets without actual evaluation on the STCA system is presented, however, convergence is shown to be severely stunted when relatively weak sources of noise are present. We describe a method of aggressively perturbing parameters outside of their known ‘safe’ ranges when complex inhibitory interactions are present that prevent an exhaustive search of permitted values. The scheme prevents the optimiser from repeating ‘mistakes’ and unnecessarily wasting evaluations. Results show that a more complete picture of the Pareto-optimal ROC curve may be obtained without increasing the number of necessary iterations. Efficacy of the new methods is discussed, with suggestions for improving efficiency. Sources of parameter interdependence and noise are explored and where possible mitigating techniques and procedures suggested. Classifier performance on training and test data is investigated and potential solutions for reducing overfitting are evaluated on a toy problem. We comment on potential uses of the ROC in characterising STCA performance, for comparison to other systems and airspaces. Many industrial systems are structured in a similar way to STCA, we hope that techniques presented will be applicable to other highly parametrised, expensive problem domains.