Development and assessment of Computer Aided Detection (CAD) software for assisting diagnosis in cervical spine projection radiography
Thesis or dissertation
University of Exeter
Introduction Cervical spine injuries are a major burden on hospital services and have serious consequences for morbidity and mortality; this also affects society due to the associated high care and medical costs. These injuries have the potential to be missed or misdiagnosed, although it must be stated that this phenomena is not unique to cervical spine injuries, and has been seen throughout most imaging services. One possible method to counter this is the use of computer aided detection (CAD) software integrated into the imaging process. This can help increase sensitivity and specificity scores (and thus area under a curve (AUC) scores) by indicating any injuries/pathologies using a pattern recognition algorithm. Methods Lateral cervical spine images were collected from clinical cases and anonymysed by the hospital. These were segmented using a Matlab script to develop ground truth images for the computer scientists to develop cervical spine CAD (CSPINE-CAD) software using machine learning algorithms. The CSPINE CAD software was then assessed in a number of studies as described below. Participants were a convenience sample recruited at the University of Exeter and the Royal Devon and Exeter hospital, and were involved in three tests. These tests all investigated the AUC differences when making a diagnosis without, and with the CSPINE-CAD software. These three tests were: The first test involving five third year radiography students each diagnosing the same five lateral C-spine radiographs, first without and then with the use of the CSPINE-CAD software. Answers were provided by the students via a comments box in which they would make an original diagnosis, then apply the CAD software and then make a re-diagnosis. Upon completion a questionnaire was filled in about their opinions, feedback and confidence whilst using the software. The second test involved 11 third year radiography students from the same cohort each diagnosing 30 lateral C-spine radiographs. This involved using a representation of the CSPINE-CAD software, and followed the same method of diagnosis (a comments box) as in the first test, concluding with a questionnaire. The third test involved 26 participants made up of junior doctors and qualified radiographers, each diagnosing 30 radiographs without and with CSPINE-CAD. This third test did not utilise a comments box, but instead used an answer sheet which contained blank boxes representing each vertebral body and each vertebral junction. These boxes were filled in by the participant using a number between one and six (one representing no injury, and six being 80-100% confident there is an injury). These boxes would all be filled for each image twice; once without CAD and once with CAD. The next image was loaded and the process repeated. Upon completion a questionnaire was again provided to allow the participants to give feedback and confidence about the software. Due to the ambiguity in the language used in the comments boxes of the first and second tests, it was concluded to analyse and produce two results per test. The first analysis was a benefit of the doubt analysis in which the diagnosis provided by the participants would receive some latitude (e.g. misalignment of C5 would be accepted if the “true” answer was misalignment C5/C6). The second analysis was more verbatim and received no latitude. All three tests were compared against the gold standard of a radiologists report, and calculated for AUC scores without and with CSPINE-CAD. Results None of the three test results were statistically significant. The first test showed an AUC increase of 1.39% (with latitude) and 9.54% (no latitude) when using CAD. The second test showed an AUC increase of 1.64% (with latitude) and a loss of 0.25% (no latitude) when using CAD. The third test showed that across all confidence values (2-6) the AUC is higher 1.65% without CAD. Additionally when reviewing only the highest confidence value (6) the AUC increases with CAD by 0.66%. Questionnaire data showed an increase in average confidence when using CAD across all three tests by 12%, 20% and 9.24% respectively, with the majority of participants agreeing that CAD was helpful as a second “pair of eyes” with scores of 100%, 100% and 73%. Conclusion Due to sample sizes and the amount of images being small a statistical significant result could not be reached. Although CSPINE-CAD has shown to be a possible method to reduce missed or misdiagnosed cervical spine injuries, further investigation and development is needed into this CAD software.
Masters by Research in Medical Imaging