Find the fracture and divide it into different types.
The hip is the joint between the pelvis and femur. Reading a hip X-ray involves identifying not only whether a fracture exists but also its pattern. Research is investigating whether AI can assist with part of this process.
This paper has developed a model that automatically classifies four types of fracture on X-rays. It combines AI's video classification capabilities with the readings when doctors refer to AI results. It's a study that distinguishes between whether AI is right and whether it helps the doctor.
We analyzed 5,168 X-rays of three organs.
The model was developed with 4,493 images from two institutions and externally tested with 675 images from another institution. Testing only within one hospital may favor familiar acquisition methods and patient characteristics, so the separate-institution results were considered independently.
The classified types were displaced, valgus-impacted, stable, and unstable fractures. The model used a convolutional neural network and was assessed against orthopedic clinicians’ diagnoses. Dataset counts are radiographs, not necessarily the same number of patients.
Sensitivity and specificity in external validation
The published abstract reports external-validation sensitivities of 0.83, 0.89, 0.88 and 0.85 for displaced, valgus-impacted, stable and unstable fractures, respectively. The corresponding specificities were 0.94, 0.90, 0.95 and 0.99.
The model was developed using 4,493 X-rays from two institutions and evaluated using 675 images from another institution. The performance of automated fracture classification and the assessment of doctors interpreting images with AI assistance need to be considered separately. Sensitivity is the proportion of images with a particular fracture type in which that type is identified. Specificity is the proportion of images without that fracture type correctly identified as such. Results obtained under different assessment conditions should not be combined into a single overall accuracy figure.
It's not that the patient's recovery is improving.
It's a retrospective study of past X-rays. It's understandable that we have external video verification, but it's not a study that actually evaluates whether the AI in an emergency room has reduced diagnostic time or changed patients' complications and recovery.
And this is the basis for the possibility of supporting video readability. We shouldn't read the diagnostic support performance to surgery success rates, or assume that new hospitals are guaranteed the same performance. We need the next step of research to see if it's helping real patients.