Recent advances in interactive keypoint estimation methods have enhanced accuracy while minimizing user intervention. However, these methods require user input for error correction, which can be costly in vertebrae keypoint estimation where inaccurate keypoints are densely clustered or overlap. We introduce a novel approach, KeyBot, specifically designed to identify and correct significant and typical errors in existing models, akin to user revision. By characterizing typical error types and using simulated errors for training, KeyBot effectively corrects these errors and significantly reduces user workload. Comprehensive quantitative and qualitative evaluations on three public datasets confirm that KeyBot significantly outperforms existing methods, achieving state-of-the-art performance in interactive vertebrae keypoint estimation.
Overview of (a) interactive keypoint estimation and (b) the KeyBot framework: The interaction model generates initial keypoint predictions from an image, which the user then revises, leading to updated results. This approach requires user input for error correction. In contrast, KeyBot offers a cost-efficient feedback mechanism via automated, rapid, and iterative refinement, without needing user input.
Training overview of KeyBot: KeyBot consists of two main components. The detector is trained to discern whether each input keypoint is accurate or not. The corrector is trained to refine these inaccurate keypoints accurately.
Prediction results: KeyBot effectively reduces three types of errors, even in challenging cases where error types are mixed.
@inproceedings{kim2024Bones,
title={Bones Can't Be Triangles: Accurate and Efficient Vertebrae Keypoint Estimation through Collaborative Error Revision},
author={Kim, Jinhee and Kim, Taesung and Choo, Jaegul},
booktitle={European Conference on Computer Vision},
year={2024},
}