Few-shot Medical Anomaly Detection through Centroid Consultation Back and Test-Time Self-Calibration

Zihan Nie1,2, Muhao Xu1,2, Yuan Cui1,2, Hua Wei1,2, Wei Yi1,2, Sijie Niu3, Yi Wan1,2, Xunbin Wei4,5,6, Weiye Song*1,2
1School of Mechanical Engineering, Shandong University 2Key Laboratory of High Efficiency and Clean Mechanical Manufacture, Shandong University, Ministry of Education 3Shandong Key Laboratory of Ubiquitous Intelligent Computing, University of Jinan 4Institute of Medical Technology and Cancer Hospital, Peking University 5Institute of Advanced Clinical Medicine and Biomedical Engineering Department, Peking University 6Peking University International Cancer Institute
Paper Video Code
Abstract of the Method

To accurately detect anomalies in medical images in the few-shot setting, the method leverages the world knowledge of visual language models and combines two innovative modules: Centerpoint Consultation Backtracking (CCB) and Test-Time Self-Calibration (TSC). This is particularly important when the target abnormal region is difficult to detect directly, requiring the model to distinguish subtle pathological features from background noise. Through dynamic clustering and test-time score calibration, our method ensures accurate and reliable detection.

Abstract

Accurate detection of anomalies in medical images is of paramount importance for the early diagnosis and effective treatment of diseases. However, this task is often impeded by the scarcity of labeled data, especially in specialized medical domains, which necessitates the development of few-shot learning methods that can generalize well from limited examples. Despite recent advances, existing few-shot anomaly detection methods still face difficulty in distinguishing subtle pathological features from background noise due to limited data which leads to high background false positives. Moreover, the anomaly scores may be disproportionately influenced by localized prominent noise or artifacts, rather than accurately reflecting the true extent of abnormalities across the entire image, resulting in unreliable detection results.

To address these challenges, we propose a novel few-shot medical image anomaly detection framework that incorporates two innovative Plug-and-Play modules: the Centroid Consultation Back (CCB) module, which enhances local anomaly features through dynamic clustering and global context backpropagation to effectively reduce background false positives while improbing the sensitivity to subtle anomalies; and the Test-Time Self-Calibration (TSC) module, which optimizes anomaly scores in a fully unsupervised manner at test time to mitigate the influence of localized prominent noise or artifacts, ensuring that the anomaly scores accurately reflect the true extent of abnormalities across the entire image without additional training.

Evaluated on medical image datasets of multiple modalities, our framework achieves state-of-the-art performance, with an average AUROC of 82.16% on anomaly classification and 96.50% on anomaly localization, providing a robust and reliable solution for clinical anomaly detection.

Method

Architecture of the Method

For textual prompts, we combine manual and learnable text as abnormal prompts, and only learnable for normal. The prompts are encoded by the CLIP text encoder.

For images, CLIP-encoder extracts global CLS features and local patch features simultaneously from few-shot input images. Subsquently, the CCB module uses dynamic clustering and central self-attention to perform global context feedback, enhancing local anomaly representation and suppressing background artifacts.

During test phase, the TSC module constructs a sparse similarity map between samples and iteratively optimizes the anomaly score using PageRank to mitigate the influence of local noise.

The anomaly map computation branch fuses global and local features and compares them with textual hint features to directly optimize the segmentation mask and indirectly improve localization accuracy.

Video

Comparison with SOTA results on AUROC

Result Comparison

In the table, the bolded items represent the SOTA methods, and the underlined ones are suboptimal. Our method achieved 6 out of 9 tasks including anomaly classification (Image) and location (Pixel) with state-of-the-art performance and ranked second in other 3 of them.

Visualization of test results

In this section, we present the anomaly map results of the anomaly localization task and heatmaps of the anomaly classification task, where the color is darker, the higher the probability that it is an anomaly. Our method is precisely targeted, closely aligning with the ground truth, and has very few false positives in the background.

Examples of localization results and classification heatmaps on different datasets.

Results of classification scores

Result Comparison

Examples of classification scores of normal samples and abnormal samples. The predicted scores by our method are shown with each sample. The higher the score, the more likely to be an anomaly.

Related links

Many excellent works have collectively contributed to our work.

[1] Li, Xiaofan, et al. "PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection" CVPR. 2024.

[2] Alec Radford, et al. "Learning Transferable Visual Models From Natural Language Supervision" PMLR. 2021.

BibTeX

@article{NIE2026113261,
title = {Few-shot Medical Anomaly Detection through Centroid Consultation Back and Test-Time Self-Calibration},
journal = {Pattern Recognition},
pages = {113261},
year = {2026},
issn = {0031-3203},
doi = {https://doi.org/10.1016/j.patcog.2026.113261},
url = {https://www.sciencedirect.com/science/article/pii/S0031320326002268},
author = {Zihan Nie and Muhao Xu and Yuan Cui and Hua Wei and Wei Yi and Sijie Niu and Yi Wan and Xunbin Wei and Weiye Song},
keywords = {Anomaly Detection, Medical Image Analysis, Few-shot Learning},
}