Jimenez L, Hernandez C, Vilaplana V. Breast Cancer Molecular Subtyping from H&E Whole Slide Images using Foundation Models and Transformers. In Deep Breast Workshop on AI and Imaging for Diagnostic and Treatment Challenges in Breast Care, MICCAI 2024. 2024.

Abstract

This study tackles the challenge of classifying breast cancer molecular subtypes using H&E-stained Whole Slide Images (WSI), avoiding the cost and labor limitations of the commonly used immunohistochemistry. We leverage the Attention-Challenging Multiple Instance Learning framework and introduce a variant, ACTrans, which utilizes a transformer aggregator for improved performance. We also compare two publicly available foundation feature extractors pre-trained on large pathology datasets. A comparison of the impact of two different patch sizes at two different magnifications is made. The results obtained in our in-house dataset demonstrate that ACTrans outperforms existing methods, particularly with the UNI model at lower resolutions and larger patch sizes. In this setting, ACTrans achieves an average F1 score of 0.687, a precision of 0.755, a recall of 0.667, and an AUC of 0.812. Furthermore, these approaches enhance interpretability when displaying the attention weights. This method can potentially advance breast cancer diagnostics by leveraging the rich information within H&E-stained WSIs.