Representative image feature extraction via contrastive learning pretraining for chest x-ray report generation

Medical report generation is a challenging task since it is time-consuming and requires expertise from experienced radiologists. The goal of medical report generation is to accurately capture and describe the image findings. Previous works pretrain their visual encoding neural networks with large datasets in different domains, which cannot learn general visual representation in the specific medical domain. In this work, we propose a medical report generation framework that uses a contrastive learning approach to pretrain the visual encoder and requires no additional meta information. In addition, we adopt lung segmentation as an augmentation method in the contrastive learning framework. This segmentation guides the network to focus on encoding the visual feature within the lung region. Experimental results show that the proposed framework improves the performance and the quality of the generated medical reports both quantitatively and qualitatively.

Recommended citation: Chen, Yu-Jen, et al. "Representative image feature extraction via contrastive learning pretraining for chest x-ray report generation." arXiv preprint arXiv:2209.01604 (2022).
Download Paper