A Novel Metric for Detecting Memorization in Generative Models for Brain MRI Synthesis

Deep generative models have emerged as a transformative tool in medical imaging, offering substantial potential for synthetic data generation. However, recent empirical studies highlight a critical vulnerability: these models can memorize sensitive training data, posing significant risks of unauthorized patient information disclosure. Detecting memorization in generative models remains particularly challenging, necessitating scalable methods capable of identifying training data leakage across large sets of generated samples. In this work, we propose DeepSSIM, a novel self-supervised metric for quantifying memorization in generative models. DeepSSIM is trained to: i) project images into a learned embedding space and ii) force the cosine similarity between embeddings to match the ground-truth SSIM (Structural Similarity Index) scores computed in the image space. To capture domain-specific anatomical features, training incorporates structure-preserving augmentations, allowing DeepSSIM to estimate similarity reliably without requiring precise spatial alignment. We evaluate DeepSSIM in a case study involving synthetic brain MRI data generated by a Latent Diffusion Model (LDM) trained under memorization-prone conditions, using 2,195 MRI scans from two publicly available datasets (IXI and CoRR). Compared to state-of-the-art memorization metrics, DeepSSIM achieves superior performance, improving F1 scores by an average of +52.03% over the best existing method. Code and data of our approach are publicly available at the following link: https://github.com/brAIn-science/DeepSSIM.

Key Contributions

DeepSSIM: a self-supervised metric trained to map images into an embedding space where cosine similarity approximates ground-truth SSIM scores, enabling alignment-free detection of memorized training samples
Structure-preserving augmentation strategy that captures domain-specific anatomical features for medical imaging memorization detection
Labeled dataset of similar, different, and duplicate generated MRI and chest X-ray scans for benchmarking memorization metrics, with +52.03% F1 improvement over prior best method

🛡️ Threat Analysis

Model Inversion Attack

The paper's primary contribution is detecting training data leakage from generative models — specifically when a Latent Diffusion Model memorizes and reproduces private brain MRI scans. DeepSSIM quantifies the degree to which generated images are near-exact reconstructions of training data, directly addressing the training data reconstruction threat. The adversary model is implicit: unauthorized parties could generate samples to recover patient data from a deployed model.