Repeatability of Semi-Quantitative and Volumetric Features from Artificial-Intelligence-Guided Lesion Segmentation on 18F-DCFPyL PSMA-PET/CT Images: Results from a Test-Retest Cohort

Objectives: This study evaluated the test–retest repeatability of semi-quantitative and volumetric features derived from artificial intelligence (AI)-assisted lesion segmentation on 18F-DCFPyL Prostate Specific Membrane Antigen (PSMA)-PET/CT imaging of patients with prostate cancer (PCa). Specifically, we assessed the reliability of maximum, minimum and total standardized uptake values (SUVmax, SUVmean, SUVtotal) and lesion volume measurements across varying lesion sizes and explored the implications of variability for clinical decision-making.

Methods: We analyzed 18F-DCFPyL PSMA-PET/CT images from 22 patients with metastatic PCa. Lesion segmentation was performed using the AI-guided TRAQinform IQ technology, followed by a manual review to eliminate potential false-positive sites of uptake. Lesion-level test–retest repeatability was evaluated using 95% limits of agreement (LOA), intra-class correlation coefficient (ICC), within-subject coefficient of variation (wCOV) and Bland–Altman analysis for SUV and volumetric parameters. Lesions were stratified by size (>1 cm3 and >1.5 cm3) to assess the impact of lesion volume cut-offs on measurement variability.

Results: A total of 297 lesions were analyzed, including 191 lesions > 1 cm3 and 161 lesions > 1.5 cm3. Test–retest variability was higher in smaller lesions, with narrower LOA and lower wCOV for larger lesions. SUVmax and SUVmean exhibited lower variability than SUVtotal and lesion volume. The 95% LOA for SUVmax ranged from −33.81% to +38.02% for all lesions, improving to −31.82% to +31.01% for lesions > 1.5 cm3. Similar trends were observed for SUVmean, SUVtotal, and volume. Bland–Altman plots confirmed reduced variability in larger lesions, with no significant systematic bias.

Conclusions: The test–retest repeatability of AI-assisted PSMA-PET/CT features varies by feature type, with semi-quantitative features demonstrating improved repeatability relative to volumetric features. Additionally, repeatability is influenced by lesion size, with larger lesions exhibiting greater reliability. These findings highlight the importance of lesion size-dependent thresholds in response assessment and variability-aware feature selection in prognostic models. Current algorithms may be better optimized for larger lesions and higher volumes of disease, with limitations remaining in the robust detection and segmentation of smaller/more subtle lesions.

Simple Summary: To date, the test–retest repeatability of lesion-level features derived from artificial intelligence (AI)-guided prostate-specific membrane antigen (PSMA)-PET lesion segmentation has not been systematically assessed. One reason for that is the lack of available data. We demonstrate that a unique test–retest dataset of PSMA-PET scans, i.e., paired scans of patients with metastatic prostate cancer obtained within one week of each other, provides a test bed for AI algorithms to demonstrate how repeatably they can identify and delineate tumors. The methodology we describe is a new means of assessing the validity of AI algorithms.