MRI-BASED DEEP LEARNING METHOD FOR DETERMINING METHYLATION STATUS OF THE O6-METHYLGUANINE–DNA METHYLTRANSFERASE PROMOTER OUTPERFORMS TISSUE BASED METHODS IN BRAIN GLIOMAS

PURPOSE Methylation of the O6-Methylguanine-DNA Methyltransferase (MGMT) promoter results in epigenetic silencing of the MGMT enzyme and confers an improved prognosis and treatment response in gliomas. The purpose of this study was to develop a deep-learning network for determining the methylation status of the MGMT Promoter in gliomas using T2-w magnetic resonance images only. METHODS Brain MRI and corresponding genomic information were obtained for 247 subjects from The Cancer Imaging Archive (TCIA) and The Cancer Genome Atlas (TCGA). 163 subjects had a methylated MGMT promoter. A T2-w image only network (MGMT-net) was developed to determine MGMT promoter methylation status and simultaneous single label tumor segmentation. The network was trained using 3D-Dense-UNets. Three-fold cross-validation was performed to generalize the networks’ performance. Dice-scores were computed to determine tumor segmentation accuracy. RESULTS MGMT-net demonstrated a mean cross validation accuracy of 94.73% across the 3 folds (95.12%, 93.98%, and 95.12%, standard dev=0.66) in predicting MGMT methylation status with a sensitivity and specificity of 96.31% ±0.04 and 91.66% ±2.06, respectively and a mean AUC of 0.93 ±0.01. The whole tumor segmentation mean Dice-score was 0.82 ± 0.008. CONCLUSION We demonstrate high classification accuracy in predicting the methylation status of the MGMT promoter using only T2-w MR images that surpasses the sensitivity, specificity, and accuracy of invasive histological methods such as pyrosequencing, methylation-specific PCR, and immunofluorescence methods. This represents an important milestone toward using MRI to predict glioma histology, prognosis, and response to treatment.


INTRODUCTION
O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation is a molecular biomarker of gliomas that has prognostic and therapeutic implications. Unlike isocitrate dehydrogenase (IDH) mutations and 1p/19q co-deletions, MGMT promoter methylation is an epigenetic event. Epigenetic events are functionally relevant but do not involve a change in the nucleotide sequence. Therefore, while MGMT promoter methylation is an important prognostic marker, it does not define a distinct subset of gliomas. MGMT is a DNA repair enzyme that protects normal and glioma cells from alkylating chemotherapeutic agents. The methylation of the MGMT promoter is an example of epigenetic silencing which results in loss of function of the MGMT enzyme and its protective effect on glioma cells. The survival benefit incurred by MGMT promoter methylation in patients treated with temozolomide (TMZ) was determined in 2005.1 Subsequent work by Stupp et al. has shown that in patients who received both radiation and temozolomide, MGMT promoter methylation improved median survival as compared to patients with unmethylated gliomas (21.7 months vs 12.7 months).2,3 Long-term follow-up from that initial study has further substantiated the survival benefit.2,3 As a result, determining MGMT promoter methylation status is an important step in predicting survival and determining treatment.
Currently, the only reliable way to determine MGMT promoter methylation status requires analysis of glioma tissue obtained either via an invasive brain biopsy or following open surgical resection. Surgical procedures carry the risk of neurologic injury and complications. Therefore, considerable attention has been dedicated to developing non-invasive, image-based diagnostic methods to determine MGMT promoter methylation status. A meta-analysis of MRI features demonstrated that glioblastomas with methylated MGMT promoters were associated with less edema, high apparent diffusion coefficient (ADC), and low perfusion. However, the summary sensitivity and specificity of these clinical features was only 79% and 78% respectively.4 Recent advances in deep-learning methods have not only made non-invasive, image-based molecular profiling possible but also highly accurate. Our group has previously demonstrated highlyaccurate, MRI-based, voxel-wise deep-learning networks for determining IDH-classification and 1p/19q co-deletion status using only T2-w MR images.5,6 The benefits of using T2-w images are that they are routinely acquired, they can be obtained quickly, and high quality T2-w images can even be obtained in the setting of motion degradation. Because MGMT promoter methylation in gliomas is such an important biomarker, the purpose of this study was to develop a highly accurate, fully automated deep-learning 3D network for MGMT promoter methylation status determination using T2-w images only.

Data and Pre-processing
Multi-parametric MR Images of brain glioma patients were obtained from the TCIA (The Cancer Imaging Archive) database.7,8 The genomic information was obtained from both TCGA (the cancer genome atlas) and TCIA databases.7,9,10 Subject datasets were screened for the availability of pre-operative MR images, T2-w images and known MGMT promoter status. The final dataset of 247 subjects included 163 methylated cases and 84 unmethylated cases. TCGA subject IDs, MGMT status, and tumor grade are listed in Table 1 of the supplementary data.
Tumor masks for 179 subjects were available through previous expert segmentation. 5,11,12 Tumor masks for the remaining 68 subjects were generated by our previously trained 3D-IDH network.5 These tumor masks were used as ground truth for tumor segmentation in the training step. Ground truth whole tumor masks for methylated and unmethylated MGMT promoter type were labelled with 1s & 2s respectively (Figure 1). Data preprocessing steps included (a) ANTs affine co-registration13 to the SR124 T2 template14, (b) skull stripping using the Brain Extraction Tool (BET)15 from FSL15-18, (c) Removing RF inhomogeneity using N4BiasCorrection19, and (d) normalizing intensity to zero-mean and unit variance. The pre-processing took less than 5 minutes per dataset.

Network Details
Transfer learning for MGMT promoter status determination was implemented using our previously trained 3D-IDH network.5 The decoder part of the network was fine-tuned for a voxelwise dual-class segmentation of the whole tumor with Classes 1 and 2, representing methylated &  6 unmethylated MGMT promoter type, respectively. The network architecture is shown in Figure   2B. A detailed schematic of the network is provided in the supplemental material section.

Network Implementation and Cross-validation
To generalize the network's performance, a 3-fold cross-validation was performed. The dataset of 247 subjects was randomly shuffled and distributed it into 3 groups (approximately 82 subjects for each group). Group 1 had 82 subjects (54 methylated, 28 unmethylated), Group 2 had 83 subjects (55 methylated, 28 unmethylated), and Group 3 had 82 subjects (54 methylated, 28 unmethylated). The 3 groups were alternated between training, in-training validation and held-out testing groups such that, each fold of the cross-validation was a new training phase based on a unique combination of the 3 groups. The network uses the in-training validation dataset to evaluate its learning after each training round and update model parameters to improve performance.
However, the network performance is only reported on the held-out testing group for each fold as it is never seen by the network. The group membership for each cross-validation fold is listed in Table 1

Statistical Analysis
Statistical analysis of the network's performance was performed in MATLAB(R) and R.
Network accuracies were evaluated using majority voting (i.e. voxel-wise cutoff of 50%). The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the model for each fold of the cross-validation procedure were calculated using this threshold. Receiver Operating Characteristic (ROC) curves for each fold were generated separately. Dice-scores were calculated to evaluate the tumor segmentation performance of the networks. The Dice-score calculates the spatial overlap between the ground truth segmentation and the network segmentation.

ROC analysis
The ROC curves for each cross-validation fold for the network is provided in Figure 3.
The network demonstrated very good performance with high sensitivities and specificities

Voxel-wise classification
The network is a voxel-wise classifier with the tumor segmentation map being a natural output. Figures 4A and 4B show examples of the voxel-wise classification for a methylated, and unmethylated MGMT promoter type respectively. The volume fusion procedure was effective in removing false positives and improving the dice-scores by approximately 6%. We also computed the voxel-wise accuracy for the network. The mean voxel-wise accuracies were 81.68% ±0.02 for methylated type and 70.83% ±0.04 for unmethylated type.

Training and segmentation times
Fine-tuning the network took approximately one week. The trained network took approximately three minutes to segment the whole tumor and determine the MGMT status for each subject.

DISCUSSION
We developed a fully-automated, highly accurate, deep-learning network for determining the methylation status of the MGMT promoter that outperforms previously reported algorithms.25- 27 Our network is able to determine MGMT promoter methylation status from T2-w images alone.
This obviates failure from image acquisition artifacts and makes clinical translation straightforward because T2-w images are routinely obtained as part of standard clinical brain MRI.
Previous approaches have required multi-contrast input which can be compromised due to patient motion from lengthier examination times, and the need for gadolinium contrast. Obviating the need of intravenous contrast makes our algorithm applicable to patients with contrast allergies and renal failure. When compared to previously published algorithms, our methodology is fully automated and utilizes minimal preprocessing. The time required for MGMT-net to segment the whole tumor and predict the MGMT promoter methylation status for one subject is approximately 3 minutes on a K80 or P40 NVIDA-GPU.
The higher performance achieved by our network compared to previous image-based classification studies can be explained by several factors. The dense connections in our 3D network architecture are easier to train, carry information from the previous layers to the following layers, and can reduce over-fitting.28,29 3D networks also interpolate between slices to maintain inter-slice information more accurately. The Dual Volume Fusion (DVF) post-processing step improved the dice-scores by approximately 6% by eliminating extraneous voxels not connected to the tumor.
Our approach also uses voxel-wise classifiers and provides a classification for each voxel in the image. This provides a simultaneous single-label tumor segmentation. The cross validation single label whole tumor segmentation performance for the MGMT network provided excellent Dicescores of 0.82 +/-0.008.
The ability to determine MGMT promoter methylation status based on MR images alone is clinically significant because it helps determine whether the glioma will be susceptible to temozolomide (TMZ). Alkylating agents such as TMZ, damage DNA by methylating the oxygen at position 6 of the guanine nucleotide (O6-methylguanine). The process by which many DNA repair enzymes remove O6-methylguanine, results in DNA breaks culminating in cell death.
However, MGMT works differently by restoring the normal guanine residue and rescuing the glioma cell. Therefore, MGMT activity leads to resistance to therapy. Methylation of the MGMT promoter leads to inactivation of MGMT and loss of resistance of glioma cells to alkylating agents.
The MGMT protein is encoded on the long arm of chromosome 10 at position 26 (10q26).
Transcription of the MGMT gene is regulated by several promoters.29 Although incompletely understood, at least nine specific regions within the promoter's gene determine whether a cell will express or not express MGMT. 29  Our non-invasive, MRI based deep learning algorithm outperformed these methods with a sensitivity and specificity of 96.3% and 91.6% respectively. The overall determination of MGMT promoter methylation status is based on the majority of voxels in the tumor. Given the variability in the cut-off values for pyrosequencing-based detection, we performed a Youden's statistical index analysis to determine if the optimal cut-off for our deep learning algorithm was different than majority voting (>50%). The analysis demonstrated that maximum accuracy, sensitivity, specificity, PPV, and NPV were obtained at an optimal cut-off of 50%, the same as majority voting.
Our algorithm was trained on ground-truth obtained from the TCGA database. The TCGA uses Infinium assays to determine MGMT promoter methylation status.32-34 Infinium assays are an immunofluorescence method that uses next generation high-throughput microchip arrays and probes. While these methods have been reported to be more sensitive and specific than the most widely available clinical assays, these methods require pre-existing probes to detect specific methylation sites. 34 The sensitivity and specificity values change depending on the probe and analytical model used to interpret the results. 34 The sensitivities for the best probes range from 87.5-90.6% while the specificity is 94.4%. 34 The overall accuracy of these probes with an optimized analytical model ranges from 91.24%-93.6%.34 Tissue based methods for determining MGMT promoter methylation status remain a complex, multi-step process that is susceptible to failure and inaccuracy even after an adequate tissue sample has been obtained. Therefore, the ability to determine MGMT promoter methylation status on the basis of routine T2-w images alone is highly desirable. Additionally, because our algorithm was trained and evaluated on the multiinstitutional TCIA database it is a better representative of algorithm robustness, real-world performance and potential clinical utilization than previously reported methods.25 The algorithm misclassified 13 cases: six subjects were misclassified as unmethylated and 7 as methylated. Despite these misclassifications, our network achieved a mean cross-validation testing accuracy of 94.73% which is higher than for the MSP, PYT, and Infinium assays.34 While these tissue based methods require an invasive procedure and subsequent tissue processing for at least 48 hours, our deep learning algorithm can segment the entire glioma and determine MGMT promoter methylation status in 3 minutes. The deep learning algorithm can also be fine-tuned to variations in institutional MRI scanners, while other tissue-based methods currently lacks standardization as mentioned above.
The limitations of our study are that deep learning studies require large amounts of data and the relative number of subjects with MGMT promoter methylation is small in the TCGA database. Additionally, acquisition parameters and imaging vendor platforms vary across imaging centers that contribute data. Despite these caveats our algorithm demonstrated high accuracy in determining MGMT promoter methylation status.

CONCLUSION
We demonstrate high accuracy in determining MGMT promoter methylation status using only T2-w MR images. This represents an important milestone toward using MRI to predict glioma histology, prognosis, and appropriate treatment.
Funding Support for this research was provided by NIH/NCI U01CA207091 (AJM, JAM).

Disclosures
No conflicts of interest