Fully automatic electrocardiogram classification system based on generative adversarial network with auxiliary classifier
Introduction
Electrocardiogram (ECG) is a clinical standard for diagnosing heart-related diseases, and is particularly valuable for arrhythmia screening and classification. Nevertheless, the interpretation of ECG signal requires extra expertise. Given the large amount of ECG recordings collected in daily clinical routine, interpreting ECG by cardiologists manually is not only resource-exhaustive but also time consuming. Hence, fully automatic heartbeats classification systems with reliable performance are highly recommended. Various classification systems have been proposed using techniques based on discrete wavelet transform (Ye, Kumar, & Coimbra, 2012), time-domain feature extraction (Mazomenos, Chen, & Acharyya, 2012), frequency-domain feature extraction (Romero & Serrano, 2001), feature selection methods (Llamedo and Martinez, 2011, Mar et al., 2011) and abstract feature (Teijeiro, Félix, & Presedo, 2018), etc. Recently, data-driven techniques including convolutional neural network (CNN) which can extract features automatically have started to attract a lot of interests. CNN has shown its strength in various machine learning tasks, including biosignal classification (Zhai et al., 2017, Ronneberger et al., 2015) and image recognition (Krizhevsky et al., 2012, Simonyan and Zisserman, 2015). Moreover, it is revealed by previous studies (Acharya et al., 2017, Hannun et al., 2019, Kiranyaz et al., 2016, Zhai and Tin, 2018, Zhai et al., 2020) that typical CNN based systems can provide superior performance in arrhythmia classification without the need of hand-crafted feature extraction.
Despite the advantages of typical CNNs in arrhythmia diagnosis tasks, they also face several challenges. First, arrhythmia ECG are commonly of imbalanced classes, where normal beats (N beats) outnumber abnormal beats such as supraventricular ectopic beats (SVEB or S beats) and ventricular ectopic beats (VEB or V beats) (Goldberger et al., 2000, Hermes et al., 1980, Moody and Mark, 2001). For example, 90% of the beats in the MIT-BIH arrhythmia database are normal beats. Some workarounds have been proposed to resolve such a class imbalance problem, including cost-sensitive learning and random re-sampling (Chawla, Japkowicz, & Kotcz, 2004). However, both random over-sampling and cost-sensitive learning such as assigning different weights to each class (Xu, Mak, & Cheung, 2018) could overfit the training data and possibly reduce generalization in actual application. On the other hand, random under-sampling, where a subset of beats can be selected from the overall dataset to fulfill balanced classes (Kiranyaz et al., 2016, Zhai and Tin, 2018), would potentially discard important samples and the training set could become too small for training CNN effectively.
Second, the limited availability of labeled ECG data (especially for arrhythmic classes) restricts the effectiveness of supervised learning for training these classifiers because overfitting would likely to happen. Moreover, as physiological signals suffer from significant inter-subject variability, these CNN classifiers might perform poorly in un-seen patients. As such, previous studies (Chazal, 2014, Chazal and Reilly, 2006, Llamedo and Martinez, 2012, Rahhal et al., 2016, Xia et al., 2018, Ye et al., 2016) have proposed to include some labeled patient-specific heartbeats for boosted classification performance. To some extent, this could also alleviate the class imbalance problem, since patient-specific arrhythmic beats could be annotated to improve the performance of patient-dependent classifiers. However, assigning annotations patient by patient would be extremely costly and consequently, those systems are not fully automatic (require expert assistance).
To tackle the challenges above and meanwhile exploit the benefits of CNN, we suggest that the generative adversarial network (GAN) (Goodfellow, Pouget-Abadie, & Mirza, 2014) is a promising approach to overcome these. GAN usually contains two neural networks, the generator (G) and the discriminator (D), that can play a zero-sum game by competing and cooperating with each other. In brief, G learns to generate “fake” data that looks like the real one to deceive D; while D learns to discriminate the fake data generated by G from the real data. As such, GAN has shown great success in generating images of specific types from noise (Reed et al., 2016, Mirza and Osindero, 2014), transferring images styles (Chen et al., 2016, Isola et al., 2017, Zhu et al., 2017), interpolating high-resolution images (Ledig et al., 2017, Odena et al., 2017), and so on. Some studies have also reported the success of GAN in biomedical applications, for example, to de-noise CT images (Wolterink, Leiner, & Viergever, 2017) and stain tissue-autofluorescence images (Rivenson, Wang, & Wei, 2019). For ECG arrhythmia classification, the GAN may provide data augmentation and relieve class imbalance problem by learning to generate relevant conditional samples. On the other hand, some researchers have looked into the trained D from GAN and shown improved performance using it as classifier than traditional deep neural network (DNN) (Odena, 2016, Shen et al., 2017, Springenberg, 2016).
In this work, we adopted the GAN framework and have designed the ACE-GAN (Generative Adversarial Network with Auxiliary Classifier for ECG). This system includes the G for data augmentation, and extracts the well-trained D as the classifier. Our system is fully automatic and does not require any manual patient-specific labeling. We tested it on the MIT-BIH arrhythmia database (Goldberger et al., 2000, Moody and Mark, 2001) and compared its performance against previous studies with or without expert assistance following AAMI recommendation (ECAR, 1987). The overall workflow of the proposed systems is summarized in Fig. 1.
Section snippets
Related work
Recently, the advantages of CNN for automatic feature extraction and classification in ECG arrhythmia time series signals have been explored. Instead of designing specific features manually with expert knowledge, Kiranyaz et al. (2016) proposed a 1-D CNN for accurate arrhythmia detection of SVEB and VEB. The classifier was trained with common pool data and subject-specific training data patient by patient and had provided top-tier performance at that time. Later, a 2-D CNN system for arrhythmia
Database
To evaluate the performance of the proposed ECG classification systems in this study, we used the publicly accessible MIT-BIH arrhythmia database (Goldberger et al., 2000, Moody and Mark, 2001). This database has been used in numerous previous studies (Chazal et al., 2004, Chazal and Reilly, 2006, Hu et al., 1997, Jiang and Kong, 2007, Kiranyaz et al., 2016, Llamedo and Martinez, 2011, Llamedo and Martinez, 2012, Mar et al., 2011, Rahhal et al., 2016, Raj and Ray, 2018, Teijeiro et al., 2018,
ACE-GAN
The most important component in our classification system is the ACE-GAN. Fig. 2 presents the architecture of our proposed G and D.
Generator architecture
As shown in Fig. 2a, G takes a scalar class label and a vector of Gaussian noise with zero mean and standard deviation of one () as inputs. The label can be any of the four values representing N, S, V or F beat for conditioned generation. This scalar is then converted into a vector using the embedding function in Keras (Chollet, 2015). The elementwise multiplication of the noise and the embedded label is taken at the first hidden layer of G. This hidden layer further connects
Discriminator architecture
We herein used the CNN structure in Zhai and Tin (2018) as our D in this study (Fig. 2b). Such CNN was originally adapted from the well-recognized LeNet (Lecun, Bottou, & Bengio, 1998), and has shown promising performance in arrhythmia classification (Zhai & Tin, 2018). In brief, it comprises of a convolutional layer, a max pooling layer, a second convolutional layer, an average pooling layer and a third convolutional layer with kernel sizes of 8, 2, 10, 3 and 5, respectively. Then a fully
Objective functions and optimization
A variant of GAN named AC-GAN (GAN with auxiliary classifier) has been proposed to generate class-conditional samples (Odena et al., 2017). In AC-GAN, G takes random noise input and class labels as inputs to generate . As a result, every generated sample corresponds to a class label and noise input. The D in AC-GAN takes or as input, and outputs the probability indicating the chance of the sample coming from a certain source (real or “fake”), namely
Implementation
Both the G and D in our ACE-GAN are randomly initialized from . We used Adam optimizer (Kingma & Ba, 2015) with a learning rate of 0.0002 for both G and D. Our proposed ACE-GAN is trained for a total of 10,000 iterations. In each iteration, D is updated twice (one with real data and one with generated data). G is updated twice with the same batch of data to implement a two time-scale update rule (Heusel, Ramsauer, & Unterthiner, 2017), which helps in local convergence. We observed that
Evaluation
We here evaluated the classification performance for S and V beats detection as in previous studies (Acharya et al., 2017, Chazal, 2014, Chazal et al., 2004, Jiang and Kong, 2007, Kiranyaz et al., 2016, Li et al., 2019, Llamedo and Martinez, 2012, Mar et al., 2011, Rahhal et al., 2016, Zhai and Tin, 2018, Zhang et al., 2014, Takalo-Mattila et al., 2018). The following five metrics are calculated using the entire 30 min. of the records in DS2, including classification accuracy (Acc), sensitivity
Results
We first show the effectiveness of GAN training in Fig. 3. Fig. 3a shows the adversarial losses for both G and D during the GAN training. MSE of G becomes quite stable after ~2000 iterations of training, while MSE of D decreases gradually, showing smaller loss values than G in this minimax game. We also assessed the quality of generated samples by G using FD (Eq. (12)). For every 100 iterations, we calculated the FD between the generated samples and real samples (Fig. 3b). It shows that the FD
Discussion
In this study, we have proposed an automatic ECG arrhythmia classifier (D) based on a GAN framework to achieve desirable performance.
To examine how the D contributes to ECG arrhythmia classifications, we have performed further simulations. Fig. 5 shows the performance of our system with or without using the extracted D, when fine-tuned with different numbers of generated samples from G, respectively. In the case without the extracted D, we used a classifier with the same structure as D but with
Conclusion
In this study, we have proposed a novel integrated framework for data augmentation and ECG beat classification based on ACE-GAN. Our system implements a GAN to synthesize additional arrhythmic heartbeats for data augmentation. After adversarial training, the discriminator is extracted and fine-tuned into a patient-dependent classifier. Its performance in detection for SVEB and VEB outperforms several state-of-art automatic systems and also some expert assisted methods. We have performed
CRediT authorship contribution statement
Zhanhong Zhou: Methodology, Investigation, Formal analysis, Writing - original draft, Writing - review & editing. Xiaolong Zhai: Investigation. Chung Tin: Conceptualization, Supervision, Funding acquisition, Writing - original draft, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
This work was supported by Research Grants Council of Hong Kong SAR (Project CityU 11213717) and City University of Hong Kong (Project 7005211).
References (77)
- et al.
Automated detection of arrhythmias using different intervals of tachycardia ECG segments with convolutional neural network
Information Sciences
(2017) - et al.
A review on deep learning methods for ECG arrhythmia classification
Expert Systems with Applications: X
(2020) - et al.
DENS-ECG: A deep learning approach for ECG signal delineation
Expert Systems with Applications
(2021) - et al.
Deep learning approach for active classification of electrocardiogram signals
Information Sciences
(2016) - et al.
Sparse representation of ECG signals for automated recognition of cardiac arrhythmias
Expert Systems with Applications
(2018) - et al.
A knowledge-based deep learning method for ECG signal delineation
Future Generation Computer Systems
(2020) - et al.
Towards end-to-end ECG classification with raw signal extraction and deep neural networks
IEEE Journal of Biomedical and Health Informatics
(2018) - et al.
Semi-supervised learning for ECG classification without patient-specific labeled data
Expert Systems with Applications
(2020) - et al.
Heartbeat classification using disease-specific feature selection
Computers in Biology and Medicine
(2014) - et al.
Wasserstein generative adversarial networks
Paper presented at the Proceedings of the 34th International Conference on Machine Learning
(2017)
A modular low-complexity ECG delineation algorithm for real-time embedded systems
IEEE Journal of Biomedical and Health Informatics
Editorial: Special issue on learning from imbalanced data sets
ACM SIGKDD Explorations Newsletter
An adapting system for heartbeat classification minimising user input
Paper presented at the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society
Automatic classification of heartbeats using ECG morphology and heartbeat interval features
IEEE Transaction of Biomedical Engineering
A. patient-adapting heartbeat classifier using ECG morphology and heartbeat interval features
IEEE Transactions on Biomedical Engineering
InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets
Paper presented at the Proceedings of the 30th International Conference on Neural Information Processing Systems
Inter-patient ECG heartbeat classification with temporal VCG optimized by PSO
Scientific Reports
PGANs: Personalized generative adversarial networks for ECG synthesis to improve patient-specific deep ECG classification
Paper presented at the Proceedings of the AAAI Conference on Artificial Intelligence
PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals
Circulation
Goldberger's Clinical Electrocardiography: A Simplified Approach
AutoGAN: Neural architecture search for generative adversarial networks
Paper presented at the 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Generative adversarial nets
Paper presented at the Proceedings of the 27th International Conference on Neural Information Processing Systems
Improved training of wasserstein GANs
Paper presented at the Proceedings of the 31st International Conference on Neural Information Processing Systems
Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network
Nature Medicine
Development, distribution, and use of the american heart association database for ventricular arrhythmia detector evaluation
Paper presented at the Computers in Cardiology
GANs trained by a two time-scale update rule converge to a local nash equilibrium
Paper presented at the Proceedings of the 31st International Conference on Neural Information Processing Systems
A patient-adaptable ECG beat classifier using a mixture of experts approach
IEEE Transaction of Biomedical Engineering
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Paper presented at the Proceedings of the 32nd International Conference on Machine Learning
Image-to-image translation with conditional adversarial networks
Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition
Block-based neural networks for personalized ECG signal classification
IEEE Transactions on Neural Networks
Adam: A method for stochastic optimization
Paper presented at the 3rd International Conference on Learning Representations (ICLR)
Real-time patient-specific ECG classification by 1-D convolutional neural networks
IEEE Transactions on Biomedical Engineering
ImageNet classification with deep convolutional neural networks
Gradient-based learning applied to document recognition
Proceedings of the IEEE
Cited by (21)
Co-learning-assisted progressive dense fusion network for cardiovascular disease detection using ECG and PCG signals
2024, Expert Systems with ApplicationsDeep Generative Models: The winning key for large and easily accessible ECG datasets?
2023, Computers in Biology and MedicineGenerative adversarial networks in electrocardiogram synthesis: Recent developments and challenges
2023, Artificial Intelligence in MedicineEffective and efficient neural networks for spike inference from in vivo calcium imaging
2023, Cell Reports MethodsA real-time embedded system to detect QRS-complex and arrhythmia classification using LSTM through hybridized features
2023, Expert Systems with Applications