New Results

Harnessing Digital Health to Objectively Assess Cognitive Impairment in People undergoing Hemodialysis Process: The Impact of Cognitive Impairment on Mobility Performance Measured by Wearables

He Zhou, Fadwa Al-Ali, Changhong Wang, Abdullah Hamad, Rania Ibrahim, Talal Talal, Bijan Najafi
doi: https://doi.org/10.1101/831545

Abstract

Background Cognitive impairment is prevalent but still poorly diagnosed in hemodialysis adults, mainly because of the impracticality of current tools. This study examined whether remotely monitoring mobility performance can help identifying digital measures of cognitive impairment in hemodialysis patients.

Methods Sixty-nine hemodialysis patients (age=64.1±8.1years, body mass index=31.7±7.6kg/m2) were recruited. According to the Mini-Mental State Exam, 44 (64%) were determined as cognitive-intact, and 25 (36%) as cognitive-impaired. Mobility performance, including cumulated posture duration (sitting, lying, standing, and walking), daily walking performance (step and unbroken walking bout), as well as postural-transition (daily number and average duration), were measured using a validated pendant-sensor for a continuous period of 24-hour during a non-dialysis day. Motor capacity was quantified by assessing standing balance and gait performance under single-task and dual-task conditions.

Results No between-group difference was observed for the motor capacity. However, the mobility performance was different between groups. The cognitive-impaired group spent significantly higher percentage of time in sitting and lying (Cohens effect size d=0.78, p=0.005) but took significantly less daily steps (d=0.69, p=0.015) than the cognitive-intact group. The largest effect of reduction in number of postural-transition was observed in walk-to-sit transition (d=0.65, p=0.020). Regression models based on demographics, addition of daily walking performance, and addition of other mobility performance metrics, led to area-under-curves of 0.76, 0.78, and 0.93, respectively, for discriminating cognitive-impaired cases.

Conclusions This study suggests that mobility performance metrics could be served as potential digital biomarkers of cognitive impairment among HD patients. It also highlights the additional value of measuring cumulated posture duration and postural-transition to improve the detection of cognitive impairment. Future studies need to examine potential benefits of mobility performance metrics for early diagnosis of cognitive impairment/dementia and timely intervention.

Introduction

With aging of population, the burden of cognitive impairment appears to increase among patients with end-stage renal disease (ESRD) including those starting hemodialysis (HD) and those already established on HD (1, 2). As more patients of older age receive HD, cognitive impairment and/or dementia has become highly prevalent in this population (3–6). At the same time, HD-associated factors, such as retention of uremic solutes, anemia, hypertension, intradialytic hypotension, and metabolic disturbances, may also increase the risk of cognitive impairment and cognitive decline among HD patients (4, 5). Cognitive impairment leads to overall diminished quality of life and high medical costs associated with coexisting medical conditions and expensive care (7). Therefore, early detection and routine assessment of cognitive function become crucial. They are cornerstones of quality care which can lead to medical intervention to delay further cognitive decline in HD patients (8).

Ideally, HD patients should undergo routine screenings of cognitive function. However, routine assessments using current tools, such as Mini-Mental State Exam, (MMSE) (9), Montreal Cognitive Assessment (MoCA) (10), and Trail Making Test (TMT) (11), could easily overload an already overburdened dialysis clinic. In addition, these cognitive tests must be administered in a clinical setting under the supervision of a well-trained professional. Studies have reported that the accuracy and reliability of these screening tools depend on the experience and skills of the examiner, as well as the individual’s educational level (12, 13). Usually, in a regular dialysis clinic, the nurse does not equip with the professional experience or skills. Regular referral to a neuropsychological clinic could be also impractical as many HD patients have limited mobility, suffer from post-dialysis fatigue, and rarely accept to go to different locations than their regular dialysis clinics for the purpose of cognitive screening. Thus, it is not surprising that emerging literature has demonstrated that although cognitive impairment commonly occurs in HD population, it is still poorly diagnosed (14, 15).

The technology progress of wearable devices along with advanced digital signal processing have enabled continuous monitoring of patients and their daily physical activities (16–18). Such rich and longitudinal information can be mined for physical and behavioral signatures of cognitive impairment. It can provide new avenues for detecting cognitive decline in a timely and cost-effective manner. In particular, as the world has gone digital, there is a need to develop novel digital biomarkers of cognitive impairment, which can be used to screen cognitive function as a routine assessment.

“Mobility performance” depicts enacted mobility in real-life situations (19). It is different than “motor capacity”, which refers to an individual’s motor function assessed under supervised condition (19). The International Classification of Functioning, Disability, and Health (ICF) differentiates between these two measures: what an individual can do (capacity) and does do (performance) (20). Mobility performance requires multifaceted coordination between different parts of neuropsychology (21). This includes motor capacity, intimate knowledge of environment, and difficulty of navigation through changing environments (22). Understanding the association between mobility performance and cognitive function could help to design an objective tool for remote and potentially early detection of cognitive decline. Previous studies have demonstrated that in older adults, people with cognitive impairment exhibit lower level of activity (23–25). However, in previous studies, the assessment of mobility performance mainly relied on self-reported questionnaires (23–25), Actigraphy (26, 27), or accelerometer-derived step count (28). Although self-reported questionnaire is easy to access without the need of any equipment or device, its main limitation is lacking of objectivity (29). Previous studies using Actigraphy or step count only provided limited information about mobility performance (activity level and daily step). They also neglected information about posture and postural-transition, which have been demonstrated to be more reliable than activity level or number of daily steps (30). Considering the motor capacity in patients undergoing HD is usually deteriorated (31), and these patients are highly sedentary with reduced daily activity level (31), it may not be efficient enough to capture cognitive impairment in HD population by just using activity level or step count alone.

In this study, we used a pendant-like wearable sensor to mine potential digital biomarkers from mobility performance for capturing cognitive impairment and tracking the cognitive decline in HD population. We measured detailed metrics of mobility performance including cumulated posture duration (sitting, lying, standing, and walking), daily walking performance (step count and number of unbroken walking bout), as well as postural-transition (daily number and average duration). We hypothesized that 1) HD patients with cognitive impairment have lower mobility performance than those without cognitive impairment; 2) the mobility performance derived digital biomarkers can determine cognitive impairment in HD patients, yielding better results than using daily walking performance alone.

Materials and Methods

Study Population

This study is a secondary analysis of a clinical trial focused on examining the benefit of exercise in adult HD patients (ClinicalTrials.gov Identifier: NCT03076528). The clinical trial was offered to all eligible HD patients visited the Fahad Bin Jassim Kidney Center (Hamad Medical Corporation, Doha, Qatar) for HD process. To be eligible, the subject should be a senior (age 50 years or older), be diagnosed with diabetes and ESRD that require HD, and have capacity to consent. Subjects were excluded if they had major amputation; were non-ambulatory or had severe gait or balance problem (e.g., unable to walk a distance of 15-meter independently with or without assistive device or unable to stand still without moving feet), which may affect their daily physical activity; had active foot ulcer or active infection; had major foot deformity (e.g. Charcot neuroarthropathy); had changes in psychotropic or sleep medications in the past 6-week; were in any active intervention (e.g. exercise intervention); had any clinically significant medical or psychiatric condition; or were unwilling to participate. All subjects signed a written consent approved by the Institutional Review Board at the Hamad Medical Corporation in Doha, Qatar. For the final data analysis, we only included those who had at least 24-hour valid mobility performance data during a non-dialysis day. Only baseline data without any intervention was used for the purpose of this study.

Demographics, Clinical Data, and Motor Capacity

Demographics and relevant clinical information for all subjects were collected using chart-review and self-report, including age, gender, height, weight, fall history, duration of HD, and daily number of prescription medicines. Body mass index (BMI) was calculated based on height and weight information.

All subjects underwent clinical assessments, including MMSE (9), Center for Epidemiologic Studies Depression scale (CES-D) (32), Physical Frailty Phenotype (33), neuropathy screening using Vibration Perception Threshold test (VPT) (34), vascular assessment using Ankle Brachial Index test (ABI) (35), and glycated hemoglobin test (HbA1c) (36). The CES-D short-version scale was used to measure self-reported depression symptoms. A cutoff of CES-D score of 16 or greater was used to identify subjects with depression (32). The Physical Frailty Phenotype, including unintentional weight loss, weakness (grip strength), slow gait speed (15-foot gait test), self-reported exhaustion, and self-reported low physical activity, was used to assess frailty (33). Subjects with 1 or 2 positive criteria were considered pre-frail, and those with 3 or more positive criteria were considered frail. Subjects negative for all criteria were considered robust (33). Plantar numbness was evaluated by the VPT measured on six plantar regions of interest, including the left and right great toes, 5th metatarsals, and heels. In this study, we used the maximum value of VPT measures under regions of interest for both feet to evaluate the Diabetic Peripheral Neuropathy (DPN) status. A subject was designated with DPN if his/her maximum VPT reached 25 volts or greater (34). The ABI was calculated as the ratio of the systolic blood pressure measured at the ankle to the systolic blood pressure measured at the upper arm. A subject was designated with the Peripheral Artery Disease (PAD) if his/her ABI value was either greater than 1.2 or smaller than 0.8 (35).

Motor capacity was quantified by assessing standing balance and walking performance (37). Standing balance was measured using wearable sensors (LegSys™, BioSensics LLC., MA, USA) attached to lower back and dominant front lower shin. Subject stood in the upright position, keeping feet close together but not touching, with arms folded across the chest, for 30-second. Center of mass sway (unit: cm2) was calculated using validated algorithms (38). We assessed walking performance under both single-task and dual-task conditions to determine the impact of cognitive impairment on motor capacity. Walking performance was measured using the same wearable sensors attached to both front lower shins. Subjects were asked to walk with their habitual gait speed for 15-meter with no cognitive task (single-task condition). Then, they were asked to repeat the test while loudly counting backward from a random number (dual-task condition: motor task + working memory) (37). Gait speeds under both conditions were calculated using validated algorithms (39).

Determination of Cognitive Impairment

Cognitive impairment was defined as a MMSE score less than 28 as recommended by Tobias et al. and Damian et al. studies (40, 41). In these studies, researchers have demonstrated that MMSE cutoff score of 28 yields the highest sensitivity and specificity to identify those with cognitive impairment compared to the commonly used lower cutoff scores.

Sensor-Derived Monitoring of Mobility Performance

Mobility performance was characterized by 1) cumulated posture duration, including percentage of sitting, lying, standing, and walking postures of 24-hour; 2) daily walking performance, including step count and number of unbroken walking bout (an unbroken walking bout was defined as at least three consecutive steps within 5 seconds interval (42)); and 3) postural-transition, including total number of postural-transition such as sit-to-stand, stand-to-sit, walk-to-stand, stand-to-walk, walk-to-sit (direct transition from walking to sitting with standing pause less than 1 seconds (43)), and sit-to-walk (direct transition from sitting to walking with standing pause less than 1 seconds (43)), as well as average duration of postural-transition (time needed for rising from a chair or sitting on a chair (44)). Mobility performance was recorded for a continuous period of 24-hour using a validated pendant sensor (PAMSys™, BioSensics LLC., MA, USA, Figure 1) worn during a non-dialysis day. We selected a non-dialysis day because the data during a day of dialysis could be biased by the long period of sitting/lying during HD process and the post dialysis fatigue. The PAMSys™ sensor contains a 3-axis accelerometer (sampling frequency of 50 Hz) and built-in memory for recording long-term data. The description of methods to extract metrics of interest was described in details in our previous studies (42–45).

Fig 1.
Fig 1.

A patient wearing the sensor as a pendant. Detailed metrics of mobility performance, including cumulated posture duration (sitting, lying, standing, and walking), daily walking performance (step count and number of unbroken walking bout), as well as postural-transition (daily number and average duration), were measured.

Statistical Analysis

All continuous data was presented as mean ± standard deviation. All categorical data was expressed as percentage. Analysis of variance (ANOVA) was used for between-group comparison of continuous demographics, clinical data, and mobility performance metrics. Analysis of Chi-square was used for comparison of categorical demographics and clinical data. Analysis of covariance (ANCOVA) was employed to compare differences between groups for mobility performance metrics, with adjustment for age and BMI. A 2-sided p<0.050 was considered to be statistically significant. The effect size for discriminating between groups was estimated using Cohen’s d effect size and represented as d (46). The Pearson correlation coefficient was used to evaluate the degree of agreement between mobility performance metrics and motor capacity variable for both groups with and without cognitive impairment. The correlation coefficient was also interpret as effect size (46, 47). A multivariate linear regression model was used to determine the association between mobility performance metrics and MMSE. In this model, MMSE was the dependent variable, and mobility performance metrics and demographics were the independent variables. R2 and p-value were calculated for the multivariate linear regression model. The Pearson correlation coefficient was used to evaluate the degree of agreement between the regression model and MMSE. Further, binary logistic regression analysis was employed to examine the relationship between each study variable and cognitive impairment. First, univariate logistic regression was employed to investigate the relationship of the test variables using “cognitive-impaired/cognitive-intact” as the dependent variable. Nagelkerke R Square (R2), odds ratio (OR), 95% confidence interval (95% CI), and p-value were calculated for each explanatory variable. Second, stepwise multivariate logistic regression, using variables found with p<0.20 in the univariate analysis, was performed to investigate independent effects of variables in predicting cognitive impairment. Then, these variables with independent effects were used to build models for prospective cognitive impairment prediction. In Model 1 (reference model), we only used demographics as independent variables. Then, to examine additional values of mobility performance metrics, two other models were examined. In Model 2, independent variables included demographics and daily walking performance. In Model 3, we added cumulated posture duration and postural-transition as additional independent variables. The receiver operating characteristic (ROC) curve and area-under-curve (AUC) were calculated for prediction models. All statistical analyses were performed using IBM SPSS Statistics 25 (IBM, IL, USA).

Results

Eighty-one subjects satisfied the inclusion and exclusion criteria of this study. However, the mobility performance data was available and valid for 69 subjects. Reasons of unavailable and invalid mobility performance data were refusal of wearing the sensor (n=9) and wearing duration less than 24-hour (n=3). Table 1 summarizes demographics, clinical data, and motor capacity of the remaining subjects. According to the MMSE, 44 subjects (64%) were classified as cognitive-intact, and 25 (36%) were classified as cognitive-impaired. The average MMSE score of the cognitive-impaired group was 22.6±3.7, which was significantly lower than the cognitive-intact group with 29.2±0.9 (p<0.001). The cognitive-impaired group was significantly older than the cognitive-intact group (p=0.001). Female percentage was significantly higher in the cognitive-impaired group (p=0.008). The cognitive-impaired group was shorter than the cognitive-intact group (p=0.009). But there was no between-group difference regarding the BMI. No between-group difference was observed for subjects’ weight, fall history, duration of HD, number of prescription medications, prevalence of depression, prevalence of frailty and pre-frailty, VPT, prevalence of DPN, prevalence of PAD, and HbA1c (p>0.050). No between group difference was observed for motor capacity metrics including standing balance and walking performance (p>0.050). For the dual-task walking, the cognitive-impaired group had lower dual-task walking speed than the cognitive-intact group. But the difference did not reach statistical significance.

View this table:
Table 1.

Demographics, clinical data, and motor capacity of the study population.

Table 2 summarizes between-group comparison for mobility performance metrics during 24-hour. The cognitive-impaired group spent significantly higher percentage of time in sitting and lying (d=0.78, p=0.005, Figure 2) but spent significantly lower percentage of time in standing (d=0.70, p=0.010, Figure 2) and walking (d=0.77, p=0.007, Figure 2). They also took significantly less steps (d=0.69, p=0.015) and unbroken walking bout (d=0.56, p=0.048) than the cognitive-intact group. Longer durations of sit-to-stand transition (d=0.37, p=0.143) and stand-to-sit transition (d=0.50, p=0.044) were observed in the cognitive-impaired group. Significant reductions of number of postural-transition were also observed in the cognitive-impaired group, including total number of transition to walk (d=0.60, p=0.035), number of stand-to-walk transition (d=0.60, p=0.036), number of walk-to-sit transition (d=0.65, p=0.020), total number of transition to stand (d=0.62, p=0.024), and number of walk-to-stand transition (d=0.58, p=0.044). When results were adjusted by demographic covariates including age and BMI, several mobility performance metrics remained significant for comparing between the cognitive-impaired and cognitive-intact groups (Table 2).

View this table:
Table 2.

Mobility performance (in 24-hour) comparison for cognitive-intact and cognitive-impaired groups.

Fig 2.
Fig 2.

Cumulated posture duration (as percentage of 24-hour) for the cognitive-intact group and cognitive-impaired group. Error bar represents the standard error. “d” denotes the Cohen’s d effect size. “*” denotes when the between-group comparison achieved a statistically significant level (p<0.050).

Figure 3 illustrates the correlation between motor capacity and mobility performance among HD patients with and without cognitive impairment. A significant correlation with medium effect size was observed between single-task walking speed and number of stand-to-sit transition among HD patients without cognitive impairment (r=0.39, p=0.012, Figure 3A). But the correlation among cognitive-impaired subjects was insignificant (r=-0.18, p=0.417). Similarly, a significant correlation with medium effect size was observed between single-task walking speed and number of sit-to-stand transition among HD patients without cognitive impairment (r=0.42, p=0.006, Figure 3B). But the correlation was diminished among cognitive-impaired subjects (r=-0.19, p=0.378).

Fig. 3.
Fig. 3.

Correlations between single-task walking speed and (A) number of stand-to-sit transition and (B) number of sit-to-stand transition among HD patients with and without cognitive impairment.

Results from the multivariate linear regression model (R2=0.400, p=0.019) revealed that “age” (B=-0.225, p<0.001) and “average duration of sit-to-stand transition” (B=-4.768, p=0.017) were independent predictors of MMSE. A significant correlation with large effect size of r=0.64 (p<0.001) was determined between the regression model and MMSE (Figure 4).

Fig. 4.
Fig. 4.

A significant correlation was observed between the multivariate linear regression model and MMSE.

In the univariate regression analysis, 5 variables in demographics and all variables in the mobility performance were associated with cognitive impairment (p<0.20) (Table 3). Two demographic variables and 11 mobility performance variables remained in the multivariate model suggesting that they are independent predictors (Table 3). These variables were used to build regression models. ROC curves for the 3 models were displayed in Figure 5. The AUC for Model 1 (demographics alone) was 0.76, with a sensitivity of 44.0%, specificity of 88.6%, and accuracy of 72.5% for predicting cognitive impairment. The AUC for Model 2 (demographics + daily walking performance) was 0.78, with a sensitivity of 44.0%, specificity of 79.5%, and accuracy of 66.7% for predicting cognitive impairment. The highest AUC (0.93) was obtained by Model 3 (demographics + daily walking performance + cumulated posture duration + postural-transition), with a sensitivity of 72.0%, specificity of 93.2%, and accuracy of 85.5% for distinguishing cognitive-impaired cases.

View this table:
Table 3.

Results of univariate and multivariate logistic regression.

Fig 5.
Fig 5.

ROCs of different models for predicting cognitive impairment: Model 1 used “demographics” (AUC = 0.76), Model 2 used a combination of “demographics” and “daily walking performance” (AUC = 0.78), and Model 3 used a combination of “demographics”, “daily walking performance”, “cumulate posture duration”, and “postural-transition” (AUC = 0.93).

Discussions

To our knowledge, this is the first study to investigate the association between mobility performance and cognitive condition in patients with diabetes and ESRD undergoing HD process. The results suggest that although HD patients with and without cognitive impairment have similar motor capacity, those with cognitive impairment have lower mobility performance. We were able to conform our hypothesis that mobility performance metrics during a non-dialysis day could be used as potential digital biomarkers of cognitive impairment among HD patients. Specifically, several mobility performance metrics measurable using a pendant sensor enable significant discrimination between those with and without cognitive impairment with medium effect size (maximum Cohen’s d=0.78). In addition, a metric constructed by the combination of demographics and mobility performance metrics yields a significant correlation with large effect size with the MMSE (r=0.64, p<0.001). By adding mobility performance together with demographics into the binary logistic regression model, it enables distinguishing between those with and without cognitive impairment. This combined model yields relatively high sensitivity, specificity, and accuracy, which is superior to using demographics alone. Our results also suggest that despite cognitive-impaired HD patients have poor daily walking performance, just monitoring daily walking performance may not be sufficient to distinguish those with cognitive impairment. Additional mobility performance metrics, including cumulated posture duration and postural-transition, could increase the AUC from 0.78 to 0.93 for detection of cognitive-impaired cases.

Previous studies investigating association between mobility performance and cognitive impairment showed that activity level and daily steps are positively associated with cognitive function in older adults (23–28). Results of this study are in line with the previous studies. They showed that cognitive-impaired HD patients have lower walking percentage and step count than cognitive-intact HD patients. Additionally, we found the cognitive-impaired HD patients have less number of postural-transition than cognitive-intact HD patients during daily living. The limited number of postural-transition has been identified as a factor which may contribute to the muscle weakness and activity limitations, causing physical frailty (43, 48). Frailty together with cognitive impairment (known as ‘cognitive frailty’) has been shown to be a strong and independent predictor of further cognitive decline over time (49, 50).

Mobility performance in daily life depends not only on motor capacity, but also on intact cognitive function and psychosocial factors (51). Studies have shown that cognitive impairment is associated with reduced mobility performance (51–53). However, an individual’s scores in supervised tests are poorly related to mobility performance in real life (51–53). Results of this study show that among cognitive-intact HD patients, mobility performance is associated with motor capacity. However among HD patients with cognitive impairment, motor capacity is poorly related to mobility performance. This demonstrates that cognitive function is a moderator between motor capacity and mobility performance among patients undergoing HD process. This is aligned with the study of Feld et al. (54), in which it was demonstrated that gait speed does not adequately predict whether stroke survivors would be active in the community. Similar observation was reported by Toosizadeh et al. study (55), in which no agreement between motor capacity and mobility performance was observed among people with Parkinson’s disease, while a significant agreement was observed among age-matched healthy controls.

In previous studies, to better link motor capacity with cognitive decline, dual-task walking test was proposed (56). By adding cognitive challenges into motor task, the dual-task walking speed can expose cognitive deficits through the evaluation of locomotion. Previous studies have shown that dual-task walking speed for cognitive-impaired older adults was statistically lower than cognitive-intact ones among non-dialysis population (57). Surprisingly, we didn’t observe significant between-group difference in our sample. A previous systematic review has pointed out that older adults with mobility limitation are more likely to prioritize motor performance over cognitive performance (58). We speculate that because of the poor motor capacity among HD population, subjects would prioritize motor task over cognitive task. Thus the effect of cognitive impairment may not be noticeable in this motor-impaired population by dual-task walking speed. If this can be confirmed in the follow up study, it may suggest that dual-task paradigm may not be a sufficient test to determine cognitive deficit among population with poor motor capacity.

In this study, we found the cognitive-impaired group had higher percentage of female. This finding is in line with the previous studies (59, 60). For example, Beam et al. examined gender differences in incidence rates of any dementia, Alzheimer’s disease (AD) alone, and non-Alzheimer’s dementia alone in 16926 women and men in the Swedish Twin Registry aged 65+. They reported that incidence rates of any dementia and AD were greater in women than men, particularly in older ages (age of 80 years and older) (59). Similarly, Wang et al. suggested that females compared to males showed significantly worse performance in cognitive function (60). In this study, we did not adjust the results by gender because previous studies have demonstrated that gender does not affect mobility performance in HD population (61–64).

A major limitation of this study is the relatively low sample size, which could be underpowered for the clinical conclusion. On the other hand, this study could be considered as a cohort study as all participants were recruited from the Fahad Bin Jassim Kidney Center of Hamad Medical Corporation, which supports the majority of HD patients in the state of Qatar. All eligible subjects who received HD in this center were offered to participate in this study. Another limitation of this study is that mobility performance metrics were only measured in a single non-dialysis day. We excluded mobility performance monitoring during the dialysis day because we anticipated that data could be biased by the long process of HD (often 4-hour). Patients are holding a sitting or lying posture during the HD process. They also suffer the post-dialysis fatigue on the dialysis day. In addition, the measured single-day mobility performance may not be able to accurately represent the condition of HD patients (including both weekdays and weekends). Several previous literature reported three or more days of accelerometry data may more reliably and accurately model mobility performance in adult population (65, 66). It would be interesting to investigate whether multiple days of monitoring could model mobility performance more accurately in HD patients in the future study, since HD patients may have fluctuation in mobility performance due to post-dialysis fatigue and change of renal function (67).

Conclusion

This study suggests that mobility performance metrics remotely measurable using a pendant sensor during a non-dialysis day could be served as potential digital biomarkers of cognitive impairment among HD patients. Interestingly, motor capacity metrics, even assessed under the cognitively demanding condition, are not sensitive to cognitive impairment among HD patients. Results suggest that despite cognitive-impaired HD patients have poor daily walking performance, just monitoring daily walking performance may not be sufficient to determine cognitive impairment cases. Additional mobility performance metrics such as cumulated posture duration and postural-transition can improve the discriminating power. Further researches are encouraged to evaluate the ability of sensor-derived mobility performance metrics to determine early cognitive impairment or dementia, as well as to track potential change in cognitive impairment over time in response to HD process. Future studies are also recommended for the potential use of sensor-derived metrics to determine modifiable factors, which may contribute in cognitive decline among HD patients.

Conflict of Interest Statement

None.

Authors’ Contributions

H.Z. wrote the manuscript and contributed in data analysis. C.W. contributed in drafting the manuscript. A.H., R.I., and T.T. contributed in data collection. F.A.-A. and B.N. contributed in study design, securing funding, and supervising the study. All authors contributed in interpretation of results and critical revision of the study.

Funding

Partial support was provided by the Qatar National Research Foundation (Award number: NPRP 7-1595-3-405 and NPRP 10-0208-170400). The content is solely the responsibility of the authors and does not necessarily represent the official views of sponsors.

Acknowledgments

We thank Mincy Mathew, Priya Helena Peterson, Ana Enriquez, and Mona Amirmazaheri for assisting with data collection.

References

  1. 1.↵
    Wolfgram DF. Filtering the Evidence: Is There a Cognitive Cost of Hemodialysis? Journal of the American Society of Nephrology: JASN. 2018 Apr;29(4):1087–9. PubMed PMID: 29496889. Pubmed Central PMCID: 5875964.
    OpenUrlFREE Full Text
  2. 2.↵
    Ying I, Levitt Z, Jassal SV. Should an elderly patient with stage V CKD and dementia be started on dialysis? Clinical journal of the American Society of Nephrology: CJASN. 2014 May;9(5):971–7. PubMed PMID: 24235287. Pubmed Central PMCID: 4011441.
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    Sehgal AR, Grey SF, DeOreo PB, Whitehouse PJ. Prevalence, recognition, and implications of mental impairment among hemodialysis patients. American journal of kidney diseases: the official journal of the National Kidney Foundation. 1997 Jul;30(1):41–9. PubMed PMID: 9214400.
    OpenUrlPubMed
  4. 4.↵
    Murray AM, Tupper DE, Knopman DS, Gilbertson DT, Pederson SL, Li S, et al. Cognitive impairment in hemodialysis patients is common. Neurology. 2006 Jul 25;67(2):216–23. PubMed PMID: 16864811.
    OpenUrlCrossRefPubMed
  5. 5.↵
    Kurella Tamura M, Larive B, Unruh ML, Stokes JB, Nissenson A, Mehta RL, et al. Prevalence and correlates of cognitive impairment in hemodialysis patients: the Frequent Hemodialysis Network trials. Clinical journal of the American Society of Nephrology: CJASN. 2010 Aug;5(8):1429–38. PubMed PMID: 20576825. Pubmed Central PMCID: 2924414.
    OpenUrlAbstract/FREE Full Text
  6. 6.↵
    Fazekas G, Fazekas F, Schmidt R, Kapeller P, Offenbacher H, Krejs GJ. Brain MRI findings and cognitive impairment in patients undergoing chronic hemodialysis treatment. Journal of the neurological sciences. 1995 Dec;134(1-2):83–8. PubMed PMID: 8747848.
    OpenUrlCrossRefPubMedWeb of Science
  7. 7.↵
    Tyrrell J, Paturel L, Cadec B, Capezzali E, Poussin G. Older patients undergoing dialysis treatment: cognitive functioning, depressive mood and health-related quality of life. Aging & mental health. 2005 Jul;9(4):374–9. PubMed PMID: 16019295.
    OpenUrlCrossRefPubMedWeb of Science
  8. 8.↵
    Bradford A, Kunik ME, Schulz P, Williams SP, Singh H. Missed and delayed diagnosis of dementia in primary care: prevalence and contributing factors. Alzheimer disease and associated disorders. 2009 Oct-Dec;23(4):306–14. PubMed PMID: 19568149. Pubmed Central PMCID: 2787842.
    OpenUrlCrossRefPubMedWeb of Science
  9. 9.↵
    O’Bryant SE, Humphreys JD, Smith GE, Ivnik RJ, Graff-Radford NR, Petersen RC, et al. Detecting dementia with the mini-mental state examination in highly educated individuals. Archives of neurology. 2008 Jul;65(7):963–7. PubMed PMID: 18625866. Pubmed Central PMCID: 2587038.
    OpenUrlCrossRefPubMedWeb of Science
  10. 10.↵
    Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society. 2005 Apr;53(4):695–9. PubMed PMID: 15817019.
    OpenUrlCrossRefPubMedWeb of Science
  11. 11.↵
    Reitan RM. Validity of the Trail Making Test as an Indicator of Organic Brain Damage. Perceptual and motor skills. 1958;8(3):271–6.
    OpenUrlCrossRef
  12. 12.↵
    Ashendorf L, Jefferson AL, O’Connor MK, Chaisson C, Green RC, Stern RA. Trail Making Test errors in normal aging, mild cognitive impairment, and dementia. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists. 2008 Mar;23(2):129–37. PubMed PMID: 18178372. Pubmed Central PMCID: 2693196.
    OpenUrlCrossRefPubMedWeb of Science
  13. 13.↵
    van der Vleuten CP, van Luyk SJ, van Ballegooijen AM, Swanson DB. Training and experience of examiners. Medical education. 1989 May;23(3):290–6. PubMed PMID: 2725369.
    OpenUrlPubMedWeb of Science
  14. 14.↵
    Kurella Tamura M, Yaffe K. Dementia and cognitive impairment in ESRD: diagnostic and therapeutic strategies. Kidney international. 2011 Jan;79(1):14–22. PubMed PMID: 20861818. Pubmed Central PMCID: 3107192.
    OpenUrlCrossRefPubMedWeb of Science
  15. 15.↵
    Gesualdo GD, Duarte JG, Zazzetta MS, Kusumota L, Say KG, Pavarini SCI, et al. Cognitive impairment of patients with chronic renal disease on hemodialysis and its relationship with sociodemographic and clinical characteristics. Dementia & neuropsychologia. 2017 Jul-Sep;11(3):221–6. PubMed PMID: 29213518. Pubmed Central PMCID: 5674665.
    OpenUrlPubMed
  16. 16.↵
    Razjouyan J, Grewal GS, Rishel C, Parthasarathy S, Mohler J, Najafi B. Activity Monitoring and Heart Rate Variability as Indicators of Fall Risk: Proof-of-Concept for Application of Wearable Sensors in the Acute Care Setting. Journal of gerontological nursing. 2017 Jul 1;43(7):53–62. PubMed PMID: 28253410.
    OpenUrlPubMed
  17. 17.
    Razjouyan J, Naik AD, Horstman MJ, Kunik ME, Amirmazaheri M, Zhou H, et al. Wearable Sensors and the Assessment of Frailty among Vulnerable Older Adults: An Observational Cohort Study. Sensors. 2018 Apr 26;18(5). PubMed PMID: 29701640. Pubmed Central PMCID: 5982667.
    OpenUrlPubMed
  18. 18.↵
    Aminian K, Najafi B. Capturing human motion using body-fixed sensors: outdoor measurement and clinical applications. Computer Animation and Virtual Worlds. 2004;15(2):79–94.
    OpenUrlCrossRef
  19. 19.↵
    Jansen CP, Toosizadeh N, Mohler MJ, Najafi B, Wendel C, Schwenk M. The association between motor capacity and mobility performance: frailty as a moderator. European review of aging and physical activity: official journal of the European Group for Research into Elderly and Physical Activity. 2019;16:16. PubMed PMID: 31624506. Pubmed Central PMCID: 6787993.
    OpenUrlPubMed
  20. 20.↵
    Organization WH. International classification of functioning, disability and health: ICF: Geneva: World Health Organization; 2001.
  21. 21.↵
    Plehn K, Marcopulos BA, McLain CA. The relationship between neuropsychological test performance, social functioning, and instrumental activities of daily living in a sample of rural older adults. The Clinical neuropsychologist. 2004 Feb;18(1):101–13. PubMed PMID: 15595362.
    OpenUrlCrossRefPubMedWeb of Science
  22. 22.↵
    Yogev-Seligmann G, Hausdorff JM, Giladi N. The role of executive function and attention in gait. Movement disorders: official journal of the Movement Disorder Society. 2008 Feb 15;23(3):329–42; quiz 472. PubMed PMID: 18058946. Pubmed Central PMCID: 2535903.
    OpenUrlCrossRefPubMedWeb of Science
  23. 23.↵
    Etgen T, Sander D, Huntgeburth U, Poppert H, Forstl H, Bickel H. Physical activity and incident cognitive impairment in elderly persons: the INVADE study. Archives of internal medicine. 2010 Jan 25;170(2):186–93. PubMed PMID: 20101014.
    OpenUrlCrossRefPubMedWeb of Science
  24. 24.
    Weuve J, Kang JH, Manson JE, Breteler MM, Ware JH, Grodstein F. Physical activity, including walking, and cognitive function in older women. Jama. 2004 Sep 22;292(12):1454–61. PubMed PMID: 15383516.
    OpenUrlCrossRefPubMedWeb of Science
  25. 25.↵
    Laurin D, Verreault R, Lindsay J, MacPherson K, Rockwood K. Physical activity and risk of cognitive impairment and dementia in elderly persons. Archives of neurology. 2001 Mar;58(3):498–504. PubMed PMID: 11255456.
    OpenUrlCrossRefPubMedWeb of Science
  26. 26.↵
    Brown BM, Peiffer JJ, Sohrabi HR, Mondal A, Gupta VB, Rainey-Smith SR, et al. Intense physical activity is associated with cognitive performance in the elderly. Translational psychiatry. 2012 Nov 20;2:e191. PubMed PMID: 23168991. Pubmed Central PMCID: 3565765.
    OpenUrlCrossRefPubMed
  27. 27.↵
    Buchman AS, Wilson RS, Bennett DA. Total daily activity is associated with cognition in older persons. The American journal of geriatric psychiatry: official journal of the American Association for Geriatric Psychiatry. 2008 Aug;16(8):697–701. PubMed PMID: 18669949.
    OpenUrlCrossRefPubMedWeb of Science
  28. 28.↵
    Blumenthal JA, Smith PJ, Mabe S, Hinderliter A, Welsh-Bohmer K, Browndyke JN, et al. Lifestyle and Neurocognition in Older Adults With Cardiovascular Risk Factors and Cognitive Impairment. Psychosomatic medicine. 2017 Jul/Aug;79(6):719–27. PubMed PMID: 28437380. Pubmed Central PMCID: 5493327.
    OpenUrlPubMed
  29. 29.↵
    Loney T, Standage M, Thompson D, Sebire SJ, Cumming S. Self-report vs. objectively assessed physical activity: which is right for public health? Journal of physical activity & health. 2011 Jan;8(1):62–70. PubMed PMID: 21297186.
    OpenUrlPubMed
  30. 30.↵
    de Bruin ED, Najafi B, Murer K, Uebelhart D, Aminian K. Quantification of everyday motor function in a geriatric population. Journal of rehabilitation research and development. 2007;44(3):417–28. PubMed PMID: 18247238.
    OpenUrlCrossRefPubMedWeb of Science
  31. 31.↵
    Zhou H, Al-Ali F, Rahemi H, Kulkarni N, Hamad A, Ibrahim R, et al. Hemodialysis Impact on Motor Function beyond Aging and Diabetes-Objectively Assessing Gait and Balance by Wearable Technology. Sensors. 2018 Nov 14;18(11). PubMed PMID: 30441843. Pubmed Central PMCID: 6263479.
    OpenUrlPubMed
  32. 32.↵
    Weissman MM, Sholomskas D, Pottenger M, Prusoff BA, Locke BZ. Assessing depressive symptoms in five psychiatric populations: a validation study. American journal of epidemiology. 1977 Sep;106(3):203–14. PubMed PMID: 900119.
    OpenUrlPubMedWeb of Science
  33. 33.↵
    Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, Gottdiener J, et al. Frailty in older adults: evidence for a phenotype. The journals of gerontology Series A, Biological sciences and medical sciences. 2001 Mar;56(3):M146–56. PubMed PMID: 11253156.
    OpenUrlCrossRefPubMedWeb of Science
  34. 34.↵
    Bracewell N, Game F, Jeffcoate W, Scammell BE. Clinical evaluation of a new device in the assessment of peripheral sensory neuropathy in diabetes. Diabetic medicine: a journal of the British Diabetic Association. 2012 Dec;29(12):1553–5. PubMed PMID: 22672257.
    OpenUrlCrossRefPubMed
  35. 35.↵
    Wyatt MF, Stickrath C, Shah A, Smart A, Hunt J, Casserly IP. Ankle-brachial index performance among internal medicine residents. Vascular medicine. 2010 Apr;15(2):99–105. PubMed PMID: 20133343.
    OpenUrlCrossRefPubMed
  36. 36.↵
    Organization WH. Use of glycated haemoglobin (HbA1c) in the diagnosis of diabetes mellitus. 2011. Geneva (Switzerland): The Organization Google Scholar. 2011.
  37. 37.↵
    Bahureksa L, Najafi B, Saleh A, Sabbagh M, Coon D, Mohler MJ, et al. The Impact of Mild Cognitive Impairment on Gait and Balance: A Systematic Review and Meta-Analysis of Studies Using Instrumented Assessment. Gerontology. 2017;63(1):67–83. PubMed PMID: 27172932. Pubmed Central PMCID: 5107359.
    OpenUrlPubMed
  38. 38.↵
    Najafi B, Horn D, Marclay S, Crews RT, Wu S, Wrobel JS. Assessing postural control and postural control strategy in diabetes patients using innovative and wearable technology. Journal of diabetes science and technology. 2010 Jul 1;4(4):780–91. PubMed PMID: 20663438. Pubmed Central PMCID: 2909506.
    OpenUrlCrossRefPubMed
  39. 39.↵
    Aminian K, Najafi B, Bula C, Leyvraz PF, Robert P. Spatio-temporal parameters of gait measured by an ambulatory system using miniature gyroscopes. Journal of biomechanics. 2002 May;35(5):689–99. PubMed PMID: 11955509.
    OpenUrlCrossRefPubMedWeb of Science
  40. 40.↵
    Luck T, Then FS, Luppa M, Schroeter ML, Arelin K, Burkhardt R, et al. Association of the apolipoprotein E genotype with memory performance and executive functioning in cognitively intact elderly. Neuropsychology. 2015 May;29(3):382–7. PubMed PMID: 25365563.
    OpenUrlPubMed
  41. 41.↵
    Damian AM, Jacobson SA, Hentz JG, Belden CM, Shill HA, Sabbagh MN, et al. The Montreal Cognitive Assessment and the mini-mental state examination as screening instruments for cognitive impairment: item analyses and threshold scores. Dementia and geriatric cognitive disorders. 2011;31(2):126–31. PubMed PMID: 21282950.
    OpenUrlCrossRefPubMed
  42. 42.↵
    Najafi B, Aminian K, Paraschiv-Ionescu A, Loew F, Bula CJ, Robert P. Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE transactions on bio-medical engineering. 2003 Jun;50(6):711–23. PubMed PMID: 12814238.
    OpenUrlCrossRefPubMedWeb of Science
  43. 43.↵
    Parvaneh S, Mohler J, Toosizadeh N, Grewal GS, Najafi B. Postural Transitions during Activities of Daily Living Could Identify Frailty Status: Application of Wearable Technology to Identify Frailty during Unsupervised Condition. Gerontology. 2017;63(5):479–87. PubMed PMID: 28285311. Pubmed Central PMCID: 5561495.
    OpenUrlPubMed
  44. 44.↵
    Najafi B, Aminian K, Loew F, Blanc Y, Robert PA. Measurement of stand-sit and sit-stand transitions using a miniature gyroscope and its application in fall risk evaluation in the elderly. IEEE transactions on bio-medical engineering. 2002 Aug;49(8):843–51. PubMed PMID: 12148823.
    OpenUrlCrossRefPubMedWeb of Science
  45. 45.↵
    Lindberg CM, Srinivasan K, Gilligan B, Razjouyan J, Lee H, Najafi B, et al. Effects of office workstation type on physical activity and stress. Occup Environ Med. 2018 Oct;75(10):689–95. PubMed PMID: 30126872. Pubmed Central PMCID: PMC6166591.
    OpenUrlAbstract/FREE Full Text
  46. 46.↵
    Cohen J. Statistical power analysis for the behavioral sciences. 2nd. Hillsdale, NJ: erlbaum; 1988.
  47. 47.↵
    Rosenthal JA. Qualitative descriptors of strength of association and effect size. Journal of social service Research. 1996;21(4):37–59.
    OpenUrlCrossRefWeb of Science
  48. 48.↵
    Bohannon RW. Daily sit-to-stands performed by adults: a systematic review. Journal of physical therapy science. 2015 Mar;27(3):939–42. PubMed PMID: 25931764. Pubmed Central PMCID: 4395748.
    OpenUrlPubMed
  49. 49.↵
    Zhou H, Lee H, Lee J, Schwenk M, Najafi B. Motor Planning Error: Toward Measuring Cognitive Frailty in Older Adults Using Wearables. Sensors. 2018 Mar 20;18(3). PubMed PMID: 29558436. Pubmed Central PMCID: 5876674.
    OpenUrlPubMed
  50. 50.↵
    Zhou H, Razjouyan J, Halder D, Naik AD, Kunik ME, Najafi B. Instrumented Trail-Making Task: Application of Wearable Sensor to Determine Physical Frailty Phenotypes. Gerontology. 2019;65(2):186–97. PubMed PMID: 30359976. Pubmed Central PMCID: 6426658.
    OpenUrlPubMed
  51. 51.↵
    Giannouli E, Bock O, Mellone S, Zijlstra W. Mobility in Old Age: Capacity Is Not Performance. BioMed research international. 2016;2016:3261567. PubMed PMID: 27034932. Pubmed Central PMCID: 4789440.
    OpenUrlPubMed
  52. 52.
    Kaspar R, Oswald F, Wahl HW, Voss E, Wettstein M. Daily mood and out-of-home mobility in older adults: does cognitive impairment matter? Journal of applied gerontology: the official journal of the Southern Gerontological Society. 2015 Feb;34(1):26–47. PubMed PMID: 25548087.
    OpenUrlCrossRefPubMed
  53. 53.↵
    Verhaeghen P, Martin M, Sedek G. Reconnecting cognition in the lab and cognition in real life: The role of compensatory social and motivational factors in explaining how cognition ages in the wild. Neuropsychology, development, and cognition Section B, Aging, neuropsychology and cognition. 2012;19(1-2):1–12. PubMed PMID: 22313173. Pubmed Central PMCID: 3775600.
    OpenUrlPubMed
  54. 54.↵
    Feld JA, Zukowski LA, Howard AG, Giuliani CA, Altmann LJP, Najafi B, et al. Relationship Between Dual-Task Gait Speed and Walking Activity Poststroke. Stroke. 2018 May;49(5):1296–8. PubMed PMID: 29622624. Pubmed Central PMCID: PMC6034633.
    OpenUrlAbstract/FREE Full Text
  55. 55.↵
    Toosizadeh N, Mohler J, Lei H, Parvaneh S, Sherman S, Najafi B. Motor Performance Assessment in Parkinson’s Disease: Association between Objective In-Clinic, Objective In-Home, and Subjective/Semi-Objective Measures. PLoS One. 2015;10(4):e0124763. PubMed PMID: 25909898. Pubmed Central PMCID: PMC4409065.
    OpenUrlCrossRefPubMed
  56. 56.↵
    Montero-Odasso M, Muir SW, Speechley M. Dual-task complexity affects gait in people with mild cognitive impairment: the interplay between gait variability, dual tasking, and risk of falls. Archives of physical medicine and rehabilitation. 2012 Feb;93(2):293–9. PubMed PMID: 22289240.
    OpenUrlCrossRefPubMed
  57. 57.↵
    Muir SW, Speechley M, Wells J, Borrie M, Gopaul K, Montero-Odasso M. Gait assessment in mild cognitive impairment and Alzheimer’s disease: the effect of dual-task challenges across the cognitive spectrum. Gait & posture. 2012 Jan;35(1):96–100. PubMed PMID: 21940172.
    OpenUrlCrossRefPubMedWeb of Science
  58. 58.↵
    Wollesen B, Wanstrath M, van Schooten KS, Delbaere K. A taxonomy of cognitive tasks to evaluate cognitive-motor interference on spatiotemoporal gait parameters in older people: a systematic review and meta-analysis. European review of aging and physical activity: official journal of the European Group for Research into Elderly and Physical Activity. 2019;16:12. PubMed PMID: 31372186. Pubmed Central PMCID: 6661106.
    OpenUrlPubMed
  59. 59.↵
    Beam CR, Kaneshiro C, Jang JY, Reynolds CA, Pedersen NL, Gatz M. Differences Between Women and Men in Incidence Rates of Dementia and Alzheimer’s Disease. Journal of Alzheimer’s disease: JAD. 2018;64(4):1077–83. PubMed PMID: 30010124. Pubmed Central PMCID: 6226313.
    OpenUrlPubMed
  60. 60.↵
    Wang L, Tian T, Alzheimer’s Disease Neuroimaging I. Gender Differences in Elderly With Subjective Cognitive Decline. Frontiers in aging neuroscience. 2018;10:166. PubMed PMID: 29915534. Pubmed Central PMCID: 5994539.
    OpenUrlPubMed
  61. 61.↵
    Avesani CM, Trolonge S, Deleaval P, Baria F, Mafra D, Faxen-Irving G, et al. Physical activity and energy expenditure in haemodialysis patients: an international survey. Nephrology, dialysis, transplantation: official publication of the European Dialysis and Transplant Association - European Renal Association. 2012 Jun;27(6):2430–4. PubMed PMID: 22172727.
    OpenUrlCrossRefPubMedWeb of Science
  62. 62.
    Johansen KL, Chertow GM, Ng AV, Mulligan K, Carey S, Schoenfeld PY, et al. Physical activity levels in patients on hemodialysis and healthy sedentary controls. Kidney international. 2000 Jun;57(6):2564–70. PubMed PMID: 10844626.
    OpenUrlCrossRefPubMedWeb of Science
  63. 63.
    Wong SW, Chan YM, Lim TS. Correlates of physical activity level among hemodialysis patients in Selangor, Malaysia. Malaysian journal of nutrition. 2011 Dec;17(3):277–86. PubMed PMID: 22655450.
    OpenUrlPubMed
  64. 64.↵
    Araújo Filho JCd, Amorim CTd, Brito ACNdL, Oliveira DSd, Lemos A, Marinho PÉdM. Physical activity level of patients on hemodialysis: a cross-sectional study. Fisioterapia e Pesquisa. 2016;23(3):234–40.
    OpenUrl
  65. 65.↵
    Hart TL, Swartz AM, Cashin SE, Strath SJ. How many days of monitoring predict physical activity and sedentary behaviour in older adults? The international journal of behavioral nutrition and physical activity. 2011 Jun 16;8:62. PubMed PMID: 21679426. Pubmed Central PMCID: 3130631.
    OpenUrlCrossRefPubMed
  66. 66.↵
    Tudor-Locke C, Burkett L, Reis JP, Ainsworth BE, Macera CA, Wilson DK. How many days of pedometer monitoring predict weekly physical activity in adults? Preventive medicine. 2005 Mar;40(3):293–8. PubMed PMID: 15533542.
    OpenUrlCrossRefPubMedWeb of Science
  67. 67.↵
    Jhamb M, Weisbord SD, Steel JL, Unruh M. Fatigue in patients receiving maintenance dialysis: a review of definitions, measures, and contributing factors. American journal of kidney diseases: the official journal of the National Kidney Foundation. 2008 Aug;52(2):353–65. PubMed PMID: 18572290. Pubmed Central PMCID: 2582327.
    OpenUrlCrossRefPubMedWeb of Science
Back to top
Posted November 05, 2019.
Download PDF
Email
Harnessing Digital Health to Objectively Assess Cognitive Impairment in People undergoing Hemodialysis Process: The Impact of Cognitive Impairment on Mobility Performance Measured by Wearables
He Zhou, Fadwa Al-Ali, Changhong Wang, Abdullah Hamad, Rania Ibrahim, Talal Talal, Bijan Najafi
bioRxiv 831545; doi: https://doi.org/10.1101/831545
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools

Subject Area