Real-time feedback reduces participant motion during task-based fMRI

Abstract

The potential negative impact of head movement during fMRI has long been appreciated. Although a variety of prospective and retrospective approaches have been developed to help mitigate these effects, reducing head movement in the first place remains the most appealing strategy for optimizing data quality. Real-time interventions, in which participants are provided feedback regarding their scan-to-scan motion, have recently shown promise in reducing motion during resting state fMRI. However, whether feedback might similarly reduce motion during task-based fMRI is an open question. In particular, it is unclear whether participants can effectively monitor motion feedback while attending to task-related demands. Here we assessed whether a combination of real-time and between-run feedback could reduce head motion during task-based fMRI. During an auditory word repetition task, 78 adult participants (aged 18–81) were pseudorandomly assigned to receive feedback, implemented in FIRMM software, or not. We quantified movement using framewise displacement (FD). We found that motion feedback resulted in a statistically significant reduction in participant head motion, with a small-to-moderate effect size (reducing average FD from 0.347 to 0.282). There was some evidence for a change of effect over the course of six runs, but no clear evidence for practice effects based on motion feedback. We conclude that under some circumstances real-time feedback may reduce head motion during task-based fMRI, although its effectiveness may depend on the specific participant population and task demands of a given study.

Introduction

Head movement during fMRI can introduce a number of different challenges to data analysis. Although long appreciated (Friston et al., 1996), the detrimental influence of head motion on resting state functional connectivity has only recently been widely recognized (Power et al., 2015). Rigid body realignment—a mainstay of fMRI analysis for decades—goes some way towards improving correspondence across images (Ashburner and Friston, 2004), but does not remove extraneous signal components introduced by movement (Friston et al., 1996). A common approach for mitigating motion-related artifacts is to include the 6 realignment parameters (translation and rotation around the X, Y, and Z axes) as nuisance regressors in first-level models. Additional approaches include wavelet despiking (Patel et al., 2014), ICA (Pruim et al., 2015), robust-weighted least squares (Diedrichsen and Shadmehr, 2005), Bayesian approaches (Eklund et al., 2017), and frame censoring (Lemieux et al., 2007).

However, recent work investigating motion correction strategies in multiple data sets suggests that the optimal strategy may depend on the specific data set and output metric (Jones et al., 2022), precluding a simple one-size-fits-all solution.

Given the lack of certainty regarding how to best handle head motion during analysis, reducing head motion in the first place is a particularly appealing option. Common approaches to reducing head motion during MRI scanning include foam padding and instructing participants to not move during the scan. Other existing strategies include using custom head molds (Power et al., 2019) and playing participants a relaxing abstract movie (Vanderwal et al., 2015).

A complementary approach for reducing head motion is to provide participants with feedback regarding their head movement so they can learn to minimize motion. FIRMM software¹ (Dosenbach et al., 2017) is one implementation of motion feedback. FIRMM uses rapid image reconstruction and rigid-body alignment to estimate frame-by-frame movement, providing visual cues to participants based on estimated movement. Feedback can either be provided in real time, or between scanning runs. FIRMM has been shown to reduce movement during resting state scans in both adults (Dosenbach et al., 2017) and in young children (Greene et al., 2018; Badke D’Andrea et al., 2022).

Despite its success in resting state fMRI, it is unclear whether motion feedback would lead to similar improvements during a typical task-based fMRI paradigm. Task-based paradigms have been shown to produce greater head motion both in terms of displacement and rotation relative to resting state scans (Huang, 2019), and could represent a natural target for motion feedback. On the one hand, if participants are able to modulate their motion in one experimental paradigm (namely, resting state studies) it stands to reason they could do so in a different experimental context (a task-based study). On the other hand, task-based fMRI studies require participants to attend to the experimental task at hand, which will reduce the cognitive resources available for noting and processing the feedback, as well as the ability to attend to proprioceptive cues related to self-motion. In addition, if experimental stimuli and motion feedback are in the same modality, subjects may also have difficulty accurately processing motion-related feedback.

To evaluate the degree to which feedback might affect head motion during task-based fMRI, we assigned participants in a task-based fMRI study of spoken word recognition to receive feedback (the feedback group) or not (the no-feedback group). All other instructions, scanning parameters, and task requirements were identical. We included both young and older adults, and a task (word repetition) that we anticipated would typically require head movement. Finally, we were able to capitalize on the fact that each subject has many observations (hundreds of frames) by using linear mixed effects analysis.

Method

MRI data are available from OpenNeuro(Markiewicz et al., 2021) at https://doi.org/10.18112/openneuro.ds004285.v1.0.0, reference number ds004285, and motion data and analysis scripts are available from https://github.com/jpeelle/motion-feedback. (Although we collected these data in the context of a task-based fMRI study, described below, here we focus only on effects of feedback on head motion and not the task results.)

Participants

We tested 78 participants aged 19–81. All reported themselves to have normal hearing, to be right handed, and not have a history of neurological disease. There were no exclusions based on sex, gender, race, or ethnicity. Participants were limited to adults because of our scientific focus on adult language processing. Because the task involved spoken language, all were native speakers of American English.

Participants were randomly assigned to control (no motion feedback) or intervention (real-time motion feedback) groups. Demographic characteristics for these groups are shown in Table 1. Because data collection was ended early due to COVID-19 safety requirements the groups did not have equal numbers of participants. All provided informed consent under a process approved by the Washington University in Saint Louis Institutional Review Board.

Table 1.

Participant characteristics

Procedure

The procedure is modeled after that in Rogers et al. (2020) and illustrated in Figure 1. Participants performed a word repetition task in which on every trial they heard a spoken word, presented in stationary background noise (3 dB SNR), and repeated it back aloud.

Figure 1.

Summary of task design and processing pipeline. a) Schematic of the word repetition task. Participants heard a word presented in the midst of background noise during the gap between volume acquisitions; during the following gap, they repeated the word aloud. b) Illustration of the two groups to which a participant could be pseudorandomly assigned. The group without feedback received standard instructions to keep still at the beginning of the study; the feedback group received instructions regarding real-time feedback, a colored crosshair indicating their scan-to-scan motion (framewise displacement; FD) during each run; and a visual report on feedback at the end of each run. c) Illustration of time variables available for modeling.

The no-feedback group was given the following instructions:

During this task, it is important that you hold your body and head very still. Please stay relaxed, stay alert, and keep your eyes open and on the fixation cross.

FIRMM was run in the background for experimenters to monitor motion during scanning sequences and participants were given verbal feedback after each sequence.

The group receiving motion feedback was given the following instructions:

It is very important to remain still during your MRI so that we can obtain clear images. Even very small movements that you are not aware of can affect the image quality. While performing the task, you will be receiving feedback corresponding to your ability to remain still. This is to help you be aware of any movements you may be making. You will see a white fixation cross on the screen. The cross will change to yellow and then red depending on how much you are moving. It will go back to white if you become still again. Sometimes even if you are doing your best you will still see a red cross. This may mean that the computer is too strict and not that you are necessarily doing anything wrong. Just keep trying your best to keep the cross in the white.

During the task, participants viewed a colored cross in the middle of the screen. FD thresholds were set to show a white cross at < 0.2 mm, a yellow cross at 0.2 mm to < 0.3 mm, and a red cross at ≥ 0.3 mm. At the conclusion of each run, participants viewed a Head Motion Report (Figure 2) which displayed their performance on a gauge of 0–100 (a percentage score of 0% to 100%), and a graph of their motion level over time to help visualize total movement during the session. Participants were encouraged to bring their score closer to 100% on subsequent sequences.

Figure 2.

Head motion report provided to participants following each run of the experiment. The upper right quadrant indicates how many minutes of data with acceptable motion has been collected (under the logic that in a typical resting state paradigm, additional data might be collected to meet a threshold for amount of good data) (Dosenbach et al., 2017).

MRI acquisition and analysis

MRI data were acquired using a Siemens Prisma scanner (Siemens Medical Systems) at 3 T equipped with a 32-channel head coil. Scan sequences began with a T1-weighted structural volume using an MPRAGE sequence [repetition time (TR) = 2.4 s, echo time (TE) = 2.2 ms, flip angle = 8°, 300 × 320 matrix, voxel size = 0.8 mm isotropic]. Blood oxygenation level-dependent (BOLD) functional MRI images were acquired using a multiband echo planar imaging sequence (Feinberg et al., 2010) [TR = 3.07 s, TA = 0.770 s, TE = 37 ms, flip angle = 90º, voxel size = 2 mm isotropic, multiband factor = 8]. We used a sparse imaging design in which there was a 2.3 second delay between scanning acquisitions and the TR was longer than the acquisition time to allow for minimal scanning noise during stimulus presentation and audio recording of participant responses (Edmister et al., 1999; Hall et al., 1999). Due to slight variations in scanning sessions, participants had 763–846 frames of data (median = 794).

Analysis of the MRI data was performed using Automatic Analysis version 5.8.1 (Cusack et al., 2015) (RRID:SCR_003560) which scripted analyses run in SPM12 (Wellcome Trust Centre for Human Neuroimaging) version 7487 (RRID:SCR_007037). Functional images underwent rigid body realignment (i.e, “motion correction”). We used framewise displacement (FD) to parsimoniously summarize frame-by-frame motion, calculated as the sum of the six head motion estimates obtained from realignment, with a dimensional conversion of the three rotations assuming the head is a 50 mm sphere (Power et al., 2012). We supplemented FD values with differential variance (DVARS), a measure of variations in image intensity. DVARS was calculated as the root-mean-squared of the time difference in the BOLD signal calculated across the entire brain, before realignment (Smyser et al., 2011). Although FD and DVARS are highly correlated, they are not identical (Jones et al., 2022), and we anticipated that these two metrics might differ in sensitivity to motion feedback.

Results

Movement parameters for young and older adults as a function of feedback are shown in Figure 3. Figure 3a shows FD values for two subjects (our first young adult without feedback, and our first young adult with feedback), as well as the mean FD for all subjects in each feedback group. Although we used all time points in the statistical analysis, for illustrative purposes we also plotted summary values in Figure 3b (each point denoting mean FD per subject).

Figure 3.

Comparison of average framewise displacement (FD) values for feedback and no-feedback groups. a) Top: FD values for two example subjects, one with feedback and one without. Bottom: Mean FD values for all participants in the feedback or no-feedback group. b) Summary of mean FD values as a function of age and feedback. Individual subjects are dots, mean ± SE shown in bars.

We analyzed these data using R version 4.2 (R Core Team, 2020) (RRID:SCR_001905). We conducted linear mixed effects analysis using the nlme package (Pinheiro and Bates, 2000). We accounted for temporal autocorrelation (that is, motion at one frame is related to motion during the following frame) using a first-order autoregressive model. The model specification was:

m2 <- lme(FD ∼ 1 + frame + age_group * feedback, random = ∼ frame | subject_number, data = df, method = “ML”, correlation = corAR1(value = autocorrelation_estimate, form = ∼ frame | subject_number, fixed = FALSE))

where df is the data frame, FD is framewise displacement, and feedback is a dichotomous variable denoting whether feedback was provided or not (reference: no feedback). We calculated the autocorrelation_estimate from the data (value of 0.52). Model results are shown in Table 2.

Table 2.

Fixed effects results for motion (FD) model by frame

Although we first included time (frame) in the model, in fMRI research it is often convenient to think about continuous scanning runs (sometimes also referred to as “sessions”). In the current study, each participant had 6 runs of data. We thus ran an additional model to see whether the effect of feedback was comparable across run. Because time was expressed as 6 runs, we did not account for temporal autocorrelation in this model:

m3 <- lme(FD ∼ 1 + run + age_group * feedback, random = ∼ run | subject_number, data = df, method = “ML”, correlation = NULL)

Results of this analysis by run are shown in Table 3, and FD as a function of run are shown in Figure 4. We again found a significant effect of feedback, but the effect of feedback did not significantly differ by run (as one might expect if there were learning effects of the motion feedback).

Table 3.

Fixed effects results for motion (FD) model by run

Figure 4.

Mean FD values for all participants in the feedback or no-feedback group as a function of run. Individual subjects are dots, mean ± SE shown in bars.

Although our analysis was focused on changes in mean FD, we wanted to look in more detail at how feedback affected the distribution of movement. We thus plotted the density of FD values for all scans as a function of feedback (Figure 5) for FD values up to 2.0. We used density rather than the count to control for the different numbers of frames in the two groups (resulting from different numbers of subjects). Although qualitative, this analysis highlights that participants receiving feedback showed fewer high-FD scans than those receiving lower feedback, with this effect becoming more apparent at higher FD values—for FD values above 1.5, the feedback group had less than a third of these scans than the group without feedback.

Figure 5.

a) Density plot of all FD values as a function of feedback. b) Density of FD values in the feedback condition relative to the no feedback condition. A value of 1 (dashed line) indicates equal density of FD values in the two conditions. The plot illustrates a decrease in high-FD values in the feedback condition, and an increase in low FD values, relative to the no feedback condition.

We next repeated our primary FD analyses, but using DVARS (rather than FD) as a dependent measure, under the logic that FD and DVARS may be differently sensitive to effects of motion, and one or the other may be of more interest in any specific study. Summary results are shown in Figure 6 and Table 4. Although DVARS was numerically lower for the feedback group than the no-feedback group, the difference was not statistically significant. Results as a function of run are shown in Figure 7 and Table 5. Here, the feedback-based reduction in DVARS was statistically significant (p < .05).

Table 4.

Fixed effects results for DVARS model by frame

Table 5.

Fixed effects results for DVARS model by run

Figure 6.

Comparison of average DVARS values for feedback and no-feedback groups. a) Top: DVARS values for two example subjects, one with feedback and one without. Bottom: Mean DVARS values for all participants in the feedback or no-feedback group. b) Summary of mean DVARS values as a function of age and feedback. Individual subjects are dots, mean ± SE shown in bars.

Figure 7.

Mean DVARS values for all participants in the feedback or no-feedback group as a function of run. Individual subjects are dots, mean ± SE shown in bars.

In addition to framewise measurements of FD and DVARS, we investigated whether motion feedback affected the temporal signal-to-noise ratio (tSNR) of the data. We first created a brain mask using each subject’s tissue class segmentation, binarizing at a gray matter probability of 0.8. We then calculated tSNR (the mean of the signal divided by its standard deviation, over frames) in every voxel within the brain mask. We then performed two complementary analyses. First, for each run, we took the mean tSNR over voxels, and used the same statistical model as for FD and DVARS, shown in Figure 8 and shown in Table 6. Although the tSNR values were numerically higher in the feedback group, there was no significant effect of feedback (p = 0.0814).

Figure 8.

Comparison of average temporal SNR (tSNR) values for feedback and no-feedback groups. Each participant is represented by a single point per run, averaging the tSNR over all voxels in the gray matter mask.

Table 6.

Fixed effects results for tSNR model

Finally, we assessed the degree to which feedback may have affected performance on the behavioral task. Accuracy on the word repetition task is shown in Figure 9 as a function of age group and feedback. We used a linear model to test for the impact of feedback on behavioral accuracy (averaging over experimental runs). We found no significant effect of age group, no significant effect of feedback, and no significant age x feedback interaction, although in older adults there was a small numerical decrease in accuracy during the feedback condition.

Figure 9.

Behavioral accuracy for young and older adults as a function of feedback.

Discussion

Given the pernicious and sometimes unpredictable effects of head movement in fMRI studies, reducing motion in the first place is on its face an appealing strategy. In the current study we evaluated the degree to which a combination of real-time and summary motion feedback could reduce head movement in adult participants by pseudorandomly assigning participants to feedback or no feedback groups. We found that motion feedback was associated with a statistically significant reduction in average FD of approximately 0.06, with consistent (though not significant) changes in DVARS and tSNR. We discuss these findings and their implications below.

First, our primary finding that providing motion feedback can significantly reduce head motion in participants undergoing task-based MRI is notable. These findings add motion feedback to the limited set of tools MRI researchers have for reducing head motion during acquisition. Although numerous strategies exist for accounting for head motion in preprocessing and analysis, none are perfect, and it is difficult to ascertain the most appropriate approach (Ardekani et al., 2001; Oakes et al., 2005; Johnstone et al., 2006; Jones et al., 2022). Reducing motion during acquisition may therefore be a preferable alternative in many circumstances.

It is important to highlight the magnitude of the effect we observed. Compared to the participants with no motion feedback, participants with feedback showed a reduction in FD of approximately .06. Although this may be helpful in reducing motion-related artifacts in data processing, it is not a large effect (though in theory it might be increased if different types of feedback were used, or in the context of a different task). It is also important to remember that we tested healthy adults aged 19–81. It could be that the effect of motion feedback would be larger (or smaller) in different populations.

That being said, the principal effect of feedback may not be the average reduction in FD observed, but rather a reduction in the number of large motion events. Rapid head motion tends to have a larger effect on the BOLD signal than slower motion. Thus, reducing the number of large, quick movements may improve signal quality to a greater extent than a simple reduction in mean FD. Indeed, we saw the biggest proportional difference between feedback groups at higher FD values. Interestingly, the reduction in high motion events (Figure 5b) roughly corresponds to an FD of 0.3, the value at which participants received “red” feedback.

An important consideration when implementing real-time feedback during fMRI is whether doing so significantly changes task demands. Although we found accuracy was statistically comparable across feedback groups, we cannot rule out other changes in task demands. Indeed, we observed a numeric (though not significant) decrease in task accuracy in older adults when they were presented with real-time feedback relative to the no-feedback group. Broadly speaking, we might expect monitoring the display for feedback, and adjusting one’s movement in response, to engage systems related to cognitive control (sometimes also referred to as attentional control or executive function) (see also McCabe et al., 2010). In one framing of cognitive control, cognitive control is proposed to operate in at least two complementary modes: a proactive mode that is concerned with maintaining goal-directed behavior, and a reactive mode that is engaged when goal-directed performance falters (Braver, 2012). There are two implications of such a framework. First, participants who are less able to engage cognitive control may benefit less from motion feedback compared to those who are better able to engage control systems. Second, researcher concerns about how task demands may affect imaging results of interest may depend on the nature of the main fMRI task. For example, tasks tapping cognitive control or executive function may be more impacted by motion feedback than those focused on primary sensory or motor systems. In our case, we used a speech-in-noise task, one that is hypothesized to engage cognitive control networks (Peelle, 2018; McLaughlin et al., 2021).

In our implementation, participants received brief instructions before entering the scanner, and once in the scanner the display was explained to them. One interesting future direction would be to provide participants with motion training over a longer period to see whether they might learn to better limit their motion in the absence of feedback (for example, by training in a mock scanner). Such an approach would take more time, but would also circumvent some of the challenges associated with real-time feedback (such as introducing a “dual-task” situation).

Although we found a significant effect of motion feedback on FD, we did not observe a significant effect on motion feedback on DVARS (although DVARS values were numerically lower in the feedback group). FD and DVARS are typically strongly correlated (Jones et al., 2022), but not identical, and thus the divergence in findings is not entirely unexpected. The numerical (albeit not significant) reductions in DVARS in participants receiving motion feedback is consistent with the effect we saw for FD.

In addition, we found that feedback led to increased whole-brain tSNR, which we used as a proxy of overall image quality. Despite the tSNR being numerically larger in the group receiving motion feedback, the difference was not statistically significant. However, it is still possible that small improvements in tSNR would result in appreciable improvements in the accuracy or reliability of statistical models.

In conclusion, we found that motion feedback as implemented in FIRMM significantly reduced the amount of head motion we observed during task-based fMRI. Real-time feedback may thus be well-suited to complement other approaches to motion reduction depending on the specific needs of a given study.

Acknowledgments

Work reported here was supported by grant R01 DC014281 from the US National Institutes of Health. Support was also provided by the Basque Government through the BERC 2022-2025 program and by the Spanish State Research Agency through BCBL Severo Ochoa excellence accreditation CEX2020-001010-S. The multiband echo planar imaging sequence was provided by the University of Minnesota Center for Magnetic Resonance Research. We are grateful to Linda Hood for assistance with data collection, and to Nico Dosenbach and Jackie Hampton for assistance implementing FIRMM.

Footnotes

• Data availability statement: The data that support the findings of this study are openly available in OpenNeuro at https://doi.org/10.18112/openneuro.ds004285.v1.0.0, reference number ds004285.

• IRB statement: The study reported here was conducted according to a protocol approved by the Washington University in Saint Louis Institutional Review Board.

• Minor update to correct an error in the text of the results section.

• https://doi.org/10.18112/openneuro.ds004285.v1.0.0

• ¹ Now a commercial product offered by Turing Medical (St. Louis, MO).