Personalized Closed-Loop Brain Stimulation for Effective Neurointervention Across Participants

Accumulating evidence from human-based research has highlighted that the prevalent one-size-fits-all approach for neural and behavioral interventions is inefficient. This approach can benefit one individual, but be ineffective or even detrimental for another. Studying the efficacy of the large range of different parameters for different individuals is costly, time-consuming and requires a large sample size that makes such research impractical and hinders effective interventions. Here an active machine learning technique is presented across participants—personalized Bayesian optimization (pBO)—that searches available parameter combinations to optimize an intervention as a function of an individual’s ability. This novel technique was utilized to identify transcranial alternating current stimulation frequency and current strength combinations most likely to improve arithmetic performance, based on a subject’s baseline arithmetic abilities. The pBO was performed across all subjects tested, building a model of subject performance, capable of recommending parameters for future subjects based on their baseline arithmetic ability. pBO successfully searches, learns, and recommends parameters for an effective neurointervention as supported by behavioral, stimulation, and neural data. The application of pBO in human-based research opens up new avenues for personalized and more effective interventions, as well as discoveries of protocols for treatment and translation to other clinical and non-clinical domains.


Introduction
There is no doubt that the human organism is complex, and the impact of nature and (4) This cycle continues until either a new subject is tested, in which case a different value for the personalized variable will be recorded; a pre-set stopping criterion is reached, such as the number of subjects to be tested; or until the potential improvement is considered negligible (convergence). In this study, we utilized a pre-set stopping criterion of 50 subjects, after which testing was ceased.
7 Figure 2 | An overview of the experimental paradigm. a) An overview of the behavioral paradigm. Subjects (n = 50) watched a fixation point that indicated the start of a trial. After 3000 ms an arithmetic multiplication was shown with two possible answer options on the left and right side with a difference of 10. Subjects responded by pressing either the left or right button on a response box. Lastly, subjects received either 'correct' or 'incorrect' as feedback to continuously capture attention. b) Subjects first completed a baseline rs-EEG of four minutes, after which 10 practice trials of multi-digit times single-digit multiplications were presented. This was followed by the baseline task, which comprised five blocks of 10 different multiplications. Three different tACS frequency-current combinations were proposed by the BO algorithm after the completion of the multiplications. Between these tACS combinations, post-block rs-EEGs were recorded before the subjects moved on to the next tACS combination. Validation of the blinding of the stimulation and perceived sensations were assessed after completion of a stimulation block. c) An illustration of the tACS electrode montage. Stimulation was applied over the left frontoparietal area (F3 and P3) with one return electrode (Cz).

155
applying combinations of tACS parameters to link behavioral changes to 156 electrophysiological (EEG) outcomes, while we report this finding, we note that it is not A more in-depth visualization of the efficacy of the pBO procedure revealed that 196 the overall fluctuation in performance improvement (e.g., normalized to the baseline 197 performance without stimulation) across subjects with low and high baseline abilities was 198 similar (Figure 4a). This result indicates that the success of our approach is equally 199 effective for people with either low or high arithmetic baseline ability. The optimal 200 frequency-current tACS parameter combinations proposed by the pBO algorithm 201 confirms a shift from higher frequencies and currents in low-baseline ability subjects to 9 Figure 4 | Results of optimizing behavior with personalized Bayesian optimization (pBO) (n = 49). a) Indepth visualization of the normalized performance according to baseline ability during pBO. Subjects on the lower part of the baseline ability spectrum showed a similar arithmetic performance improvement during tACS compared to subjects on the higher baseline ability spectrum. Note that a normalized performance score of 1 indicates no difference with baseline arithmetic performance when no stimulation was applied. A normalized performance score higher than 1 indicates improved performance as measured with drift rate. The blue shaded area indicates 95% credibility intervals. b) The change in frequency-amplitude tACS parameters proposed by the pBO algorithm based on the individualized baseline ability in arithmetic at the end of optimization. c) Predicted best performance at each iteration (i.e., different blocks), calculated as the best performance predicted by the GP at any parameter combination. Three subsequent iterations were assessed for each participant. Surrogate uncertainty is shown by the shaded area in pink. Note that during some iterations uncertainty is higher due to new baseline abilities introduced in the pBO and due to outliers. These outliers are retested later which then reduces uncertainty.  Thus, if the behavioral evaluations of the experimental procedure are too noisy, the 245 pBO procedure's ability to make correct judgements about the optimum parameters is 246 diminished but it is still able to outperform random sampling. Critically, as Figure 5 247 illustrates within these estimated noise variance ranges our pBO leads to improved 248 optimization compared with the standard BO approach that does not take baseline ability 249 into account. In particular, the standard BO is unable to enhance performance, thus 250 highlighting the benefit of personalization vs. the one-size-fits-all approach. The simulation was run 30 times on each of the six different levels of noise, lines represent the mean performance and shaded areas the standard deviation of 30 repeats. a) Shows the best found value identified by each algorithm at each iteration, demonstrating that the pBO algorithm is able to find higher values more quickly than the BO and random search algorithms. b) Shows the Euclidean distance of the identified optima from the true optima of the Hartmann function (i.e., accuracy of the algorithm). The pBO algorithm is shown to be more accurate than the BO and random search algorithms, except at very high levels of noise, where they are comparable.  such as behavior or brain function, but use a one-size-fits-all approach that leads to  For example, a recent multiplication study indicated possible behavioral constraints of a one-size-fits-all brain stimulation protocol on performers in a certain subgroup of the 308 population (e.g., high performers) (47). Notably, the present results cannot be explained    (Figure 4). Secondly, our study shows that non-personalized interventions, 336 as in standard BO, are ineffective due to the inability to optimize performance effectively ( Figure 5). Our simulations further show that pBO is reliable even when there is a 338 considerable amount of noise present in the models. Previously, most BO applications 339 have been in a noise-free context, in contrast with human-based studies that are prone to 340 noisy evaluations. More precisely, as noise increases, the pBO algorithm is less able to 341 evaluate the stimulation parameters correctly, but still outperforms random sampling and 342 the standard BO algorithms. This furthermore applies to the estimated noise variance 343 ranges that were observed from our data.  One constraint of this approach that should be considered, is that the distribution of 361 subject baseline abilities in this study was weighted towards the lower end of the ability 362 range, leading to fewer subjects with higher baseline ability being tested (Figure S1

414
Over the course of the experiment, subjects completed four blocks of fifty 415 multiplication problems -one baseline block and three stimulation blocks (see Figure 2). 416 After recording an initial 4-minute baseline resting state EEG, the task was explained to 417 the subjects and they completed 10 practice trials, followed by the baseline block during Supporting Information for the full questionnaire). We used this data to assess the 513 relationship between the intensity rating of every sensation and tACS amplitude.

Resting State-EEG Recordings and Pre-processing
Resting state-EEG recordings were made at the start of the study (before baseline 516 measurements) and immediately after every stimulation block. Electrophysiological data 517 were obtained with eight gel Ag/AgCl electrodes (F3, P3, F4, P4, Fz, Cz, Pz, AF8) 518 according to the international 10/10 EEG system using the wireless Starstim R32 sensor Hz. Visual inspection was carried out to remove artefacts caused by muscle movement. 527 We rejected an EEG recording from analysis if more than 25 percent of the data in a 528 given block were removed. This resulted in four rejected datasets (2% of the data).    value. We needed to measure this value separately for every subject, as was done 560 during the baseline task. We expected to see that the optimal parameters will vary with 561 different baseline abilities. That is, the optimal parameter * depends on the different 562 values of . For this reason, the standard BO presented in the previous section may not 563 have been appropriate. Therefore, we proposed to solve the following optimization 564 problem, defined formally as: where is the baselines ability given for each subject. The optimal parameter * is not 567 defined globally, but specifically to a variable . This is the key difference of our pBO in

571
In the present tACS study, we used the following objective function ( , ):      steps of 1 Hz and the beta wPLI was calculated from 14-30 Hz in steps of 4 Hz.

704
Furthermore, we normalized wPLI by calculating the wPLI at the applied tACS 705 frequency divided by the baseline wPLI at the same frequency.

706
First, outliers were removed with Cook's distance before running statistical models.

707
To focus on the relation between arithmetic baseline ability and spectral power, separate 708 regression models were run with theta and beta power as dependent factors. Likewise, we 709 tested whether there was a relation between frontoparietal theta and beta connectivity      As stated in our pre-registration, we also investigated EEG changes induced by tACS during arithmetic performance (see Tables S1 and S2 in Supporting Information). tACS on performance levels between subjects with low and high baseline ability. Neural 1115 entrainment due to tACS could possibly serve as a compensatory mechanism to improve 1116 performance for subjects with low baseline ability (50).   Figure S5 | The interaction between EEG power, current, and frequency in predicting arithmetic performance for subjects with low arithmetic baseline ability (n = 25). Arithmetic performance (log transformed drift rates) during stimulation is shown on the y-axis and the normalized (post stimulation/pre stimulation) EEG power μV 2 /Hz (log transformed) based on the applied tACS frequency after stimulation is shown on the x-axis for four different tACS frequencies (4 Hz, 15 Hz, 30 Hz, and 50 Hz). Current intensity is indicated by the blue line (0.1 mA), the black line (1 mA), and the grey line (1.6 mA). Shaded areas indicate 95% confidence intervals. Note that different tACS categories and current intensities are presented for visualization purposes, to allow a better grasp of an interaction that is based on continuous variables.