ABSTRACT
Anticipating meaningful actions in the environment is an essential function of the brain. Such predictive mechanisms originate from the motor system and allow for inferring actions from environmental affordances, the potential to act within a specific environment. Using architecture, we provide a unique perspective to the abiding debate in cognitive neuroscience and philosophy on whether cognition depends on movement or is decoupled from our physical structure. To investigate cognitive processes associated with architectural affordances, we used a Mobile Brain/Body Imaging approach recording brain activity synchronized to head-mounted virtual reality. Participants perceived and acted upon virtual transitions ranging from non-passable to easily passable. We demonstrate that early sensory brain activity, upon revealing the environment and before actual movement, differed as a function of affordances. Additionally, movement through transitions was preceded by a motor-related negative component also depended on affordances. Our results suggest that potential actions afforded by an environment influence perception.
SIGNIFICANCE STATEMENT By using electroencephalography and virtual reality, our research provide a unique perspective to the centurial open-ended debate in cognitive neuroscience and philosophy on the relation between cognition, movement and environment. Our results indicate that cortical potentials vary as a function of bodily affordances reflected by the physical environment. Firstly, the results of this study implies that cognition is inherently related to potential movement of the body, thus we advance that action is interrelated with perception, actively influencing the perceivable environment. Secondly, as cortical potentials are influenced by the potential to move, which in turn is the task of architectural design, architects holds largely a privilege of human health, and thus potentially capable of provoking and preventing physiological conditions.
INTRODUCTION
The affordance of a given spatial environment, defined as the perception of possibilit ies for, or restraints on, action that the environment offers, is essential for an agent to produce meaningful behavior. Thus, the affordances of the spatial environment becomes a central concept for humans interacting with their world. The term affordances was first introduced by Gibson (1), and later specified by various authors including Clark who defines affordance as “[…] the possibilitiesfor use, interventionandactionwhich the physical world offers a given agent and are determined by the ‘fit’ between the agent’s physical structure, capacities and skills and the action-related properties of the environment itself.” (2). In light of emerging theories of embodied cognition, the perception of the environment may be dependent on proprioceptive mechanisms. According to predictive processing, a neuroscientifically basedtheory of embodied cognition (3–5), motor systems, similar to perceptual processes, aim at cancelling out continuously incoming bottom-up sensory signals with top-down predictions. In this perspective, movement emerges as a result of an active inference that attempts to either minimizing motor trajectory prediction errors by acting, and thus perceiving the unfolding of the predicted movement, or by changing perception itself (6–8). From the standpoint of active inference, motor systems suppress errors through a dynamic interchange of prediction and action. In other words, there are two ways to minimizing prediction errors; one is to adjust predictions to fit the current sensory input, while another is to adapt the unfolding of movement to make predictions come true. It is a unifying perspective on perception and action suggesting that action is both perceived and caused by perception (9). Hence, action, perception, and cognition coordinate to move the body in ways that conform a transitional set of expectations (10). The claim we seekto investigate in the present study is that perception is rooted in action, creating an action-perception loop, informed by dynamically (top-down/bottom-up) generated prediction errors. Ultimately, the argument is that perception is not the sole result of sensing the physical world, but unfolds as an ongoing interaction between sensory processes and bodily actions. Such a claim has philosophical and neuroscientific significance as the neural dynamics underlying perception would be intimately dependent on the affordances of a given environment.
To further investigate this claim, we used electroencephalographic (EEG) recordings to address the neural dynamics of action-perception interactions through affordance manipulations in architectural experiences. More specifically, we investigated the affordances of transitions as they form an ideal candidate due to their dynamic nature concerning the duration of altering one condition to another (11). We here confine transitions to the passage between spaces which, according to the enactivists’ proposed action-perception loop, will be experience dependent on the affordances offered by the passage itself. From an architecturally historical point of view, the use of transitions have evidently been exploited at least since eleventh-thirteenth dynasties (e.g., Fazio et al., 2008, chaps. 1, 2, 5). Written interest in human experience of architectural settings has been established at least for the last two millennia (e.g., Norberg-Schulz, 1965; Palladio, 1997; Pallasmaa, 2011; Rasmussen, 1959; Vitruvius and Morgan, 1960). Despite transitions being ubiquitous in architecture, the underlying mechanisms of how transitions affect human perceivers appears to have taken an implicit, overlooked, and close to nonexistent position in architecturaldiscourse, with few exceptions (15, 18–20). Due to the dynamic nature of architecture, an essential part of transitions and experiencing architecture is that of being able to act (21). Traditionally, investigations of architectural experiences are phenomenological - the description of phenomena in how experience gives access to a world of space and time (14, 22–24). Such descriptions find specifically movement of the individual to be an expression of a holistic experience of architecture (14, 22), linking the nature of movement to architectural experiences (25). Transitions in architecture depend on voluntary movement and thus a prerequisite for any transit is a goal, which in turn calls for action planning. Coarsely three parameters compose a transition: a motivated goal, a change in physical environment and the unfolding of action. All three parameters are interdependent, as reaching a goal depends on the affordance offered by an environment, and also propels the body in space contributing to experience. Architectural transitions thus include the attenuation of an agent’s experience through movements and how such movements animate the body through environmental changes.
Data from neuroscientific experiments addressing this issue might contribute to discussions centered on philosophical questions on how we relate to the world. For long, enactivists have implicated the reciprocal dependency of the living organism, as a self-organized living system, and the embedded body in a world for cognition (26–28). Enactivism is rooted in phenomenology (21, 29), similar to prominent architectural theorists, who put body, action, and cognition central to experience. Active inference closely relates to enactivism, in the sense that we act to perceive, and vice versa. Such a thesis rests on a hierarchical and dynamic model of the world, which temporally dissociates lower sensorimotor inferences from higher motivated goals, as fast and slow, respectively (30). Fast, lower sensorimotor inferences depict processes of affordances, which thereby must be present in early stages of perception. Hierarchical affordance competition (HAC; Pezzulo and Cisek, 2016) takes the temporal aspect of affordances much further, by suggesting that cortical activity relates to the immediate decision of action selection, which occurs fluently during movement. Such an account of temporally extended affordance is in accordance with active inferences.
To investigate the impact of environmental affordances on early sensory processing in actively transiting humans, we used a Mobile Brain/Body Imaging approach (32–34) recording brain activity with EEG synchronized to movement recordings and head mounted virtual reality (VR). This approach allows for investigating brain dynamics of participants perceiving an environment and the transitions contained therein as well as brain dynamics during the transitions itself. Previous studies investigating event-related potential (ERP) activity in stationary participants demonstrated slow cortical potentials to indicate anticipative motor behavior (for an overview, see Luck and Kappenman, 2011, chap. 8). Known motor-related cortical components (MRCPs) are the readiness potential (RP; Kornhuber and Deecke, 2016), contingent negative variation (CNV), and the stimulus-preceding negativity (SPN; Brunia, 2003), which can be seen as indicators of predictive behavior (38). MRCPs are negative going waveforms preceding an actual, or imagined, motor execution. However, these negative components are associated with multiple processes including sensory, cognitive, and motor systems. In a study by Bozzacchi et al. (39), the authors attempted to measure affordances of a physical object by evaluating whether the anticipated consequence of action itself influence the brain activity preceding a self-paced action. The authors compared MRCPs of situations where it was possible to reach out and grasp a cup, versus situations where it was impossible to grasp the cup, by tying the hands of the participants. A motor execution was forced at all times. In situations where it was impossible to grasp the cup, the authors reported an absence of early activity over the parietal cortex, and found instead increased activity over the prefrontal cortex. The results were interpreted as reflecting an awareness of the inability to execute a goal-oriented action. Closely related to the MRCPs is the post-imperative negative variation (PINV),a negative going waveform that is present succeeding an imperative stimulus. It reflects the immediate motor execution related to the onset of an imperative stimulus and was observed during experiments investigating learned helplessness or loss of control (40, 41). The PINV thus allows linking of motor relatedpotentials to anticipation of affective states (42).
If an enactive account of perception, action and cognition is correct, affordances intimately relate to higher hierarchical levels through low-level perceptualcues. Such an account would situate processing of affordances at a similar stage as early perceptual processes and should reveal differences in sensory and motor-related ERPs associated with the perceived affordance of an environment. To investigate whether brain activity is altered depending on affordances offered by the environment, we presented human observers with environmental stimuli that allowed or prohibited a transition from one room to the next. To this end, participants were presented with a view into a room containing one door of different widths, allowing or prohibiting a transition into the next room and thus providing different affordances. We expected to find differences in cortical responses to co-vary as a function of affordances over sensory and motor areas. In addition, we expected differences in motor-related cortical potentials as a function of the environmental affordances when participants were instructed to walk through the door or to remain in the same room.
METHODS
Participants
20 participants (9 female) without history of neurological pathologies were recruited from a participant pool of the Technical University of Berlin, Berlin. All participants read and signed a written informed consent about the experimental protocol, which was approved by the local ethics committee. Participants received either monetary compensation (10€/hour) or accredited course hours. The mean age was 28.1 years (σ = 6.2), all participants had normal or corrected to normal vision and none had a specific background in architecture (no architects or architectural students). One participant was excluded due to technical issues of the experimental setup.
Paradigm description
The experiment took place in the Berlin Mobile Brain/Body Imaging Laboratories (BeMoBIL) with one of the experimental rooms providing a space of 160 m2. The size of the virtual space was 9 × 5 meters with a room size of 4.5 × 5 meters for the first room and a room size of 4.5 × 5 meters for the second room. Participants performed a forewarned (S1-S2) Go/NoGo paradigm (pseudo-randomized 50/50) in the virtual reality environment that required them to walk from one room to a second room. Doors of different width ranging from unpassable (20 cm, Narrow) to passable (100 cm, Mid) to easily passible (1500 cm, Wide) manipulated the transition affordance between rooms. The experiment consisted of a 3 × 2 repeated measures design including the factors door width (Narrow, Mid, Wide; pseudo-randomized) and movement instruction (Go, NoGo). A total of 240 trials per participant was collected with 40 trials for each of the factor levels. One trial consisted of a participant starting in a dark environment on a predefined starting square (see Figure 1). The “lights” would go on after a random inter-trial-interval (mean = 3 s, σ = 1 s), and participants faced a room with a closed door. They were instructed to wait (mean = 6 s, σ = 1 s) for a color change of the door with a change to green indicating a Go trial and a change to red indicating a NoGo trial. In case of a green door, the participant walked towards the door, which would slide aside. Upon entering the subsequent space, participants were instructed to find and virtually touch a red rotating circle by using the controller. The circle would inform the participant to have earned another 0.1€ to their basic reimbursement of 10 Euro per hour. After each trial, participants had to give an emotional rating for the environment irrespective of whether they transitioned through the door (Go condition) or whether they remained in the same room (NoGo condition) without transition. To this end, participants were instructed to go back to the starting square, and fill in a virtual Self-Assessment Manikin (SAM) questionnaire, using a laser pointer from the controller, and to subsequently pulling the response button located at the pointer finger to turn the “lights off”. The lights would go back on automatically to start the next trial.
In Go-trials, participants were instructed to walk towards the door and into the second room even in case the door was too narrow to pass. This was done to control for motor execution in the Go-condition and to allow movement towards the goal irrespective of the affordance (passable vs. unpassable). Upon touching the surrounding walls, the walls would turn red and inform the participants they have failed to pass, and thus must return to the start square, fill in the virtual SAM and start the next trial by pulling the trigger. Participants would quickly notice that the narrow door (20 cm) was impossible to pass without producing the warning feedback that they have failed to pass. All participants had a training phase to get accustomed to the VR environment and the different conditions. The experimenter observed the participants from a control room, separated from the experimental space, using two cameras and a mirrored display of the virtual environment to reduce interactions to a minimum during the experiments.
Subjective and Behavioral data
To investigate the subjective experience of the transitions, we introduced the participants with a virtual Self-Assessment Manikin (SAM) questionnaire after each trial. The SAM is a pictorial assessment of pleasure, arousal and dominance on a 5-point Likert scale (43). The manikin display ranges from smiling to frowning (pleasure), from a dot in the stomach to an explosion (arousal) and from being very small to very big (dominance). Participants were asked to self-assess their current state after each trial. Furthermore, we measured the reaction time from the onset of the Go-stimulus (door color change) to reaching the opening-threshold itself, to assess the behavior. The data was analyzed using ANOVA with the width of the doors as repeated measures factor. In case of violation of normality and homogeneity, corrected p-values are reported. For post-hoc analysis, the data was contrasted using Tukey HSD.
EEG Recording and data analysis
To investigate the impact of transitional affordances on human cognition and brain dynamics, we used a MoBI approach (32–34, 44) recording human brain dynamics in participants actively transitioning through virtual rooms. All data streams were recorded and synchronized using LabStreamingLayer (LSL; Kothe, 2014). Participants wore a backpack, which held a high-performance gaming computer to render the VR environment (Zotac, PC Partner Limited, Hong Kong, China) attached to two batteries and an EEG amplifier system. We combined a Windows Mixed Reality (WMR; 2.89”, 2880 × 1440 resolution, update rate at 90 Hz, 100 degree field of view with a weight of 440 grams, linked to the Zotac computer through HDMI) headset and one controller by ACER to display and interact with the virtual environment based on Unity (see Figure 2). Events for recordings of performance and physiological data were triggered by the position of the participant in the tracking space or by the respective response buttons of the remote control. Specific events, such as touching the wall, all button presses, transitioning through the door, answering the questionnaire and all cases of “lights on” (and off), were synchronized with the recorded brain activity and the presented VR environment through LSL.
EEG data was acquired continuously with a 64 channels EEG system (eegoSports, ANT Neuro, Enschede, Netherlands), sampled with 500 Hz. Impedances were kept below 10 kOhm. The computational delay generated by the interaction of ANT Neuro software, Windows Mixed Reality and Unity was measured to be 20 ms (σ = 4), which was taken into account during the analysis by subtracting the average delay from each event latency. With a jitter of 4 ms, we considered the delay to have little to no impact on the ERPs. Offline analysis were conducted using MATLAB (MathWorks, Natick, MA, USA) and the EEGLAB toolbox (46). The raw data were band-pass filtered between 1 Hz and 100 Hz and down-sampled to 250 Hz. Channels with more than five standard deviations from the joint probability of the recorded electrodes were removed and subsequently interpolated. The datasets were then re-referenced to an average reference and adaptive mixture independent component analysis (AMICA; Palmer et al., 2011) was computed on the remaining rank of the data using one model with online artifact rejection in five iterations. The resultant ICA spheres and weights matrices were transferredtothe raw datasetthat was preprocessed using the identical preprocessing parameters like the ICA dataset, except the filtering, which used a band-pass filter from 0.2 Hz to 40 Hz. Subsequently, independent components (ICs) reflecting eye movements (blinks and horizontal movements) were removed manually based on their topography, their spectrum, and their temporal characteristics.
Epochs were created time-locked to the onset of the room including the closed door (“Lights on”) from −500 ms before to 1500 ms after stimulus onset for Narrow, Mid and Wide door trials. Similarly, another set of epochs were time-locked to the second stimulus Go/NoGo from −500 ms before to 1000 ms after onset of the stimulus for Narrow, Mid and Wide door trials. On average, 15% (σ = 10.8) of all epochs were automatically rejected when they deviated more than five standard deviations from the joint probability and distribution of the activity of all recorded electrodes.
The visual-evoked potentials as well as MRCPs were analyzed at central midline electrodes (Fz, FCz, Cz, Pz, POz and Oz) covering all relevant locations including the visual and the motor cortex as reported in previous studies (39, 48). As stimuli were distributed across the complete visual field and participants walked through the virtual spaces, we did not expect any lateralization of ERPs. All channels were analyzed, however only three channels (FCz, Pz and Oz) are reported and discussed in-text according to reported results by Bozzacchi etal. (39). The analysis results of all six channels canbe found in the supplementary material. For peak analysis of the P1-N1 complex, the grand average peaks were estimated and individual peaks were defined as the maximum positive and negative peak in the time window surrounding the grand average P1 and N1 peak (+/− 10 ms from peak), respectively. An automatic peak detection algorithm detected the peaks in the averaged epochs for each participant. Multiple peaks were detected and systematically weighed depending on the magnitude, the distance to the grand-average peak latency that was determined by visual inspection of grand average ERP, and the polarity (please see algorithm in the supplementary material). For anterior N1 and posterior P1, by visual inspection of the grand average ERPs, the grand-average latency was estimated to be 140 ms with a search window for individual peaks ranging from 50 - 200 ms. For the anterior P1 and posterior N1 the grand-average peak latency was estimated to 215 ms with a searchwindow for individual peaks ranging from 140 - 290 ms.
Mean peak amplitudes were analyzed using a 3 × 3 repeated measures ANOVA using the door width (Narrow, Mid, Wide) and electrode as repeated measures. The results descriptions focus on the visual evoked P1 component at posterior electrodes (Pz, POz and Oz) and the N1 component at frontal leads (Fz, FCz and Cz) basedon separate ANOVAs. For the N2 and P2 component atposterior electrodes (Pz, POz and Oz) and frontal leads (Fz, FCz and Cz), separate ANOVAs were computed in the time-range of 140 - 290 ms. For the later motor related potentials, an ANOVA was computed for the mean amplitude in the time-range from 600 to 800 ms. The data was analyzed using a 2 × 3 × 6 factorial repeated measures ANOVA with the factors imperative stimulus (Go and NoGo), door width (Narrow, Mid and Wide), time window (600-700 ms, 700-800 ms) and electrode location (Fz, FCz, Cz, Pz, POz and Oz). For post-hoc analysis, the data was contrasted using Tukey HSD. In case of violations of the sphericity, corrected p-values are reported. All ANOVA were computed as linear mixed models and all p-values for Tukey HSD contrasts were adjusted using Bonferroni method to account for “within” study design.
RESULTS
Subjective and Behavioral results
SAM Ratings
A 2 × 3 factorial repeated measures ANOVA with the factors imperative stimulus (Go and NoGo) and door width (Narrow, Mid and Wide) for each emotional dimension of the SAM questionnaire revealed differences in the main effect for width: Arousal (F2,4326 = 95.12, p < 0.0001), Dominance (F2,4326 = 46.42, p < 0.0001) and Valence (F2,4326 = 188.65, p < 0.0001). For the imperative stimulus, differences were found for Arousal (F2, 4326 = 443.54, p < 0.0001), Dominance (F2, 4326 = 435.49, p < 0.0001), and Valence (F2, 4326 = 446.20, p < 0.0001). Interaction effects revealed significant difference for all interactions (all p < 0.0001). Post-hoc contrasts using Tukey HSD (Figure 3) showed no significant differences for NoGo in Arousal, however significant differences were identified for Go between Narrow-Mid (p < 0.0001), Narrow-Wide (p < 0.0001) and Mid-Wide (p < 0.0001). For NoGo in Dominance no significant differences were revealed between Narrow-Mid (p = 0.1376), as opposed to Narrow-Wide (p < 0.0001) and Mid-Wide (p = 0.0334), whereas for Go no significant differences were found for Mid-Wide (p = 0.2199), as opposed to Narrow-Mid (p < 0.0001) and Narrow-Wide (p < 0.0001). For Valence, significant difference were revealed for all contrasts for Go, Narrow-Mid (p < 0.0001), Narrow-Wide (p < 0.0001) and Mid-Wide (p < 0.0001). However, for NoGo significant differences were only identified for Narrow-Mid (p < 0.0001) and Narrow-Wide (p < 0.0001).
Performance
To investigate the time it took participants from the Go-stimulus to passing the door, a one-way ANOVA with repeated measures for different door widths was computed revealing a significant difference for the factor door widths (F2,36 = 6.404, p = 0.0042; Figure 4). Post-hoc comparison (Tukey test) showed no significant differences in behavior when approaching the Narrow or Mid wide doors (p > 0.1), a tendency to be slower when approaching Mid as compared to Wide doors (p < 0.1), and a significant difference between approaching Narrow as compared to Wide door (p < 0.001) with significantly faster approach times for the Wide door condition.
EEG - Early event-related potentials
Posterior P1
With onset of the lights that allowed participants to see the room including the door (“Lights on”), the ERPs demonstrated a clear P1-N1 complex most pronounced over the occipital midline electrode with a first positive component around 100 ms, followed by a negative peak around 200 ms (Figure 5.1 and see Figure 5.2 in supplementary materials for full six channels). At the frontal midline electrode, this pattern was inversed and a negative component around 100 ms was followed by a positive peak observed around 200 ms. The 3 × 3 repeated measures ANOVA on P1 amplitudes for posterior electrodes revealed significant main effects for both the factors widths (F2,108 = 8.163, p = 0.005) and channel (F2,36 = 15.868, p < 0.0001). The interaction effect was not significant (F4,108 = 1.669, p = 0.1624). Post-hoc comparisons using Tukey HSD test revealed significant differences in peak amplitudes at channel Oz between Narrow and Mid wide transitions (p = 0.0021) and between Narrow and Wide transitions (p = 0.0065) but no differences between Mid and Wide transitions (p = 1). Tukey contrasts yielded no significant differences between electrodes, with differences in P1 amplitudes at POz comparing Narrow and Wide transitions (p = 0.028).
Posterior N1
The 3 × 3 repeated measure ANOVA on N1 amplitudes for posterior electrodes revealed a significant main effect for the factor door widths (F2,108 = 4.348, p = 0.0153) and no significant impact for the factor channels (F2,36 = 0.0893, p = 0.9147), nor the interaction (F4,108 = 1.304, p = 0.2731). Post-hoc Tukey HSD contrasts revealedno significant differences for Pz and POz. However,similar to posterior P1, significant differences at Oz for the comparison of Narrow and Mid wide transitions (p = 0.0113) and for the comparison of Narrow and Wide transitions (p = 0.0372) were found (Figure 6).
Anterior P1
An inverse pattern was observed for amplitudes over anterior leads with a main effect of door widths that differed depending on the affordances (F2,108 = 11.071, p < 0.0001). The main effect of channels also reached significance (F2,36 = 5.3627, p = 0.0092). Tukey HSD contrasts revealed significant differences only between Narrow and Wide transitions for FCz (p = 0.0071) and Cz (p = 0.0214), and a tendency at Fz (p = 0.0717). The interaction was not significant.
Anterior N1
The 3 × 3 repeated measures ANOVA on N1 amplitudes for anterior electrodes revealed no significant main effect for the factor door widths (F2,108 = 1.823, p = 0.1663). In contrast, the main effect of channels reached significance (F2,108 = 8.109, p = 0.0012). The interaction did not reach significance.
EEG - Motor-related processes
After onset of the imperative stimulus a positive peak at anterior leads and a negative peak at posterior leads were observed. For sake of brevity, this potential complex is referred to as early post imperative complex (EPIC). Reflecting similar cortical polarity as the P1-N1 complex, the EPIC was analyzed in a similar way, separating anterior leads (Fz, FCz and Cz) from posterior leads (Pz, POz and Oz), and detecting single peaks in individual averages.
Anterior EPIC
A 2 × 3 × 3 repeated measures ANOVA revealed significant difference in the main effect for widths (F2,270 = 4.21, p = 0.0157), imperative stimulus (F1,270 = 23.66, p < 0.0001), and for channel (F2,36 = 6.70, p = 0.0033). Nointeraction effectwas observed. The Bonferroni-corrected post-hoc Tukey HSD revealed no significant differences between the transition widths for different channels or imperative stimuli.
Posterior EPIC
The identical ANOVA for the posterior potentials of the EPIC revealed no significant impact of transition widths (F2,270 = 2.001, p = 0.1371) nor imperative stimulus (F1,270 = 2.30, p = 0.1298). Significant differences in EPIC amplitude were observed for the factor channel (F2,36 = 5.45, p = 0.0085). Since topographical differences were not in the focus of this study, no further post-hoc contrasts were computed. No interaction was significant.
PINV
In the preparation time prior to the onset of the door color change, indicating either to walk through the door or to remain in the same room, we observed no systematic negative going waveform as reported in previous studies (37, 49). However, after onset of the color change, a pronounced positivity, followed by a long-lasting negative waveform over fronto-central locations was observed in the ERP (Figure 7.1 and see Figure 7.2 in supplementary material for full six channels). This negative waveform resembled a post-imperative negative variation (PINV) as described in previous studies (40, 42, 50). The PINV component was observed 600-800 ms post imperative stimulus (color change of the door) and varied as a function of the affordance of the environment (door width). A global 2 × 3 × 6 factorial repeated measures ANOVA was computed to analyze the MRCPs using Go/NoGo, Width and Electrode as repeated measures. The ANOVA revealed significant differences in the main effect for Go/NoGo (F1,540 = 19.54, p < 0.0001) and for Electrode (F5,90 = 16.69, p < 0.0001). Significant differences were reported for the interaction effect of Go/NoGo:Channel (F5,540 = 5.25, p = 0.0001) and for Width:Channel (F10,540 = 2.61, p = 0.0042).
Post-hoc contrasts, using Tukey HSD, revealed significant differences only for the Go condition, as opposed to the NoGo condition (Figure 8.1). Similar to the early evoked potentials, differences were only observed in frontal and occipital sites and between Narrow and Mid door widths over FCz (p = 0.0059) and Oz (p < 0.0001), as wellas between Narrow and Wide doors at FCz (p = 0.0323) and Oz (p < 0.0001). No differences were observed betweenthe Mid and Wide doors (Figure 8.2 in the supplementary material for all six channels).
DISCUSSION
The main goal of this study was to assess whether brain activity is altered depending on the affordances offered by the environment. If such an account holds true, affordances should systematically modulate behavior and brain activity. Specifically, we hypothesized that perceptual processes co-vary with the environmental affordances leading to behavioral changes and that motor-related cortical potentials would vary as a function of affordances.
SAM and approach time
The results of the questionnaire should be interpreted with caution due to the amount of trials per participant, the varying sensitivity to VR and the different skills of subjective emotional evaluation. The analysis of subjective ratings revealed significant differences between different Go trials, but no differences for NoGo trials regarding Arousal ratings. When given a NoGo, participants responded perhaps arbitrarily, feeling unburdened, causing no significant difference among the three door widths. Notably, in cases of NoGo, all participants perceived a similar scene standing in front of a red (NoGo) door, turning around and answering the virtual SAM. The only variable in this sense was the door width, while the only difference from NoGo to Go, was the action itself. The subjective ratings highlight the influence of action on evaluating the environment. If space was to be investigated statically (comparable to the case of NoGo), we would not have been able to detect any differences for Arousal for varying door sizes, potentially due to the absence of action. Varying door sizes for Go trials yielded differences between passable and impassable conditions for Dominance, reporting that Narrow door was more dominating than Mid and Wide. However, for Valence we observed an increasing score the narrower the door, which is the opposite behavior observed for Arousal. These results indicate that being able to pass easily is more exciting, less pleasant and less dominating. This effect is perhaps grounded in the monetary reward participants could receive only when successfully passing through to the next room. Most importantly, however, the findings indicate that subjective reports differ significantly dependent on whether participants actively moved through the rooms or not implying an impact of action affective ratings of an environment. We speculate whether the omnipresent significant differences may be rooted in uniqueness of emotional states that varies from participant to participant. Such an account of emotional ratings is currently gaining credibility (51, 52).
The time it took participants to reach the door after onset of the imperative color change varied according to the environmental affordance. Participants approachedthe Wide door significantly faster than Mid and Narrow doors, while there was no significant difference for Mid and Narrow transitions. While the Wide door clearly offered a passage without greater computational demands_regarding the motor plan and execution, the Mid door width, being ambiguously wide/narrow, might have triggered motor processes simulating a transition to estimate whether the door was passable or not. In this sense, the Mid and Narrow doors, causing uncertainty, might have delayed approach times due to increasing processing demands. Admittedly, results derived from the approach time are limited, partly due to the caused fatigue of operating a physically demanding task for a relatively long time period, and partly due to the subjective manner and interpretation of passing a door that is seemingly impossible to pass. This caused participants to develop different approach strategies which caused different delays. However, the fact that participants, in general, spent significantly more time approaching the Narrow doors compared to Wide doors provides sufficient guidance for the analyses of cortical measures associated with these differences.
Cortical measures
Early evoked potentials
As an initial insight into the association of affordances and cortical potentials, we analyzed the early visual-evoked potentials. We expected to find differences in the stimulus-locked ERP at occipital channels reflecting differences in sensory processing of affordance-related aspects of the transition. Importantly, based on the assumption of fast sensorimotor active inferences that should be reflected in action-directed stimulus processing influencing not only sensory but also motor-related activity, we hypothesized to also find differences in the ERP over motor areas in the same time window as sensory potentials (i.e., between 50 and 200 ms). As illustrated in the analysis, we found significant differences in amplitudes of the visually evoked P1 component over the central occipital electrode dependent on the affordance of the transition. In addition, in line with our hypothesis, we also found a difference over fronto-central leads starting around 50 ms and lasting until 200 ms after onset of the doors display. Taken together, no significant differences in peak amplitudes were found when comparing the passable Mid and Wide doors while peak amplitude associated with both door widths significantly differed from impassable Narrow doors. Note that the visual scene of the three doors are comparable as they contained same physical contrasts, and that participants at this point did not know whether to go or not as they were merely introduced to the setting they might have to pass in a couple of seconds. As no significant differences were found for NoGo, it functions as a matching control, and thus we can interpret the differences in Go as affordance manipulation. These results indicate that impassable doors with poor affordances produce significantly different early evoked potentials compared to passable doors particularly at fronto-central and occipital sites. Thus, environmental affordances, in terms of being able to program bodily trajectory to transit spaces, yield a significant measurable effect on early cortical potentials best pronounced over frontal and occipital sites at approximately 200ms after first view of the environment.
Considering the affordance-specific pattern observed for the early P1-N1-complex, prior studies have shown this visual evoked potential complex to reflect attentional processes associated with spatial or feature-based aspects of stimuli (53–57). Attended stimuli elicit larger P1-N1 amplitudes than unattended ones. Based on these findings, the results suggest that passable transitions were associated with increased attentional processing. Approaching the affordance-specific pattern of P1-N1-complex using active inferences (58), the difference confirms the assumption that perceptualprocesses co-vary with environmental affordances. In this sense, the amplitude difference might be credited to the process of active inference of whether the body can actively move and transit at all. This implies that visual attention is also guided by action-related properties of the environment and support the concept of fast, lower sensorimotor active inferences, explained as hierarchical and dynamic model of the world. Similar to HAC (31) and active inference (30, 59), these findings are in line with parallel cortical processes integrating sensory information to specify currently available affordances. Similarly, this means that, how one might act upon the environment is an ongoing process of affordances, taking place as early as perceptual processes, and which situates actions in an intimate position with perception. Such early processes are deeply involved in the impression of the environment for an agent pointing towards the importance of movement in cognition, and of how an agent enacts the world. Given affordances are processed at such an early stage, we speculate whether the impression of an environment compose the immediate experience of the environment in a particular setting. Such an immediate experience fits with the term atmospheres as defined by Zumthor (60) “I enter a building, see a room, and - in a fraction of a second - have this feeling about it”, and thus relating the instantaneous emerging experience of space to affordances and action in general.
Motor-related potentials
Although the ERP plots indicate an affordance-trend of the EPIC, statistical tests revealed no significant differences. However, Narrow door width elicited the greatest amplitude, both in case of anterior positivity and posterior negativity. In line with prediction errors and affordances, the increased amplitude associated with Narrow transitions can be interpret as a reflection of the body simply not fitting, and yet forced to interact with the transition. Recall that prior to the imperative stimulus, participants have been standing for 6 s (σ = 1 s). The EPIC may have an influence on the PINV. The nature of the PINV component is not as well investigated as other ERP components, limiting the reliability of an interpretation based on only a few studies that treat the component as modality-unspecific, and rather “consider the PINV as an electrocortical correlate of a cognitive state” (61). Since the study by Gauthier and Gottesmann (62) the PINV, similar to affordances, has been hypothesized to act as a marker of change in psychophysiological state. Ever since, the PINV has been used to investigate depression, schizophrenia, learned helplessness and loss of control (40–42, 63, 64). Results show depressive and schizophrenic participants to exhibit an increased PINV that is explained as increased vulnerability for loss of control, as well as increased anticipation for future affective events (40, 42, 50). If an increased PINV reflects increased vulnerability for future events, as we observed for impassable doors, then the component, constituted by continuous motor potential activity, sheds new light on affordances as an intrinsic affective property of action itself. Casement and colleagues (42) even suggested the PINV to depend on lack of control as the state of having no influence; depriving the potential to act. This could explain the difference in the Narrow condition, as participants were instructed to attempt to pass at all times until failure leading to a sense of loss of control. Only in cases of Go did we observe a difference in the PINV component, which varied similar to the P1-N1-complex. Amplitudes of the component for Narrow doors were significantly different from Mid and Wide doors, while the passable conditions did not differ from one another. Further, there were no significant differences in the PINV component in cases of NoGo, emphasizing the importance of the motor execution itself to evoke the PINV component. These results point towards the PINV component as an expression of willingness to execute an actrestricted beyond ones’ own control, i.e. a designed environment. Thus, the PINV might serve as an excellent marker for affordances.
The presented results of the PINV are consistent with the observed increase in activity over fronto-central sites by Bozzacchi et al. (39). Bozzacchi and colleagues concluded that the meaning of the action and awareness of being able to act - affordances - affect action preparation, which is here understood as the motor-related potential prior to movement onset. We argue that the PINV component might reflect a willingness, or even intentional, aspect of affordances. This would mean that the PINV is not modulated by the perception (that the door is a different visual information), but reveals something about the intention of movement - which we translate to affordances. For this reason, we find significant differences in cases of Go, but not in NoGo, and further for passable compared to impassable. In light of HAC (31), a potential explanation for the absence of differences in the NoGo trials, is related to the immediate action selection, which in all cases (Narrow, Mid and Wide) is a simple turn to answer the questionnaire, and thus present the participant with identical affordances. When instead given a Go, cortical processes require an action selection related to the anticipated motor trajectory, which differs according to the affordances of the door width. Regarding the temporal aspect of transitioning to the next room, HAC suggests the higher levels bias the lower level competitions, which operate at the level of action itself, through a cascade of expected next affordances. The lower levels have a continuous competition of how to satisfy the higher expectations. Action selection, executed while unfolding the planned movements in a continuous manner, depend on the expectation of next affordances. Taken together, the post-hoc analyses revealed differences grouped for passable as compared to impassable doors throughout all channels, except for Pz. We do not observe any differences between Mid-Wide, but find significant differences between Narrow-Mid and Narrow-Wide. The greatest differences were found over fronto-central and occipital sites. Similar to the early evoked potentials, these results indicate that environmental affordances impact neural activity prior to action depending on whether one has to act or not.
Notably, regarding architectural experience, since the PINV component was only expressed in the Go condition (forced interaction with the environment), these findings support the importance of movement for architectural experience, in a sense that action or even only the perception of action possibilities alters brain activity. Visually guiding and propelling the body in space greatly influences the continuous emerging of affordances, which in turn affect the human experience. We found differences in fronto-central and occipital areas, prior to movement through space with the post-imperative negative going waveform most pronounced over FCz indicated an involvement of the supplementary motor area (SMA) as reported by Bozzacchi et al. (39). Interestingly, earlier studies showed involvement of SMA in visually guided actions (65), which is the essence of active inferences. The PINV can be generated independently from the re-afferent signal, which is, in terms of active inference, understood as ascending (bottom-up) proprioceptive prediction-errors (66). This suggests the PINV component might reflect descending (top-down) predictions, rendering SMA as anessential area of action-perception loop, and thus crucial for processing continuous affordances. This account might resolve the finding of fronto-centraldifferences in Go trials only. The SMA is anatomically bridging the frontal cortex with motor cortex - perhaps also functionally as argued by Adams et al. (66), as this anatomical nature fits with the proposed hierarchical characteristics of forward and backward projections in active inferences.
CONCLUSION
The present study provides strong evidence for affordances to be processed as early as perceptual processes, linking action and perception in a similar manner to active inference. The results points towards a conception of the brain that seems to dealwith “how can I act” while in parallel processes referringto “what do I perceive” take place. The results thus support the assumption that perception of the environment is influenced by affordances and action itself - hence, affordances and action can influence experience of an environment. Due to the importance of affordances and action for brain dynamics, this further emphasizes and qualifies the general idea of enactivism as a holistic approach to investigate cognition. We do not claim that architectural affordances are directly represented as a specific event-related potential component; however, we provide evidence for an action-perception account of cognition, which systematically differentiates according to the definition of affordances.
The nature of the analyzed brain activity emphasizes the importance of the intentional movement. Our results are consistent with the concept of continuous affordances as explained by active inferences. In terms of architecture, the results shed light on why transitions have been a constant throughout the history of architecture, perhaps especially in religious and other buildings that actively aimed at producing a certain experience of presence. Thus, the fact that we are predictive beings, in terms of architecture, means we should take into consideration how bodily movement alters perception. By altering perception, this would ultimately lead spaces to have a potentially physiological impact on users. Much remains to be uncovered in architectural cognition. Moving and transitioning in space, is continuously constructing a prediction of a world, a world that we perceive dependent on our action potentials, which informs brain, body and mind. Transitions in architecture form a holistic entity of architectural experience expressed as the unfolding of motor planning, spatial sequences and predictive mechanisms. Similar to Zeki (67), we speculate whether the ancient interest in tailoring transitions and sequences may have developed as a trial-and-error of active-narration, perhaps rooted in ancient knowledge of the predictive mind and action-perception parallel processing nature of the human being.
COMPETING INTERESTS
The authors declare no competing interests.
ACKNOWLEDGEMENTS
We thank Lukas Gehrke, Marius Klug and Federica Nenna for their support with the technical setup. We thank in particular Federica Nenna for her additional support in conducting the experiment. In addition, we would like to thank Andrea Jelic for important discussions and bringing essential insights of enactivism to our knowledge.