What do blind people “see” with retinal prostheses? Observations and qualitative reports of epiretinal implant users

Introduction: Retinal implants have now been approved and commercially available for certain clinical populations for over 5 years, with hundreds of individuals implanted, scores of them closely followed in research trials. Despite these numbers, however, few data are available that would help us answer basic questions regarding the nature and outcomes of artificial vision: what do participants see when the device is turned on for the first time, and how does that change over time? Methods: Semi-structured interviews and observations were undertaken at two sites in France and the UK with 16 participants who had received either the Argus II or IRIS II devices. Data were collected at various time points in the process that implant recipients went through in receiving and learning to use the device, including initial evaluation, implantation, initial activation and systems fitting, re-education and finally post-education. These data were supplemented with data from interviews conducted with vision rehabilitation specialists at the clinical sites and clinical researchers at the device manufacturers (Second Sight and Pixium Vision). Observational and interview data were transcribed, coded and analyzed using an approach guided by Interpretative Phenomenological Analysis (IPA). Results: Implant recipients described the perceptual experience produced by their epiretinal implants as fundamentally, qualitatively different than natural vision. All used terms that invoked electrical stimuli to describe the appearance of their percepts, yet the characteristics used to describe the percepts varied significantly between participants. Artificial vision for these participants was a highly specific, learned skill-set that combined particular bodily techniques, associative learning and deductive reasoning in order to build a “lexicon of flashes” - a distinct perceptual vocabulary that they then used to decompose, recompose and interpret their surroundings. The percept did not transform over time; rather, the participant became better at interpreting the signals they received. The process of using the device never ceased to be cognitively fatiguing, and did not come without risk or cost to the participant. In exchange, participants received hope and purpose through participation, as well as a new kind of sensory signal that may not have afforded practical or functional use in daily life but, for some, provided a kind of “contemplative perception” that participants tailored to individualized activities. Conclusion: Attending to the qualitative reports of participants regarding the experience of artificial vision provides valuable information not captured by extant clinical outcome measures. These data can both inform device design and rehabilitative techniques, as well as grant a more holistic understanding of the phenomenon of artificial vision.

223 both companies at the two hospital sites. Open ended, semi-structured interviews were conducted 224 and all participants were interviewed by one of the two authors. Face-to-face interviews were 225 conducted at an agreed-upon place, at the hospital during the days of their trainings. On occasion 226 follow up questions were asked of participants over the phone. These interviews were either tape 227 recorded upon patient agreement and subsequently transcribed, or notes were taken directly during 228 informal conversations.

230
Data analysis 231 232 The data were analyzed using Interpretative Phenomenological Analysis (IPA) [45]. This particular 233 form of qualitative analysis was selected because of its emphasis on how participants experience 234 their world. The process of analysis derives themes or categories from the data itself, rather than 235 using predefined categories. These data were supplemented with observational data collected by 236 the authors, translated into field notes that were then subject to qualitative coding methods [46]. 252 253 The individuals we observed and spoke with -as well as the majority who have elected to receive 254 these devices -were diagnosed with retinitis pigmentosa, a group of disorders that involves the 255 gradual degeneration of the eye's light-sensitive cells. These individuals had fully functioning 256 visual systems before the first symptoms appeared -often in adulthood -and had functioned in the 257 world as sighted persons until they were no longer able. Those who were older at disease onset 258 often did not learn the compensatory or assistive techniques that are more readily available to 259 younger people with visual impairments (e.g. schools for the blind, which teach braille and other 260 techniques) 261 262 Individuals have unique life histories and reasons for wanting the device, but commonalities 263 underlying most of their stories was the desire for greater independence and autonomy, to 264 contribute to research for future generations (a hereditary disease, some of the individuals had 265 children who had since been diagnosed), and a desire to challenge themselves. Whether it be an 266 assistive tool with which to supplement their mobility, something that would allow them to return 267 to the workforce or allow them greater social connection to individuals around them, the 268 individuals we spoke with desired greater agency and connection within the world around them. 269 We also found that participants often expressed wanting to prove their capability (to themselves as 270 well as others), as a kind of psychological emancipation from the "handicap" status they 271 unwillingly represented.

272
273 According to industry researchers we spoke with, the most important predictor of device success 274 is subject selection; in particular the psychological profile individual subjects who elect to get the 275 device. The ideal subjects, these researchers said, were soldiers and former athletes. They had the 276 endurance and commitment to "do what it took" to get through the training. They had a self-277 sacrificial mentality and often a stoicism that made them attractive participants. These individuals 278 were referred to by clinical researchers as "fighters" ("des battants"). "Excellent" candidates who 279 were neither soldiers nor athletes but who were thought to have the qualities of a "fighter" included 280 individuals who held jobs that involved challenging cognitive tasks (e.g. a computer scientist or a 281 teacher). Other predictors of "successful" participants included shorter disease duration and 282 younger age. A "reasonable" participant was someone who both met the diagnostic criteria and 283 whom they judged to have "realistic" expectations. Tempering expectations, we would hear many 284 of these researchers say, is a crucial factor in subject selection and preparation. Subjects who had 285 accepted or come to terms with their low vision condition often do best, a rehabilitation specialist 286 stated. They are more likely to accept the difference between the reality of artificial vision to what 287 they might have been expecting it to be like.

309
Initial activation 310 311 A number of weeks after implantation, but before the camera is turned on, the device is aligned 312 and activated in order to make sure the device fits correctly, that the base components are 313 functioning, and to introduce the subject to the perception invoked with electrical stimulation. They 314 follow with the "systems fitting," in which global thresholds or settings for the stimulation 315 parameters that yields a reliably "good" perception are determined (first amplitude followed by 316 phase duration and frequency). The subjects' expectations are tempered once again at this point: 317 they are told that this is not when they will "see" (they are told this will be when the camera is 318 turned on). Nevertheless we came to learn that this two-day period is associated with significant 319 change and learning, as the participant learns to identify the signal, or "phosphene" -the building 320 block of artificial vision. 321 322 The devices are built and rehabilitation protocols implemented with the expectation that activation 323 of a single electrode will produce a single point of light -a phosphene -ideally with a low threshold 324 of activation and a brightness that corresponds to the amplitude of the electrode. If each electrode 325 produces a single, isolated point of light, it would allow a visual image to be recreated using a 326 pixel-based approach, assembling phosphenes into objects and images similar to an electronic 327 scoreboard (Fig 1) [49,50]. 340 During the course of our observations we would explicitly ask the participant to describe the 341 percept associated with stimulation, either during the protocol or after the session was over, and 342 found there to be significant variability and ambiguity in this reeducative process. In UK 343 participants who were explicitly asked by the author (CED) during activation, many reported 344 "glitter" or "sparkles" during single electrode stimulation. One subject called their percepts "cheese 345 puffs" that whizzed by laterally; another compared them to "exploding, pink popcorn;" for yet 346 another they appeared as red diamonds, sometimes in a cluster, sometimes single (even if only one 347 electrode was being activated). French participants met by the author (HK) usually stuck with the 348 terms the rehabilitation specialists used: "flashes" or "signals", or "flickering lights" 349 ("clignotements").  440 441 The reasons for ambiguity or uncertainty are multiple: 1) Description: on one hand it is difficult to 442 describe "the quality" one's visual experience the way it is for anyone to describe the qualia or 443 "what it is like" of conscious experience. 2) Discernment: It also may be difficult to discern: the 444 signal is being produced within a "background" of visual distortion that characterizes blindness 445 (that is, it is not a calm backdrop of darkness on which these phosphenes make their appearance, 446 but instead can be a stormy sea of light and shadow, color and shape). 3) Difference: finally, it may 447 also be that these signals are something significantly different than natural vision, and for that 448 reason the same vocabulary that we use for natural vision just might not do. Camera activation 500 501 Two weeks after the initial activation and systems fitting, the time comes to turn the camera on.
502 The subject is told that this is when they will begin to gain back a kind of functional vision, and so 503 it is a time that is often greeted with a lot of excitement. News media and camera crews who are 504 interested in the sensational aspect of these devices are often told to come to this session.

505
506 When the camera is activated and all of the electrodes can be activated together, we found 507 participants reported that things became much more chaotic and "noisy." The mass of flashing 508 lights coalesces, and the vocabulary subjects use when describing their percepts focuses on changes 509 in the overall signal: e.g. "stronger," "more signal," "busy," "calm." 510 511 The first task that is performed when the camera is activated is tracking and localization -often 512 with a piece of white paper, or the beam of a flashlight on the wall of a dark room. The participant 513 is asked to indicate if and when they see the light, and if possible, in which direction it is moving.
514 This process is also marked by significant ambiguity.  543 544 The movement of the paper is associated with something -a "flash" …in a "curvy shape." This 545 "something" is the first step. With the therapist's guidance, suggesting certain expressions to 546 describe the sensations, the individual learns to define "movement" with the device associated to a 547 sensation appearing then disappearing in different spots, and hence comes to recognize its 548 trajectory. The main idea is that with time, the individual will learn to identify shapes. The hope is 549 that the flash(es) that correspond(s) to the paper will be different than flash(es) associated with a 550 different object; that over time, an individual will develop a lexicon of flashes corresponding to 551 various shapes and objects. The next step is to build out this lexicon.

553
Building a lexicon: simple geometries 554 555 The first phase of this learning protocol takes place in the radically simplified context, where the 556 participants is seated in front of a computer screen or a table covered in a black cloth. This 557 simplified, high contrast situation is considered an ideal environment for the device in which they 558 use a "building blocks" approach, inspired by simple geometries, to learn to identify simple shapes 559 that they will later use to "decompose" more complex visual spaces. That is, this training is based 560 on the logic that the visual environment can be deconstructed into a series of simple geometric 561 shapes that can then be assembled into the mind of the individual and reinterpreted into a coherent 562 visual scene.
563 564 The first exercises consist of presenting simple shapes to the participants and have them learn to 565 use their bodily techniques to first locate the objects, and later to identify those objects. In a typical 566 training task, the trainer will place an object -say a white styrofoam ball -in the middle of the 567 black table, and then instructs the individual on using the eye, head and camera alignment and head 568 scanning techniques, giving them hints and reminders until the individual is ready to locate the 569 object, by reaching out and touching it. Through repeated trial and error attempts, the individual 570 learns to interpret the signals they are receiving in conjunction with the movements of their head.
571 The subjects are also handed the ball, encouraged to sense of how "ballness" corresponds to the 572 signals they receive. Over the course of a session, different-sized balls are used, progressing to 573 different shapes (ball versus rod, ball versus ring, etc. -Fig 4), and then low vision computer 574 monitor tasks (e.g. grating acuity - Fig 5). Through associative learning, the subject learns to pair 575 the kind of signal they receive with a certain shape, a skill which they can later use to decipher the 576 environment. Decomposing space 598 599 The subject is then asked to put these skills of simplified geometries to work during the second 600 phase of the training, in orientation and mobility tasks. They begin in the hallway outside of the 601 training room, where they are encouraged to rethink the environment through an arrangement of 602 lines. Recalling the vertical, horizontal and diagonal lines they were taught to identify on the 603 computer screen, individuals are led to reconstruct space mentally, according to the basic angles 604 composing it. The vision therapists are told by the companies to assume that the hallway is 605 transformed by the device into a high contrast, black-and-white scene (Fig 6) and they coach them