Facial expressions of emotional stress in horses

Horses have the ability to generate a remarkable repertoire of facial expressions, some which have been linked to certain emotional states, for example pain. Studies suggest that facial expressions may be a more ‘honest’ expression of emotional state in horses than behavioral or physiological parameters. This study sought to describe the facial expressions during stress of healthy horses free of pain, using a standardized method of recording facial expressions in video. Stress was induced in 28 horses by subjecting them to road transport and 10 of these horses were also subjected to social isolation. The horses served as their own control. A body-mounted, remote controlled heart rate monitor provided continuous heart rate measurements during the interventions. The horses’ facial expressions were video-recorded during the interventions. Frequency and duration of each facial expression were then determined, according to the Equine Facial Action Coding System. Heart rate increased during the stressful interventions (p=0.01), confirming that the interventions were stressful. Using both the human investigation- and the co-occurrence methods, the following facial traits could be observed during stress: eye white increase (p<0.001), nostril dilator (p<0.001), upper eyelid raiser (p<0.001), inner brow raiser (p=0.042), tongue show (p<0.001) along with an increase in ‘ear flicker’ (p<0.001) and blink frequency (p<0.001). The facial actions were successfully used to train a machine-learning classifier to discriminate between stressed and calm horses, with an accuracy of 74.2 %. Most of the facial features identified correspond well with previous research on the subject, for example flared nostrils, repetitive mouth behaviors, increased eye white, tongue show and ear movements. Some features selected as indicative of emotional pain-free stress are used in face-based pain assessment tools, such as dilated nostrils, eye white increase or inner brow raiser. The relation between facial expressions of stress and pain should therefore further be studied.

7 138 roughly the same body weight. They were kept at an authorized research facility at "Blinded 139 for Review". These horses were fed hay four times a day, and oats once a day according to a 140 nutritional plan that supported normal condition. All horses were allowed out on pasture for 6 141 hours a day and otherwise kept in individual 3 m x 4 m boxes. During the experiment, horses 142 were moved to other boxes in the same facility and acclimatized for at least 16 hours. Horses 143 were moved together in pairs, stabled besides each other, and kept in their regular stable herd 144 (together for at least the previous 6 months). Each pair of horses had the same feeding and 145 housing routine and had the same caretakers in all stables.

146
147 Eighteen privately owned horses (PRI) were included. They comprised 10 geldings, seven 148 mares and one stallion, of the breeds Thoroughbreds (n=5), mixed-breed ponies (n=4), 149 Standardbred trotters (n=3), and Swedish warmblood/riding breeds (n=6), with body weight 150 ranging between approximately 400 and 600 kg. The median age of horses in this group was 151 10 years (range 3-24 years). They were considered healthy by their caretakers, had not been 152 subjected to veterinary treatment for the previous two months, and had not been treated with 153 analgesics during that period. The horses were managed at home, by the horse owner, in the 154 routines to which they were accustomed. All were kept in stables except for the thoroughbreds, 155 which were kept in a free-range system. Three of the PRI horses were kept at the university but 156 were treated as though they were privately owned.
209 The identity of the video-recordings of the social isolation group was blinded in relation to 210 horse and intervention before annotation. Selection of videos for the social isolation group was 211 performed using an automated face detection system [32], where sequences were selected if 212 the head position of the horse was suited for annotation. Thirty-second sequences of video with 213 a side-or front-view confidence of at least 60% were selected. If several selections were 214 available, a random number generator was used to select one clip. The selected clips were 215 manually inspected to ensure that the software had successfully identified a face. If not, a new 216 clip was randomly selected.

217
218 All films were annotated in a blinded manner by two EquiFACS-certified veterinarians with a 219 minimum of 70% correct annotations compared with expert raters. All transportation and 220 baseline films were also annotated by JL who is also certified in EquiFACS. Annotation was 221 performed using a template consisting of all codes in EquiFACS, including supplemental codes 222 and the visibility code VC74 (code for unscorable), but without head movements (AD51-

Selection of EquiFACS codes in stressed horses
233 Since inter-rater agreement was good, one set of annotations was randomly selected and used 234 for each video. For each selected action unit or action descriptor, frequency and duration were 235 observed. Frequency of ear flicker movements was also investigated. In order to do this, a 236 movement index was created, by describing ears forward (EAD101) and ear rotator (EAD104) 237 occurring together within a one-second interval. It is important to note that this is not an action 238 descriptor but a definition of an occurrence, where the selected action descriptors occur in 239 succession to constitute the "ear flicker". 240 241 EquiFACS codes were analyzed using the method described by Kunz et al. [28], here called 242 the Human FACS Investigation (HFI) method. Action units that accounted for more than 5% 243 of total action unit occurrences in stress videos were selected. From this subset, action units 244 detected at higher frequency in stress videos than in no-stress videos were selected as the final 245 set of stress action units. While the HFI action unit selection method ensures that selected codes 246 are frequent and distinct, they may have only a slightly stronger correlation with the emotional 247 state and can exclude less frequent, but highly discriminative, action units. Therefore, the 248 relative temporal distribution of action units was also considered. In order to do this, the method 249 of Rashid et al. [27], here referred to as the Co-occurrence method, was used to calculate the 250 co-occurrence of action units. This method selected EquiFACS codes that occurred together 251 with other EquiFACS codes more frequently in stress than in no-stress states. Since onset and 252 offset of EquiFACS codes were recorded in ELAN, codes which appeared simultaneously or 253 in close relation to each other could be further studied. EquiFACS codes that occurred within 254 a predetermined period (observation window size, OWS) were recorded as co-occurring.
255 Action units that exhibited the largest difference in co-occurrence patterns between stress and 256 no-stress states were selected. The method uses directed graphs to record and calculate 257 differences in co-occurrence patterns. Furthermore, a paired t-test for mean values was used to 258 test significance, with p<0.05 considered significant. 259 260 For both the HFI and Co-occurrence methods, occurrences of ears forward (EAD101) and ear 261 rotator (EAD104) that were included in the "ear flicker" category were not double counted for 262 EAD101 and EAD104 separately. As a result, occurrence counts of EAD101 and EAD104 did 263 not occur within a one-second interval of one another. 278 Results 279 Heart rate during interventions 280 The heart rate during interventions is shown in Fig 1. Heart rate increased from a pooled mean 281 of 41 bpm (SD 10.6) during baseline to 70 bpm (SD 24.3) during transportation and to 55 bpm 282 (SD 21.9) during social isolation. The increase in both groups was significant (p<0.01). In 283 general, the spread of samples in the transportation group indicated that these measurements 284 were somewhat more disrupted, due to more movement of the horse, but in general the heart 285 rate samples were of good quality. Heart rate after the interventions decreased fully to the 286 baseline level, indicating that it was the intervention that caused the rise in heart rate. The data 287 also indicated that the transportation intervention was more stressful to the horses than the 288 social isolation intervention. Based on the similarities in these results, the PRI and UNI groups 289 were regarded as one stress group in the following analysis of facial expressions.

Selected EquiFACS codes (HFI method)
295 Table 1 shows action units selected using the HFI method. All action units that comprised at 296 least 5% of stress action unit occurrences were more frequent in transportation videos than 297 baseline videos. Blink action units (AU145 and AU47) and inner brow raiser (AU101) had the 298 most similar rate of occurrence between stress and no-stress states, while eye white increase 299 (AD1), nostril dilator (AD38) and upper lid raising (AU5) exhibited the largest difference in 300 frequency between transportation and baseline recordings. The selected action units for social 301 isolation stress mostly showed similar codes. Unlike for the transportation group, half blink 302 (AU47) was not selected for the social isolation group due to occurring more frequently in no-303 stress videos, and upper lid raiser (AU5) is not selected due to low frequency in social isolation 304 videos. On the other hand, ear rotator (EAD104) was selected during social isolation. "Ear 305 flicker" was more frequent and more pronounced in transportation than in social isolation.
306 The selected action units when both groups were combined were used as a larger 'stress' group.
307 All action units that comprised at least 5% of stress action unit occurrences were also more 308 frequent in stress videos than baseline videos. The chosen action units for social isolation were 309 identical to those selected for transportation stress, but the percentage difference between no-310 stress and stress frequency counts was noticeably larger for inner brow raiser (AU101).    (Fig 3). According to the HFI method, the 371 action units of a stressed horse included upper lid raiser (AU5) and inner brow raiser 372 (AU101), as well as blink (AU145) and "ear flicker". The frequency of the action descriptors 373 nostril dilator (AD38) and eye white increase (AD1), not describing certain muscle-induced 374 codes but rather the effects of other muscle movement, was also significantly increased.
375 According to the Co-occurrence method, tongue show (AD19) and mouth open (AU25) were 376 also important. When comparing the HFI method with the Co-occurrence method for 2-