The reliability and validity of video analysis for the assessment of the clinical signs of concussion in Australian football

doi:10.1016/j.jsams.2016.02.015

Journal of Science and Medicine in Sport

Volume 19, Issue 10, October 2016, Pages 859-863

https://doi.org/10.1016/j.jsams.2016.02.015 Get rights and content

Abstract

Objectives

The objective of this study was to determine the reliability and validity of identifying clinical signs of concussion using video analysis in Australian football.

Design

Prospective cohort study.

Methods

All impacts and collisions potentially resulting in a concussion were identified during 2012 and 2013 Australian Football League seasons. Consensus definitions were developed for clinical signs associated with concussion. For intra- and inter-rater reliability analysis, two experienced clinicians independently assessed 102 randomly selected videos on two occasions. Sensitivity, specificity, positive and negative predictive values were calculated based on the diagnosis provided by team medical staff.

Results

212 incidents resulting in possible concussion were identified in 414 Australian Football League games. The intra-rater reliability of the video-based identification of signs associated with concussion was good to excellent. Inter-rater reliability was good to excellent for impact seizure, slow to get up, motor incoordination, ragdoll appearance (2 of 4 analyses), clutching at head and facial injury. Inter-rater reliability for loss of responsiveness and blank and vacant look was only fair and did not reach statistical significance. The feature with the highest sensitivity was slow to get up (87%), but this sign had a low specificity (19%). Other video signs had a high specificity but low sensitivity. Blank and vacant look (100%) and motor incoordination (81%) had the highest positive predictive value.

Conclusions

Video analysis may be a useful adjunct to the side-line assessment of a possible concussion. Video analysis however should not replace the need for a thorough multimodal clinical assessment.

Introduction

Concussion is a common problem in contact sports such as Australian football.¹ While the majority of athletes recover uneventfully, concussion may be associated with adverse outcomes in some individuals.² Acute complications are postulated to be more common with premature return to play.³ Moreover, recurrent concussion has been implicated in long-term problems such as depression and chronic traumatic encephalopathy (CTE),4, 5 although a causative link remains unproven.⁶ Overall, there has been an increased emphasis on a conservative management approach to sports concussion.² Critical to this approach is early recognition of concussion and removal of the athlete from training or competition.²

In the clinical setting, the diagnosis of concussion remains challenging. On-field and sideline assessments can be particularly difficult, especially given the variable presentation, non-specific nature of many of the clinical features and the absence of a reliable direct concussion biomarker.⁷ Moreover, symptoms and signs of concussion often evolve over time.² Consequently, a multimodal approach is currently recommended for the assessment of concussion to maximise the sensitivity of the clinical evaluation.²

Despite the development of simple clinical tools (such as the Sport Concussion Assessment Tool, SCAT3) to assist clinicians in the diagnosis and assessment of players with a possible concussion, it is acknowledged that many concussions are missed on the sidelines. Contributing factors may include the reluctance of some athletes to report symptoms following head trauma, and the often-transient nature of early (on-field) physical signs, which may have resolved and be missed by the time the player is subsequently assessed on the sideline.

In some elite sports, team physicians are at an advantage due to access to video review of collisions and impacts. Video review may facilitate assessment of the mechanism and impact forces of the injury, and allow the identification of brief early signs of concussion. In recent years, there has been an increased focus on the use of video technology on the sidelines, with a number of professional sports now recommending the use of video review for the assessment of a possible concussion. A number of studies are also appearing in the literature on the use of video review for the assessment of concussive injuries.8, 9, 10 It is important to note however that current data have largely been derived from retrospective review of videos, rather than live, in-game assessment.

In Australian football, a systematic study on video analysis of concussion has been conducted over the past 5 years. In the first year of the study, all player, game and injury related factors were assessed for reliability and association with risk of concussion.(reference paper 1) From this initial study, eight observable clinical signs of concussion were identified and defined. These included loss of responsiveness, impact seizure, slow to get up, motor incoordination, rag doll appearance, blank or vacant look, clutching of the head and facial injury (see Table 1).

The aim of this study was to determine the reliability and validity of identifying clinical signs of concussion using video analysis in Australian football.

Section snippets

Methods

This prospective cohort study was conducted in professional Australian football over two seasons (2012 and 2013). Approval for the study was obtained from the Cabrini Human Research Ethics Committee (CHREC 09-09-11-15). The Australian Football League (AFL) competition is the elite professional football code played in every state and territory in Australia, with 18 teams competing in 2012 and 2013. Games are played weekly between March and September each year (22 weeks of regular season games

Results

Two hundred and twelve incidents resulting in possible concussion were identified in 414 AFL games during 2012 and 2013 seasons. Ninety-nine of the incidents identified on video were subsequently confirmed as having a clinical diagnosis of concussion. A further 30 cases of concussion were reported by team doctors where the incident was not obvious or not observed on the initial video review.

One hundred and two videos were randomly selected for analysis. Forty-five of these cases had a clinical

Discussion

In recent years, there has been an increased focus on the use of video technology to review the mechanism of injury and assist clinicians with the identification of possible concussive injuries. This study used post-game video review in professional Australian football as a model of injury, to examine the reliability and validity of video signs of concussion. Eight observable clinical signs were chosen, based on the results of a preliminary video analysis study performed in Australian football.

Conclusion

Video analysis may be a useful adjunct to the assessment of possible concussions, and in some cases it may be the only means of confirming the diagnosis following injury. Video analysis however should not replace the need for a thorough multimodal clinical assessment. The role of video analysis in the assessment of concussion may be improved by improved definitions of the video signs, and high quality video feeds with the option of multiple angles of view to ensure an unobstructed assessment of

Practical implications

•
The addition of video analysis to the assessment may help improve consistency in diagnosis of concussion, as well as facilitate diagnosis in cases with brief signs that may have resolved by the time of the medical assessment.
•
Video signs such as loss of responsiveness, impact seizures/tonic posturing, motor incoordination and loss protective mechanisms are reliable and valid for the on-field recognition of sports-concussion. No video sign however is 100% reliable for a diagnosis of concussion.

Acknowledgements

The authors are honorary members of the AFL concussion working group.

No external financial support was received for this study.

The authors wish to thank: Dr. Margaret Staples for statistical advice and assistance with data analysis; Dr. Hugh Seward (CEO, AFL Doctors Association (AFLDA)), Dr. Andrew Potter (President, AFLDA), members of the AFLDA, Dr. Patrick Clifton (AFL Football Research Analyst) and Dr. Peter Harcourt (AFL Medical Director) for their support of the project; and the AFL for

References (14)

M. Makdissi et al.
Using video analysis for concussion surveillance in Australian football
J Sci Med Sport
(2016)
M. Makdissi et al.
A prospective study of postconcussive outcomes after return to play in Australian football
Am J Sports Med
(2009)
P. McCrory et al.
Consensus statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012
Br J Sports Med
(2013)
M. Makdissi et al.
Revisiting the modifiers: how should the evaluation and management of acute concussions differ in specific groups?
Br J Sports Med
(2013)
K.M. Guskiewicz et al.
Recurrent concussion and risk of depression in retired professional football players
Med Sci Sports Exerc
(2007)
A.C. McKee et al.
The spectrum of disease in chronic traumatic encephalopathy
Brain
(2013)
G.A. Davis et al.
Neurodegeneration and sport
Neurosurgery
(2015)

There are more references available in the full text version of this article.

Cited by (52)

Use of the concussion check protocol for concussion assessment in a female soccer team over two consecutive seasons in New Zealand
2024, Journal of the Neurological Sciences
Address deficiencies in access to sports sideline medical care by using a Concussion Check Protocol (CCP) for non-medically-trained people.
A prospective observational cohort study was undertaken on a single amateur female club-based soccer team over two consecutive years in New Zealand utilising a non-medically trained support person termed a Safety officer. CCP is an extension of the King-Devick test with features such as warning signs and symptoms of concussion built into the application. All players suspected of having a potential concussive injury were tested on the match sideline.
The study overall incidence of match-related concussions was 20.8 (95% CI: 11.8 to 36.6) per 1000 match-hrs, with mean missed-match duration of 31 (95% CI: 27.9 to 34.1) days. Twelve players over the study had a significantly slower post-injury KD (49.9 [44.3 to 64.1]s; χ²₍₁₎ = 11.0; p = 0.0009; z = −2.9; p = 0.0033; d = 0.30) and/or reported symptoms, compared with their own baseline (47.2 [44.3 to 64.1]s). CCP had an overall sensitivity of 100% (95% CI: 73.5% to 100.0%), specificity of 100% (95% CI: 69.2% to 100.0%) and positive predictive value (PPV) of 100% (84.6% to 100.0%).
Sideline use of CCP was undertaken successfully by non-medically trained people and provided a reliable platform for concussion identification.
Concussion incidence and mechanisms differ between elite females and males in Australian Football
2024, Journal of Science and Medicine in Sport
To investigate the incidence and the game circumstances of concussion in the professional female (AFLW) and male (AFL) competitions of Australian Football, to identify potential targets for risk reduction.
Retrospective cohort.
Concussion data were obtained from the AFL injury database, which included all concussions sustained by AFL (Male) players from 2015 to 18 and AFLW (Female) players from 2017 to 19. Concussions were diagnosed by experienced clinicians utilising standardised concussion assessment tools and injury definitions, as well as video review. Video footage was analysed to determine the circumstances each concussion occurred, which included the action and the contact-point of impact.
The incidence of concussion was higher in the AFLW (Female) compared to the AFL (Male) (IRR = 2.12, 95 %CI 1.54 to 2.92). Video footage was available for 194/252 (77 %) concussions in the AFL and 35/44 (80 %) concussions in the AFLW. Male players were most frequently concussed during marking contests (28 %) with primary impact from the upper limb (22 %) or the shoulder (19 %). Conversely, being bumped (23 %) or tackled (20 %) were the main actions associated with concussion in female players, with the head (29 %) or the ground (23 %) the most common contact-points of impact.
In elite Australian Football a higher incidence of concussion was demonstrated in female compared to male players. The mechanisms associated with concussion were also found to differ between male and female competitions, suggesting that different injury prevention interventions may be beneficial. In particular, a review of tackling and bumping skills training and education in the AFLW may reduce the risk of concussion.
On-Field and Pitch-Side (Sideline) Assessment of Sports Concussion in Collision Sports: An Expert Consensus Statement Using the Modified Delphi Technique
2024, Arthroscopy - Journal of Arthroscopic and Related Surgery
To perform a Delphi consensus for on-field and pitch-side assessment of sports-related concussion (SRC).
Open-ended questions in rounds 1 and 2 were answered. The results of the first 2 rounds were used to develop a Likert-style questionnaire for round 3. If agreement at round 3 was ≤80% for an item, if panel members were outside consensus, or there were >30% neither agree/disagree responses, the results were carried forward into round 4. The level of agreement and consensus was defined as 90%.
Loss of consciousness (LOC) or suspected LOC, motor incoordination/ataxia, balance disturbance, confusion/disorientation, memory disturbance/amnesia, blurred vision/light sensitivity, irritability, slurred speech, slow reaction time, lying motionless, dizziness, headaches/pressure in the head, falling to the ground with no protective action, slow to get up after a hit, dazed look, and posturing/seizures were clinical signs of SRC and indicate removal from play. Video assessment is helpful but should not replace clinical judgment. LOC/unresponsiveness, signs of cervical spine injury, suspicion of other fractures (skull/maxillo-facial), seizures, Glasgow Coma Scale score <14 and abnormal neurologic examination findings are indications for hospitalization. Return to play should only be considered when no clinical signs of SRC are present. Every suspected concussion should be referred to an experienced physician.
Consensus was achieved for 85% of the clinical signs indicating concussion. On-field and pitch-side assessment should include the observation of the mechanism, a clinical examination, and cervical spine assessment. Of the 19 signs and red flags requiring removal from play, consensus was reached for 74%. Normal clinical examination and HIA with no signs of concussion allow return to play. Video assessment should be mandatory for professional games but should not replace clinical decision-making. Sports Concussion Assessment Tool, Glasgow Coma Scale, vestibular/ocular motor screening, Head Injury Assessment Criteria 1, and Maddocks questions are useful tools. Guidelines are helpful for non-health professionals.
Level V, expert opinion.
The potential of head acceleration measurement to augment current best practice in concussion screening in professional Australian football players
2020, Physical Therapy in Sport
Citation Excerpt :
The positive predictive value of HAEs above threshold to diagnose concussion is therefore small (Patricios et al., 2017). In the current professional level of male AFL and female AFLW, where the practice of concussion screening is rigorous and guided by consensus derived guidelines (McCrory et al., 2013, 2017) in conjunction with video review during and after match by experienced clinicians (Makdissi & Davis, 2016a; 2016b), measurements of HAEs, even if reliable, may only deliver minimal marginal benefit in the overall process of detecting concussion. The potential and marginal diagnostic yield from the use of accelerometers must be balanced against cost, player and staff inconvenience and interruptions to the match.
To explore the potential utility of head acceleration event (HAE) measurements to augment identification of players for further concussion screening in non-helmeted contact sport.
Prospective observational pilot study.
210 (118 female) professional Australian football players in 2017 season.
Players wore the X-Patch® accelerometer for one match each with data collected across 14 matches. Players with HAEs above thresholds associated with concussion, 95 g (males) or 85.5 g (females), were compared to players identified to have suspected concussion by club personnel during the inspected matches. Video review of matches was undertaken by a physician blinded to HAEs to identify players with concussive signs.
Among 26 players (50% female) with HAEs above threshold, two players were screened for concussion. Of the remaining 24 players, nine were not visible on video at the HAE time, six sustained verifiable head impacts, and nine sustained verifiable body impacts with no head impacts. Among 184 players with HAEs below threshold, five players were screened.
Players were identified to have head impacts and suspected concussion in the absence of HAEs above threshold. Use of X-Patch® was not sufficiently reliable for identifying players for further concussion screening in professional Australian football. Video review of head impacts remains essential in concussion screening.
Diagnostic predictive values for sport-related concussions: a systematic review and diagnostic meta-analysis
2024, Journal of Neurosurgery
Examining Whether Loss of Consciousness Is Associated With Worse Performance on the SCAT5 and Slower Clinical Recovery After Concussion in Professional Athletes
2023, Journal of Neurotrauma

View all citing articles on Scopus

View full text

Original researchThe reliability and validity of video analysis for the assessment of the clinical signs of concussion in Australian football

Abstract

Objectives

Design

Methods

Results

Conclusions

Introduction

Section snippets

Methods

Results

Discussion

Conclusion

Practical implications

Acknowledgements

J Sci Med Sport

A prospective study of postconcussive outcomes after return to play in Australian football

Am J Sports Med

Consensus statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012

Br J Sports Med

Revisiting the modifiers: how should the evaluation and management of acute concussions differ in specific groups?

Br J Sports Med

Recurrent concussion and risk of depression in retired professional football players

Med Sci Sports Exerc

The spectrum of disease in chronic traumatic encephalopathy

Brain

Neurodegeneration and sport

Neurosurgery

Original research
The reliability and validity of video analysis for the assessment of the clinical signs of concussion in Australian football