Authors: Scott L. Bruce, EdD, AT, ATC
Sarah Stauffer, AT, ATC
Andrew Chaney, AT, ATC
Kelsey Garrison, AT, ATC
Wright State University
Corresponding Author:
Scott L. Bruce, EdD, AT, ATC
Assistant Professor/Director of Research
Wright State University
3680 Colonel Glenn Hwy
Dayton, OH 45435
937-245-7622
scott.bruce@wright.edu
Scott Bruce is an Assistant Professor and the Director of Research for the Athletic Training Program at Wright State University.
Taking Concussion Vital Signs Neurocognitive Test Under Varied Conditions
ABSTRACT
Neurocognitive test batteries are commonly used tools for concussion assessments in the medical professions. Administered at baseline and post-injury these tests provide information on a patient’s neurocognitive ability during the recovery and return-to-activity phases. In athletics, student-athletes usually take the baseline exam as a group in a computer lab prior to the season beginning. If a medical professional believes an individual has sustained a possible concussion, they will retest them and compare their post-injury and baseline results. A deficit in one of more areas of the neurocognitive test may be indicative of a possible concussion. The purpose of this study was to examine whether or not there was a difference in neurocognitive test scores from Concussion Vital Signs when tested under two different conditions: “lights out” and with distractions. Our study was a randomized control trial performed at a Midwestern NCAA, “mid-major” Division I Institution. The subjects were 15 college-aged students with an overall mean age of 19 years (1.2). There were seven females (mean age was 19 ± 0.77) and eight males (mean age was 20 ± 1.2). A paired t-test was used to determine if a difference in the neurocognitive test section scores between the conditions existed. On three of the ten test sections, there was a statistically significant difference between the baseline and distraction condition. On four of the ten test sections between baseline and the lights out condition for males, but not for females. Testing should be done in a quiet room, with distractions minimized, as distraction hinders focus and performance. The results of this study indicate males may have a more difficult time concentrating while taking neurocognitive tests than females. Administering neurocognitive testing in a quiet, well-lit room is the best condition for the patient to take these concussion-related tests.
Keywords: Neurocognitive test batteries, reaction time, concussion, test- retest
INTRODUCTION
Concussions among athletes have become a major concern in the United States. The Centers for Disease Control and Prevention states 1.6 to 3.8 million concussions happen in the United States each year (Langois, Marr, Mitchko, & Johnson, 2005) Diagnosing a concussion is difficult for two reasons: 1. the injury is completely subjective and the clinician must rely on the patient to provide all of the information related to the injury; 2. the lack of a biomarker that objectively measures and indicates whether or not someone has a concussion or does not have a concussion complicates the assessment. To assist in the assessment and management of concussions, neurocognitive test batteries have been developed (Broglio, Ferrara, Piland, & Anderson, 2006; Lau, Collins, & Lovell, 2011; Littleton, Register-Mihalik, & Guskiewicz, 2015; Roebuck-Spencer, Vincent, Schlegel, & Gilliland, 2013). Concussion Vital Signs (CVS) is one of many neurocognitive test batteries developed.
Neurocognitive tests are a tool clinicians can use in their assessment and management of concussions. Athletes take these tests during the pre-season to provide baseline scores for the clinician. If a clinician suspects that an athlete has suffered a concussive blow, as part of their assessment he/she would have the athlete re-take the exam at prescribed intervals based on the protocol used by the clinician (Resch, McCrea, & Cullum, 2013).
Normative data for CVS was established in 2006 with a large scale study of 1,609 subjects ranging in age from 7-99 years. The test-retest reliably for CVS was similar to other well-known computerized neurocognitive studies and comparable neuropsychological tests. Concurrent validity was also assessed and found to be comparable to similar type tests (Gualtieri & Johnson, 2006).
Littleton et al., (2015) examined college-age subjects, administering the CVS test three times, with one-week between each test session. The subjects were all tested individually, in a private setting. A Pearson r test was used to assess the test-retest correlations, and were found to be quite wide, ranging from 0.11 to 0.87. The calculated intraclass correlation coefficients (ICC) ranged from 0.10 to 0.86. The results found between test 1 and test 2 may indicate a practice effect (Littleton et al., 2015).
In comparing four different neurocognitive test batteries, of which CVS was one, the authors found a Pearson r correlation from a low of 0.34 on verbal memory to a high of 0.79 on complex attention. The ICC values ranged from a low of 0.29 (verbal memory) and high of 0.79 (complex attention). The researchers qualitatively assigned “very high” reliability for coefficients ≥ 0.90, and “high” reliability for values of 0.89 – 0.80. They further classified 0.79 – 0.70 as adequate reliability, values between 0.69 – 0.60 were classified marginal, while values of 0.59 or less were considered low. The researchers stated that “one-third of the scores . . . had adequate or better reliability.” Cole, Arrieux, Schwab, Ivins, Qashu, & Lewis, 2013).
The instructions for CVS indicate that administration of the test should be in a controlled setting with groups ranging from 10-20 participants taking at a time (Concussion Vital Signs, 2016a). There is an absence of research examining the effects of neurocognitive test batteries under varied conditions. The purpose of this study was to examine whether or not there was a difference in neurocognitive test scores, from Concussion Vital Signs, when tested under two different conditions: “lights out” and with distractions. The current study was a randomized control trial performed at a Midwestern NCAA, “mid-major” Division I Institution.
METHODS
The study was reviewed and conducted under the guidelines of the university’s Institutional Review Board. All participants signed the approved informed consent form before beginning the study. The study took place at an NCAA Division I, Midwestern institution. A convenience sample of 15 healthy, physically active, college-age, students (7 females and 8 males) volunteered to participate in the study. The mean age of the participants was 19 (±1.2) years (females = 19 [±0.77] yrs; males = 20 [±1.2 yrs]). Students were excluded from the study if they had suffered a concussion within the last six months. (One subject was disqualified after the study began due to suffering a concussion.)
All participants took the CVS baseline test at the same time in a university computer lab under “normal conditions” as per CVS instructions: lights on, distractions kept to a minimum, participants were instructed to remaining quiet and in place until all students had completed their test (Concussion Vital Signs, 2016a). Participants were seated with at least one computer station between each of the subjects. The subsequent CVS test administrations were done with a two-week interval between tests.
The baseline test was used as the control condition, for which the other two conditions were compared. After baseline testing, participants were then randomly assigned to one of two groups (which functioned as the treatment conditions): “lights out,” and “lights on with distractions.” For group assignment, participants were alphabetized by their last name. Those participants in the top half of the order were assigned to “Group A”, the other half were assigned to “Group B”. The participants were not informed of which condition they were taking at the second CVS administration until they arrived at the computer lab. The condition for the second administration was determined by a flip of a coin, heads for “lights out” for the “Group A”, tails for “light on with distractions” for “Group B”. For the third and final CVS test administration, participants took the exam under the condition they had not previously been assessed.
The environment for the “lights out” condition was similar to the normal baseline conditions except that the room’s overhead lighting was turned off. The participants were instructed to remain quiet and in place until all students had completed their test. The environment for “lights on with distractions” condition was similar to the normal baseline conditions, except a variety of music was played throughout the test period and the participants were permitted to leave once they had completed their test. Subjects were instructed, that if possible, to sit in the same seat as they did during their previous test administration.
The main outcome measures examined were as follows: verbal memory, visual memory, psychomotor speed, reaction time, cognitive flexibility, continuous performance test, and executive functioning. A brief description of each of these tests is presented in Table 1. To determine if there was a statistically significant difference between test conditions a series of paired t-tests were utilized. The alpha level was established a priori at ≤ 0.05.
RESULTS
All statistical analyses were performed on SPSS 22 (IBM, Chicago, IL). Comparison were made for all subjects from the two conditions to their baseline assessments. Although participants were not segregated when taking the CVS tests, data were analyzed for all subjects, and then by gender. Paired t-tests were used for all comparisons (Tables 2-7).
Comparing baseline scores to the “lights out” condition, there were no statistically significant results for comparisons made for all subjects or for when examining only females (Tables 2 & 3). Only the male subjects had statistically significant results, and only three of the 10 tests were statistically significant (p ≤ 0.05). Those tests included, shifting attention test (p ≤ 0.001), cognitive flexibility (p = 0.014), and executive function (p = 0.006) (Table 4).
When comparing the baseline to the lights on with distractions condition, statistically significant results were found in half of the tests for all subjects. Those test that were statistically significant included verbal memory (p = 0.006), visual memory (p = 0.016), psychomotor speed (p = 0.048), simple reaction time (p = 0.021), and the shifting attention test (p = 0.014) (Table 5). None of the CVS tests were statistically significant for the lights on with distractions condition for female subjects only (Table 6). When this comparison was restricted to male subjects only, four of the ten tests were found to be statistically significant. Those tests were verbal memory (p = 0.042), visual memory (p = 0.020), psychomotor speed (p = 0.043), and the shifting attention test (p = 0.002) (Table 7).
DISCUSSION AND APPLICATIONS IN SPORT
The majority of the statistically significant results were found in males, but not their female peers. A possible reason for why males had statistically significant results is that for this group of males, they appeared to be less focused and more easily distracted compared to the female participants. A possible explanation for the statistically significant results in males compared to the non-statistically significant results in the females during the lights out condition is that some of the male subjects volunteered that their eyes began to fatigue during the latter part of the test. The three tests for which the males were statistically significant with the light out condition were three of the final four tests taken in the test battery.
The results of this study, reinforces the importance of administering neurocognitive exams in a quiet room, free of distractions. Perhaps the best condition for administering these tests would be to test each individual alone, in a room free of distractions. However, this may not be possible due to the number of individuals to be tested and the time restraints placed on athletic trainers and sports medicine personnel. The best results for this study occurred when the test batteries were taken according to the recommended testing procedure. Distractions can be seen to hinder focus, which can affect the scores of the test. Lights should remain on, since a dark room may cause the participant to encounter eye fatigue, which can hinder test scores. The results of this study may indicate that males have a harder time concentrating compared to their female counterparts.
Another reason for ensuring an appropriate testing environments is the need for accurate baseline data. When the baseline test data is used for comparisons to post-injury test administrations inaccurate or poor baseline results may lead to erroneous post-injury assessments. Neurocognitive testing is but one tool that is available to the clinician in their concussion assessments. In addition to neurocognitive testing, clinicians have graded symptom scales, short-term and long-term memory assessments, balance assessment, functional testing, and their clinical evaluation available to use in their concussion assessments and management. If the clinician uses only neurocognitive test data to guide their clinical practice, and they are not looking at the entire patient’s presentation, they are more likely to make mistakes and erroneous decisions.
A limitation of this study is that only one neurocognitive test battery, Concussion Vital Signs, was assessed. Repeating this study examining other neurocognitive test batteries under the conditions utilized in the administration of CVS in this study would be beneficial to determine if the current study results are consistent throughout all of the various neurocognitive test batteries.
CONCLUSIONS
This study was able to demonstrate the importance of providing the proper environment as recommended for the neurocognitive testing outlined in the instructions for test administration. Giving these exams in environments where distractions are present may lead to erroneous or inaccurate results.
REFERENCES
1. Broglio, S., Ferrara, M., Piland, S. G., & Anderson, R. (2006). Concussion history is not a predictor of computerised neurocognitive performance. British Journal of Sports Medicine, 40(9), 802-805.
2. Concussion Vital Signs. (2016a). CVS Test Administration Guide. Retrieved from CNS Vital Signs, LLC., Morrisville, NC http://www.concussionvitalsigns.com/index.html
3. Concussion Vital Signs. (2016b). Helping Solve the Sports Concussion Post-Injury Puzzle. Retrieved from CNS Vital Signs, LLC., Morrisville, NC http://www.concussionvitalsigns.com/index.html
4. Gualtieri, C. T., & Johnson, L. G. (2006). Reliability and validity of a computerized neurocognitive test battery, CNS vital signs. Arch Clin Neuropsychol, 21, 623-643. doi:3. 10.1016.j.acn.2006.05.007
5. Langois, J., Marr, A., Mitchko, J., & Johnson, R. (2005). Tracking the silent epidemic and educating the public: CDC’s traumatic brain injury-associated activities under the TBI Act of 1996 and the Children’s Health Act of 2000. J Head Trauma Rehab, 20(3), 196-204
6. Lau, B. C., Collins, M. W., & Lovell, M. R. (2011). Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sports-related concussion. The American journal of sports medicine, 39(6), 1209-1216.
7. Littleton, A. C., Register-Mihalik, J., & Guskiewicz, K. M. (2015). Test-retest reliability of a computerized concussion test: CNS vital signs. Sports Health, 7(5), 443-447. doi:10.1177/1941738115586997
8. Resch, J. E., McCrea, M. A., & Cullum, C. M. (2013). Computerized neurocognitive testing in the management of sport-related concussion: an update. Neuropsychology review, 23(4), 335-349.
9. Roebuck-Spencer, T. M., Vincent, A. S., Schlegel, R. E., & Gilliland, K. (2013). Evidence for added value of baseline testing in computer-based cognitive assessment. Journal of athletic training, 48(4), 499.
Tables
