More of What Alan Schwarz of The New York Times Doesn’t Cover About the Inadequacy of Dr. Joseph Maroon’s ‘ImPACT’ Concussion Management

Near-Death in Japan Pro Wrestling, With Real Fallout
June 8, 2011
Brent Boyd Loses NFL Disability Court Case – It Shouldn’t Be the Last Round
June 9, 2011

No doubt some casual readers of this blog think I’m doing an excessive metaphorical tap dance on Dr. Joseph Maroon, the visionary co-founder of ImPACT Applications, Inc., as well as the team neurosurgeon for the Pittsburgh Steelers, a widely quoted spokesperson for the National Football League’s concussion policy committee, a stalwart of the University of Pittsburgh Medical Center, partner in the company licensing the research behind the supplement Vindure, endorser of the supplement Sports Brain Guard, author of The Longevity Factor, and septuagenarian triathlete. Plus, of course, medical director of World Wrestling Entertainment, in which capacity he treats head injuries both real and imagined. Maroon is also the co-author of the NFL-funded study of a new football helmet design, and the overzealous promotion of that study by the Riddell helmet company is now under federal investigation.

These same readers may also wish I’d call a halt to my proverbial soft shoe on Alan Schwarz of The New York Times, the baseball statistical nerd who, by his own modest account, “[killed] myself for six months to expose a serious safety problem – and even conspiracy – in youth football.” Maroon praises Schwarz in The New Yorker, Schwarz quotes Maroon uncritically in The Times, and Schwarz tells me flatly that Maroon is a not an issue in the larger national sports concussion scandal, “for reasons of which you are totally unaware.”

I point these readers and others to author-journalist-blogger Matt Chaney’s January 28 post, “Brain Expert Omalu Wants Longer Rest for Concussed Football Players,”

The subhead of Chaney’s article: “Sideline concussed juveniles for three months, says the breakthrough neuropath; Neuropsychological testing lacks validation and might be harmful, critics caution; NFL players rebuke theory of ‘safer’ football through their ‘behavior modification’”

Here’s most of the section headed “Critics Doubt Efficacy of NP testing for concussion diagnosis, ‘return to play’”:

Today’s general view that concussion management works or can work in tackle football is rendered highly suspect, if not effectively discredited, by independent review and mounting adverse opinion of experts and witnesses like players.

Linebacker Fujita notes he hasn’t been measured on neural baseline for two NFL seasons. Might not matter, anyway, for NP testing has taken a systematic beating by reviewers of late. Observations and findings of medical literature from 2005 to 2010, listed without full author groups or first names, include:

*Randolph et al, 2005, for Journal of Athletic Training: “Despite the theoretic rationale for the use of NP testing in the management of sport-related concussion, no NP tests have met the necessary criteria to support a clinical application at this time. Additional research is necessary to establish the utility of these tests before they can be considered part of a routine standard of care… until NP testing or other methods are proven effective for this purpose.”

*Patel et al, 2005, for Sports Medicine: “Numerous guidelines have been published for grading and return-to-play criteria following concussion; however, none of these have been prospectively validated by research and none are specifically applicable to children and adolescents.”

*Mayers, 2008, for Archives of Neurology: “Current guidelines result from thoughtful consensus recommendations by expert committees but are chiefly based on the resolution of symptoms and the results of neuropsychological testing, if available. Adherence to this paradigm results in most injured athletes resuming competition in 1 to 2 weeks.”

*Duff, 2009, for ASHA Leader: “Indeed, the identification and management of concussion has become a growing public health issue. Considered to be the fastest-growing sub-discipline in neuropsychology, concussion management poses unique challenges and opportunities for those working with school-aged children. … There is no consensus on the best course of action for concussion management. In fact, there are as many as 22 different published guidelines for grading concussion severity and determining return to play. … Developers are working to collect data regarding reliability, validity, and clinical utility of these (NP) tools; independent replication is still forthcoming.”

*Echemendia et al, 2009, for British Journal of Sports Medicine: “Post-injury assessment requires advanced neuropsychological expertise that is best provided by a clinical neuropsychologist. Significant international differences exist with respect to the training and availability of clinical neuropsychologists, which require modification of these views on a country by country basis.”

*Covassin et al, 2009, for Journal of Athletic Training: “…little is known about the use of baseline neurocognitive testing in concussion assessment and management. … We found that the majority of ATs (athetic trainers) are interpreting ImPACT results without attending a neuropsychological testing workshop. … The use of baseline-testing, baseline testing re-administration, and post-concussion protocols among ATs is increasing. However, the ATs in this study reported that they relied more on symptoms than on neurocognitive test scores when making return-to-play decisions.”

*Maerlender et al, 2010, for The Clinical Neuropsychologist: “Although computerized neuropsychological screening is becoming a standard for sports concussion identification and management, convergent validity studies are limited.”

*Piland et al, 2010, for Journal of Athletic Training: “Obtaining (self-reported symptom) statements before a concussion occurs assists in determining when the injury is resolved. However, athletes may present with concussion-related symptoms at baseline. … In other words, some post-concussive symptoms occur in persons who have not sustained concussions, rendering the specificity of alleged post-concussive symptoms suspect.”

*Schatz, 2010, for American Journal of Sports Medicine: “Computer-based assessment programs are commonly used to document baseline cognitive performance for comparison with post-concussion testing. There are currently no guidelines for how often baseline assessments should be updated, and no data documenting the test-retest stability of baseline measures over relevant time periods.”

*Comper et al, 2010, for Brain Injury: “Despite the proliferation of neuropsychological research on sports-related concussion over the past decade, the methodological quality of studies appears to be highly variable, with many lacking proper scientific rigour. Future research in the area needs to be carefully controlled, repeatable and generalizable, which will contribute to developing practical, evidence-based guidelines for concussion management.”

*Eckner et al, 2010, for Current Sports Medicine Reports: “The sports medicine practitioner must not rely on any one tool in managing concussion and must be aware of the strengths and limitations of whichever method is chosen…”

Unfortunately, software packages like ImPACT, long criticized for its direct connections to the NFL, are widely employed as cornerstone for concussion evaluation and typically by untrained clients, as literature and news reports confirm.


Irv Muchnick


  1. Steve Jones says:

    Also check out Broglio, Journal of Athletic Training, 2007 who reported poor reliability among all computer tests used for concussion management and a nearly 40% false positive rate on the ImPACT.

  2. Joe Bloggs says:

    Broglio’s study is severely flawed. He is an ATC and simply does not have the math to conduct the type of study he published. He simply does not undertsand statistics well enough to make the conclusions he made. Nonetheless, once proper statistics are applied ImPact is a poor test although not as poor as the paper states.

    Tarring all tests with the ImPact brush is ridiculous.

    As far a Dr. Randolph’s study, Chris has an agenda that is not supported by research. He doesn’t like computerized tests. In fact, computerized tests have their place and when used properly are critical to assessing the condition of the injured. On the other hand, ImPact is not a good test and has never been used properly.

Concussion Inc. - Author Irvin Muchnick