In part one of this review of the JSCR’s most recent “CrossFit” publication, “The nature and prevalence of injury during CrossFit training,” we addressed the misquotation of the paper’s reported findings. In this post, we will detail some of the serious problems within the body of the paper.
First, the data collected in this study came from an online survey, which solicited respondents via distribution “…amongst international CrossFit online forums.” A large, randomly selected group of CrossFit trainees would have reduced the chances of such a bias, but this is not what we get. The authors acknowledge this phenomena on page 5 of the study:
Next, in the introduction to the study, the authors cite the CHAMP paper on CrossFit. They write that the Consortium “highlighted significant risk of injury in those involved in extreme conditioning programs such as crossfit.” This is false. The CHAMP paper did nothing more than speculate about the risk of performing CrossFit, provide no data to support its claim and make a number of fundamental errors in describing the CrossFit program.
Finally, the paper makes some fundamental mistakes in categorizing the CrossFit program:
This is simply inaccurate, and the authors cite no source for this claim.
But these issues are minor compared to the unscientific and sloppy conclusion the authors claim to reach from their data. After explaining the collected survey data, the authors break down the total number of injury complaints into anatomical categories. They note that the bulk of these injuries were to the shoulder and low back, with the number of shoulder injuries at 25.8% of total injury complaints.
Next, they compare this percentage to that reported to one other demographic – elite competitive Olympic weightlifters.
Based only on this comparison, they state that CrossFit has a “high prevalence of shoulder injuries.” They go on to suggest that
Note the use of the qualifier “may” in this statement. Yet later in the paper, under the header “Practical Applications”, the authors conclude that
If I write this progression of ideas down as a formal argument, the authors have created something that looks like this:
- Compared to Elite Olympic weightlifters, CrossFit athletes have higher % of shoulder injuries.
- This may be due to higher repetitions, intensity, and load.
- Therefore, CrossFit’s shoulder injury incidence is high, and this is due to higher repetitions, intensity, and load.
This is a non sequitur. First, CrossFit athletes are not Olympic weightlifters, and CrossFit incorporates a huge variety of movements that olympic weightlifters do not do. Any opinion based on the expectation that injury profiles between these very different activities should be similar, is illogical.
Yet based on this assumption the authors qualify CrossFit’s shoulder injury incidence as “high.” Compared to what? Competitive Olympic weightlifting. How does the distribution of shoulder injury in CrossFit compare to gymnastics, swimming, or general fitness training? On these comparisons, the paper is silent.
In order to explain this, the authors speculate that this may be because of “high repetition range and at high intensity, often with heavy weights.” One obvious issue with this formulation is that high repetitions cannot be performed with heavy weight. The number of repetitions an athlete is capable of performing given a certain weight are what determine how heavy or light a weight is. This is a fundamental contradiction in terms.
Looking past this error, the authors provide no evidence that this claim is true, and yet astonishingly, two pages later this “may” statement has transformed into a statement that the authors have identified a causal relationship between high-rep, high-intensity overhead lifts and shoulder injury:
“The high prevalence of shoulder and low back injuries [in CrossFit], however, due to the high intensity, high repetition and heavy weight movements, needs to be considered…”
How these authors made this leap, without any citation, argument, or evidence to support their claim, is unknown. In conclusion, this “study” seems to consist of benign injury data that the authors manipulated according to their biases and assumptions.
In order to get to the bottom of this, I contacted corresponding author Paul Hak, who was more than happy to allow me to interview him.
Following the reasoning of his paper, I asked Dr. Hak why a larger percentage of injuries from CrossFit might occur in the shoulder. While the paper suggests, and then concludes that this is due to high-rep Olympic lifts (a popular, yet baseless cry from the anti-CrossFit world), my conversation with Dr. Hak leads to a far more reasonable answer. He explains that this is “….because we tend to do a lot with the barbell, we tend to hang from the pull-up bar, and we tend to invert ourselves and do gymnastic movements…”
Dr. Hak even goes on to say that “I think my data doesn’t allow you to say that particularly one form of exercise or training predisposes to one form of injury….it only describes what injuries we see.”
And yet this is exactly what Dr. Hak’s paper does in the conclusion, where the prevalence of shoulder injury is described as being “due to the high intensity, high repetition and heavy weight movements….”
How does Dr. Hak explain this discrepancy between his personal conclusion and the conclusion published in his paper? His response surprised me:
“I think you might find it’s um, been mistyped…”
Dr. Hak goes on to explain that “the numbers that we have and the way the paper and their evidence stacks up, you can’t say that there is a causal relationship between the two….it’s a typo.”
While the study has only been published ahead of print, the “accepted” watermark in the copy we obtained indicates that it has been through the JSCR’s peer review process.
It appears that this this yet another example of the NSCA’s peer review process failing to identify basic problems with the validity of the “CrossFit” research it publishes. Is this a systemic problem, or does their peer review only fail to identify “typos” that implicate CrossFit as dangerous? Note that the NSCA has already claimed they lack sufficient information to form an opinion on this study.
Again, we find ourselves asking what purpose the NSCA’s peer review process even serves if it is capable of missing such fundamental mistakes in the scientific process.