“The nature and prevalence of injury during CrossFit training,” Dr. Paul Hak, Part Two

9e0b72f3d274687_421145

In part one of this review of the JSCR’s most recent “CrossFit” publication, “The nature and prevalence of injury during CrossFit training,” we addressed the misquotation of the paper’s reported findings. In this post, we will detail some of the serious problems within the body of the paper.

First, the data collected in this study came from an online survey, which solicited respondents via distribution “…amongst international CrossFit online forums.” A large, randomly selected group of CrossFit trainees would have reduced the chances of such a bias, but this is not what we get. The authors acknowledge this phenomena on page 5 of the study:

Screen shot 2014-06-05 at 1

Next, in the introduction to the study, the authors cite the CHAMP paper on CrossFit. They write that the Consortium “highlighted significant risk of injury in those involved in extreme conditioning programs such as crossfit.” This is false. The CHAMP paper did nothing more than speculate about the risk of performing CrossFit, provide no data to support its claim and make a number of fundamental errors in describing the CrossFit program.

Finally, the paper makes some fundamental mistakes in categorizing the CrossFit program:

Screen shot 2014-06-04 at 4

This is simply inaccurate, and the authors cite no source for this claim.

But these issues are minor compared to the unscientific and sloppy conclusion the authors claim to reach from their data.  After explaining the collected survey data, the authors break down the total number of injury complaints into anatomical categories. They note that the bulk of these injuries were to the shoulder and low back, with the number of shoulder injuries at 25.8% of total injury complaints.

Next, they compare this percentage to that reported to one other demographic – elite competitive Olympic weightlifters.

Screen shot 2014-06-04 at 4

Based only on this comparison, they state that CrossFit has a “high prevalence of shoulder injuries.” They go on to suggest that

Screen shot 2014-06-04 at 4

Note the use of the qualifier “may” in this statement. Yet later in the paper, under the header “Practical Applications”, the authors conclude that

Screen shot 2014-06-04 at 4

If I write this progression of ideas down as a formal argument, the authors have created something that looks like this:

  1. Compared to Elite Olympic weightlifters, CrossFit athletes have higher % of shoulder injuries.
  2. This may be due to higher repetitions, intensity, and load.
  3. Therefore, CrossFit’s shoulder injury incidence is high, and this is due to higher repetitions, intensity, and load.

This is a non sequitur.  First, CrossFit athletes are not Olympic weightlifters, and CrossFit incorporates a huge variety of movements that olympic weightlifters do not do. Any opinion based on the expectation that injury profiles between these very different activities should be similar, is illogical.

Yet based on this assumption the authors qualify CrossFit’s shoulder injury incidence as “high.” Compared to what? Competitive Olympic weightlifting. How does the distribution of shoulder injury in CrossFit compare to gymnastics, swimming, or general fitness training? On these comparisons, the paper is silent.

Asking why CrossFit has a higher rate of shoulder injuries than weightlifting is kind of like asking why weightlifters have more knee injuries than specialists in the gymnastics rings.

Asking why CrossFit has a higher rate of shoulder injuries than weightlifting is kind of like asking why weightlifters have more knee injuries than specialists in the gymnastics rings.

In order to explain this, the authors speculate that this may be because of “high repetition range and at high intensity, often with heavy weights.” One obvious issue with this formulation is that high repetitions cannot be performed with heavy weight. The number of repetitions an athlete is capable of performing given a certain weight are what determine how heavy or light a weight is. This is a fundamental contradiction in terms.

Looking past this error, the authors provide no evidence that this claim is true, and yet astonishingly, two pages later this “may” statement has transformed into a statement that the authors have identified a causal relationship between high-rep, high-intensity overhead lifts and shoulder injury:

“The high prevalence of shoulder and low back injuries [in CrossFit], however, due to the high intensity, high repetition and heavy weight movements, needs to be considered…”

How these authors made this leap, without any citation, argument, or evidence to support their claim, is unknown. In conclusion, this “study” seems to consist of benign injury data that the authors manipulated according to their biases and assumptions.

In order to get to the bottom of this, I contacted corresponding author Paul Hak, who was more than happy to allow me to interview him.

Following the reasoning of his paper, I asked Dr. Hak why a larger percentage of injuries from CrossFit might occur in the shoulder. While the paper suggests, and then concludes that this is due to high-rep Olympic lifts (a popular, yet baseless cry from the anti-CrossFit world), my conversation with Dr. Hak leads to a far more reasonable answer. He explains that this is “….because we tend to do a lot with the barbell, we tend to hang from the pull-up bar, and we tend to invert ourselves and do gymnastic movements…”

Dr. Hak even goes on to say that “I think my data doesn’t allow you to say that particularly one form of exercise or training predisposes to one form of injury….it only describes what injuries we see.” 

And yet this is exactly what Dr. Hak’s paper does in the conclusion, where the prevalence of shoulder injury is described as being “due to the high intensity, high repetition and heavy weight movements….”

How does Dr. Hak explain this discrepancy between his personal conclusion and the conclusion published in his paper? His response surprised me:

“I think you might find it’s um, been mistyped…”

Dr. Hak goes on to explain that “the numbers that we have and the way the paper and their evidence stacks up, you can’t say that there is a causal relationship between the two….it’s a typo.” 

While the study has only been published ahead of print, the “accepted” watermark in the copy we obtained indicates that it has been through the JSCR’s peer review process.

It appears that this this yet another example of the NSCA’s peer review process failing to identify basic problems with the validity of the “CrossFit” research it publishes. Is this a systemic problem, or does their peer review only fail to identify “typos” that implicate CrossFit as dangerous? Note that the NSCA has already claimed they lack sufficient information to form an opinion on this study.

Again, we find ourselves asking what purpose the NSCA’s peer review process even serves if it is capable of missing such fundamental mistakes in the scientific process.

7 comments

  1. John Weatherly

    Good article. This is minor but it shows how easy it is to make mistakes. I have seen typos in JSCR (and contacted Dr. Kraemer about it) before. Here, in the fourth paragraph from the bottom it says “the numbers that we have and the way the paper and THERE evidence stacks up” has the word THEIR spelled wrong, It should be”their” and not “there. “

  2. Thanks John, I corrected the spelling.

    What’s interesting about the error in the Hak study is it doesn’t seem to be a typo in the sense you’ve described (this isn’t a simple spelling error). This is an entire sentence that has been structured to causally link a certain prevalence of injury to CrossFit training. I don’t think this happened on accident. To change the sentence to what the author claims to have concluded would require some significant changes.

  3. Jordan

    The term “may” is used consistently in research because a hypothesis isn’t proven/accepted until it has been supported by several other studies. The researcher uses the term “may” because regardless of the findings of his study nothing can be accepted as fact. The fact that you highlighted that and used it as part of your argument hints that you have little experience reading or conducting research.

  4. Jordan

    A lot of people love CrossFit the same way some people love Boxing, MMA, Football, etc. Many people go into it knowing you’re going to hurt at some point. The desperation in some of the CrossFit community in the denial that you can get hurt when participating is crazy. CrossFit is fun. You might get hurt. Kickboxing is fun too, but you might get hurt. Skateboarding is fun, but..you might get hurt.

  5. Landon

    My wife injured a hamstring playing tennis last week and a calf playing today. I’m now embarking on a world wide campaign to do away with tennis.

  6. Alex Bedard

    Another issue with this research that was not addressed is a fundamental flaw in sampling techniques and inferences. Non-probabilistic sampling, such as that used in this study, cannot be used to make inferences about a population. Where the researchers used the information from the sample to make generalizations about all CrossFitters (the population) and compare those generalizations to other sports is an inherent fallacy in statistics. These inferences can only be made as a result of probabilistic sampling, in which all CrossFitters have an equal chance of being selected for the study. Since these researchers did not employ probabilistic sampling, they can only use statistics to describe their sample, not to make inferences about all the injuries of all CrossFitters.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: