Media Misrepresents Sketchy Study on Recovery from Exercise

This past Sunday, Gizmodo proclaimed, “Take a day off between CrossFit workouts, study says.” And the New York Post published “Doing CrossFit two days in a row could get you sick: study.”

Surely, rigorous research supports these headlines’ claims, right? Scientists must have found significantly higher rates of illness in a group that trained at a CrossFit affiliate two days in a row, as compared to a group that took a rest day in between. And obviously, the study’s researchers must have explicitly recommended that CrossFit athletes take a day off after a single workout day to avoid this illness. Otherwise, the press would avoid baselessly defaming CrossFit affiliates.

Not exactly. Well, not at all. Contrast the news stories with the study itself, and with the official statement from the paper’s authors. In the press release, author Dr. Ramires Tibana stated,

“we recommend that they decrease their training volume after two consecutive days of high intensity training to prevent possible immunosuppression …”

Strangely, the Gizmodo article itself quoted this exact statement from Tibana toward the bottom of the article. How did Gizmodo’s headline writer take a recommendation to rest after two days of training, and turn it into headline recommendation to rest after a single day of training?

Worse, the researchers did not record any of the subjects becoming sick, calling the Post’s headline into question. Instead, they wrote that two days of training,

“reduced the levels of anti-inflammatory cytokines, which are proteins produced by white blood cells that act to dampen inflammation.”

So the paper’s data supports claims about a correlate of sickness, not sickness itself.

Again, the media has favored sensationalist headlines about CrossFit. This is both annoying and a dereliction of duty since it represents time they could have spent addressing real problems, such as the lifestyle-related chronic diseases that kill 1.8 million Americans yearly. A search for New York Post coverage on “obesity” produces an article exclusively dedicated to “healthy s’mores” – complete with vegan marshmallows, which somehow manage to contain even more sugar than regular marshmallows. This may not surprise you, though. Does anyone still consider American journalism to be near its heyday?

Scientists still enjoy considerable prestige, especially among non-scientists, as witnessed by rampant Bill Nye and Neil Degrasse Tyson memes. Unfortunately, science’s practice does not always live up to its ideals. American exercise science, especially. 

7ce.jpg
History of Fraud Against CrossFit Affiliates

Let us now turn to the study itself. Normally, we may trust researchers’ basic integrity. Their conclusions and assumptions may be off base, but if the researchers write that the subjects followed the Paleo diet strictly for two months, we do not conclude that they instead ate ice cream on a regular basis. We cannot assume integrity in this study, however.

Author James Navalta has a history of recklessly tolerating fraudulent research, at least when it harms CrossFit affiliates. Navalta is the editor of the International Journal of Exercise Science, the journal that published Steven Devor’s and Michael Smith’s second fraudulent study on CrossFit 614. Known to Russell’s Blog readers as the “Paleo Fraud,” this study claimed that subjects ate strict Paleo, when some in fact violated the Paleo diet at every meal. Not only did the subjects not follow strict Paleo, the lifestyle challenge they participated in did not require them to. I reported these findings to the IJES, which described my email as “compelling” and “serious.” IJES then emailed Devor and Smith, who insisted that their study was accurate. This was enough for Navalta and his Co-Editor-in-Chief T. Scott Lyons. Navalta took Devor and Smith at their words, and kept the study published. And this despite the fact that Devor and Smith had already failed to substantiate their fraudulent data in a related study, which was already the subject of multiple lawsuits.

Should we trust Navalta’s integrity this time?

If The Researchers Didn’t Lie, What Does The Study Mean?

Even assuming for sake of argument that the researchers did not fabricate their data—does the recovery study still make any sense?

Let’s look at the programming the subjects followed for two consecutive days. According to the New York Post and other papers which re-ran its story, the researchers had the subjects “perform workouts such as kettle bells, tire flipping and ring pull-ups.”

But that is not even close to the truth. Below is what the subjects actually did:

fphys-07-00260-t002.jpg

First, the researchers failed to record the performances the athletes achieved. This oversight is especially problematic for the two AMRAP workouts, where both the volume and intensity depend entirely on an athlete’s effort.

Any CrossFit athlete knows the difference between lazily getting through an AMRAP, and pushing the pace for the duration of the workout. And 45 reps of double-unders and snatches will affect biological markers differently than 400+ reps will. We cannot know which end of the spectrum the athletes fell on in this study, which further muddies the waters.

Unhelpfully, the researchers allege that “the goal of the endurance conditioning sessions” was to “complete each training session in the quickest time possible, without compromising exercise technique.” Again they were AMRAP workouts: in this type of workout the time domain is fixed for all participants.

I for one, would like to know how the subjects attempted to complete a 10-minute AMRAP “in the quickest time possible.” Chuck Norris must have been a subject—he once completed a 10-minute AMRAP in 4 minutes.

Second, both days follow the same format with similar movements, near identical time domains and loadings, and relatively high volume compared to what is taught by CrossFit Inc. and commonly practiced at CrossFit affiliates:

– Olympic lifts from blocks for 5 X 1 at 80 %
– 3 X 5 touch-and-go Olympic lift at 70-75 % 5 RM, 90 seconds rest
– 3 sets of a gymnastics movement, 90-120 seconds rest
– 10-12 minute AMRAP of a low-resistance pulling movement and a jumping movement

If the researchers intended to assess the “influence of different CFBT sessions on immune response in our cohort of adult men,” these highly similar training sessions fall short. The subjects were “recruited through advertisements,” the workouts were performed in a laboratory, not in a CrossFit affiliate, and the researchers refer to the programming as their “experimental design.” These facts imply that the researchers designed this programming themselves, specifically for the purpose of this experiment. Programming faults therefore are the researchers’ complete responsibility.

Is performing nearly identical programming, with four separate training components, two days in a row truly representative of most training at CrossFit affiliates? Say the athletes instead performed Cindy on the first day, and 7 sets of 2-rep deadlifts the next day. Could we reasonably expect a similar physiological response to more varied CrossFit workouts? 

If the answer is no, then we cannot generalize from the results to what happens in “CrossFit” generally. The study’s results would only apply to gyms that regularly perform nearly identical, higher-volume programming on consecutive days (more likely to happen in a bootcamp or spin class than in CrossFit classes).

What if the Data Still Applies Beyond This Particular Case?

Let us grant, for the sake of argument, that the data is not fraudulent AND that the programming effectively relates to what happens every day at CrossFit affiliates. What implications might the data have for a trainer or athlete?

This is where it gets a little technical. Here are the changes in cytokines the study presents:

fphys-07-00260-g003.jpg
If you don’t understand all of those terms, know that the researchers intended to use these measurements to get a glimpse at the immune response to the workouts. Still, what follows is a brief summary of what these Y-axises mean:

IL-10, Interleukin-10: “an anti-inflammatory cytokine normally released locally from immune cells to help resolve inflammation.”

IL-6, Interleukin-6: a cytokine that exhibits both “pro- and anti-inflammatory properties.”

OPG, Osteoprotegerin: “a cytokine member of the TNF receptor superfamily that binds to two ligands, the receptor activator of nuclear factor kappa-B ligand (RANKL), a key cytokine for the differentiation of osteoclasts, and a ligand related to the induction of apoptosis (TRAIL), involved in immune surveillance.”

(Be careful if you use the paper’s own faulty reference section to research these terms yourself—their Sasso study link mistakenly goes to an unrelated study on ghrelin. If you click on Tibana’s own previous writing on CrossFit, you’ll find yourself instead reading about the benefits of statins.)

It is important to avoid reductionist views of the immune response to exercise. We should not simplify this into “good” and “bad” markers. For example, chronic inflammation is linked to heart disease and type-2 diabetes. And yet, inflammation seems to be a necessary component of adaptation to exercise. The NSCA, normally quite risk-averse to non-sponsored products and services, suggests that the,

“acute inflammatory response to damage is a primary mediator of hypertrophic adaptations.”

So, inflammation equals gains?

The researchers in this study present IL-6 as “pro-inflammatory,” yet other research suggests that IL-6 also exhibits “anti-inflammatory properties.” They present IL-10 as “anti-inflammatory,” yet in certain situations, IL-10 may have pro-inflammatory effects.

To muddy the waters further, the researchers repeatedly contradicted themselves. They claimed in the abstract that, 

“In conclusion, two consecutive Crossfit® training sessions increase pro/anti-inflammatory cytokines with no interference on muscle performance in the recovery period.”

Yet their conclusion section states,

“In conclusion, 2 consecutive days of Crossfit training elicited a significant decrease in anti-inflammatory cytokines without impairments in muscle power.”

Did the anti-inflammatory cytokines significantly increase or decrease following two consecutive days of CrossFit training? It’s not clear. Why does their abstract’s conclusion directly contradict their paper’s conclusion?

Let’s look closer. Keep in mind that by anti-inflammatory cytokines, they’re referring to IL-10. The abstract claims that IL-10,

“… displayed an increase immediately after only WOD 1 [44 ± 52%] (p = 0.046), and decreased 24 and 48 h following WOD 2 (~40%; p = 0.018) as compared to pre-exercise values …”

So according to this line, IL-10 decreased 24 and 48 hours after the second workout, in a statistically significant manner (since the p value for this decrease was “0.018”, easily meeting the paper’s test for significance). And the Results section claimed that,

“IL-10 concentration presented a statistically significant decrease (~40%; p = 0.018) 24 and 48 h after WOD 1.”

Yet they contradicted both those lines in the discussion section:

“… in the present study the response of IL-10 concentration presented a statistically significant increase immediately after only training session 1 (44 ± 52%). But 24 h after training session 1, post training session 224 and 48 h after training 2, IL-10 concentrations were not statistically significant different (P > 0.05) from pre intervention.”

So did IL-10 levels significantly decrease 24 hours after the first workout, and 24 and 48 hours after the second workout, or not? It depends what line you read. (Though, in fact, both claims could be false). Perhaps the conflicting claims about significance stem from the researchers first calculating the changes in absolute values, and then calculating them in terms of percentages. They do not elucidate, nor negotiate, this apparent contradiction. We are left to guess. And this is not some tangential issue—the alleged decline in anti-inflammatory cytokines was the primary finding of the study, judging by the researchers’ press release and conclusion. If the authors are not sure whether this decrease was statistically significant, one wonders why they chose to highlight it.

Then again, their abstract claimed the anti-inflammatory cytokines increased, not decreased, after the two training sessions, so who knows?

To Be Determined: Consequences and Significance 

In the end, the study raises more far more questions than it answers. And the researchers acknowledge that,

“Further training and longitudinal investigations are necessary to determine the consequences of this finding … future research is needed to determine the significance of this result.”

Perhaps the authors could have waited for the “consequences” and “significance” to be determined before shopping their premature findings to the press. Dr. Tibana also mentioned in his release that, “A rest day is important for recovery for subsequent training sessions.” He could have acknowledged that CrossFit Inc. has employed a standard work-rest strategy for at least 14 years: “three-days-on, one-day-off.”

And while they were at it, the researchers could have corrected the media that misrepresented and misinterpreted their work. On the other hand, spreading misinformation about CrossFit affiliates has become a minor industry. And would we expect James Navalta, who already tolerated fraud about CrossFit, to suddenly discover integrity?

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: