Show me the Evidence
Last week’s blog inspired a long discussion based on a quote from a friend re: using advanced training means on developmental athletes early in the off-season. The quote states: “It cheats the athlete twice. Firstly, using advanced means when a basic mean can create similar adaptation, and second, when it’s time to use the advanced mean, it’s no longer a new stimulus to the athlete.” A long Twitter debate ensued on the subject matter of using advanced training means for developmental athletes and and potential compromised adaptation in the future. Inevitably someone essentially said, “show me the evidence.” This got me thinking. Where would we find such evidence? Who would run a study like this? What kind of evidence do we need as practitioners? Does it need to be published to be “proven?” At its essence, what is science?
From my experience in both in the pragmatic setting, and at the graduate level in academia, “evidence” is not the same. I’ve written more about this in a previous blog post. As coaches we perform observational research. Very similar to Texas Sharpshooters. We look for clues (bullet holes), observe trends, look for patterns, and hypothesize (circle the target). We also perform N=1 research on individuals within the group/team. The most important data we can pull from is longitudinal data from our unique sample. Sampling bias is a major issue in all forms of academic research, so for the performance coach the gold standard is the team, our athletes, our individuals, and our data. This is what drives decision making.
Academic research in sports science, in my opinion, is not the gold standard for evidence for most practitioners but it is important. The goal is deductive in nature (hypothesize – collect data - infer). The internal and external validity of performing work in a lab while attempting to control for confounders is extremely different than the weight room, the pitch, or on the ice. The two worlds are not the same, and most certainly not the athletes you’re working with. This is not ceteris paribus, it’s more like apples to oranges. Finally, just because something is published, doesn’t mean it’s “proven.” There is a replication crisis happening today in scientific fields such as molecular biology and traditional epidemiology with much more scrutiny than sports science. John Ionnidis’s article titled “Why Most Published Research Findings are False” is an interesting read. He states: “for many current scientific fields, claimed research findings may simply be accurate measures of the prevailing bias.” This may be due to small sample size, poor power, and smaller effect size. All issues plaguing sports science.
Science is not about truth, it’s about conjecture and refutation. It’s about getting closer to the approximation of “truth” through a temporary theory/hypothesis. Temporary theories are superior in two ways: 1) they provide superior explanation, 2) they are tested more frequently. So, what is the evidence? Is it a p of <.05? Is it a large effect size? Is it a great study design? Is it because the results were published in a reputable journal? I don’t claim to know the answers to these questions, but the first place I’d start is with the athletes I train, in the environment in which they play.
