Start with a Problem: The Conceptual Framework
Sir Karl Popper famously stated is his book Conjectures and Refutations that science starts and ends with problems. “The conscious task of the scientist is always the solution of a problem through the construction of a theory which solves the problem.” This is an important concept to remember considering the current landscape in performance. Metrics, technology, dashboards, testing numbers, it’s a never-ending cycle. We are swimming in a sea of signal and noise. Perhaps the best place to start is to throw a life preserver and ask the rudimentary question: What problem(s) are we attempting to solve? Certainly, every situation is different as private and professional performance coaches have various constraints placed upon them (budget, facility, key decision makers, managers, etc.), but defining the problem is the first step in the problem-solving process.
The essence of science beautifully displayed by Karl Popper. So simple, it can be written on a napkin.
Problem Solving: The Conceptual Framework
What is it that you’d like to measure? How will you measure? Several books that I strongly recommend prior to constructing my own conceptual models were Measurement of Health and Health Status, Measurement and Medicine, and Methodological Thinking. All reads dive deep into testing philosophy, reliability, validity, and responsiveness in measure. Although these reads pertain more towards the fields of social science, psychology, and economics, they are imperative (in my opinion) in having a baseline understanding re: test construction.
Here are the ingredients:
A Conceptual Framework is simply a model representing the relationship between the items and the construct being measured. It’s an important step for coaches to pinpoint what it is they plan to measure, conceptualizing it while reducing researcher degrees of freedom. Degrees of freedom is the number of values which are free to vary. Too much flexibility (i.e., too many working definitions etc.) makes it difficult to obtain consistent results in the measurement process. It also leads to forking paths, and increasing the number of moving targets. Prior to creating a conceptual framework, I tend to ask myself the following questions:
What are the most relevant constructs for our population?
Can we measure the relevant constructs without overtaxing the athlete?
What variables are related to the construct?
Off the ice, can we measure the relevant constructs without dynamic skill acquisition (ie. heavy squatting or Oly lifts)?
Can we use reliable metrics with valid tools?
Can testing be training, and training be testing?
What resources do I have at my disposal?
Can we use the metrics for return to play baselines?
Does measurement matter? Are we measuring low hanging fruit and confusing it with the bountiful harvest?
How do we communicate the results to both athletes & coaches?
Here are a few examples of the Conceptual Frameworks:
Conceptual Framework for “Hockey Shape.”
Conceptual Framework for Stride Signature
Science starts and ends with problems. Don’t be fooled with solutions desperately seeking problems yet to take place. When dealing with measurement, whether latent or physically observable, a conceptual framework is an excellent place to start.
“Theories are hypotheses that cannot be verified but can be falsified.” -Popper
