The 6 problems of scientific proof

The 6 problems of scientific proof

Share on facebook
Share on google
Share on twitter
Share on linkedin

In this article I would like to address an important topic, namely, 'what is scientific proof'. Nowadays you (fortunately) see less and less bro-science and people use research to reinforce their arguments and points. This is a very good development. However, this comes with its own problems. Because can you approach all discussions and arguments in this way? And is this really the 'holy grail' when it comes to substantiating your vision?

This article is especially interesting for people who take their performance and results very seriously and who look beyond the usual approach. In addition, also for coaches, or people who have a coach. I hope you learn something from this!

Before we go any further, I am absolutely in favor of gaining knowledge from studies and research; no matter how bad some studies are, it remains better than the argument 'this is how it works for me, so this is how it works' or 'everyone is different'. Yes, everyone is different, but Pietje is not a brick and Henk is not a flying spaghetti monster; we are all human and even though there may be some variation in how the body reacts to certain stimuli, diet or supplements, no one arrives in a calorie deficit, no one has a higher output on fats than on carbohydrates, no one is allergic to carbs , etc.

Different types of evidence

Before we continue, it is good to make a distinction between Evidence Based and Research Based.

There are different kinds of 'evidence' for an opinion or an argument, each with its own characteristics.

Anecdotal evidence

At the top we have anecdotal evidence. This is the least concrete of all the forms of evidence here, but at the same time perhaps the most important of this triangle. Anecdotal evidence comes from curiosity; people who experiment and report their findings. The piece I just quoted "this is what works for me" falls below. When we gather a lot of this kind of anecdotal evidence from many different people, it can send us in a direction of perhaps something interesting; because this is not real research and people who are curious can provide this form of 'proof', this is the kind of evidence in which most discoveries are made. A lot will be nonsense, but if many people come to the same conclusion, such a topic might be interesting for further research. Without anecdotal evidence there would be no subjects for further investigation.

In addition, there is the well-known saying 'success leaves clues''

Logical proof

The next one we discuss is logical proof. This is the 'proof' that we can make ourselves, based on our knowledge. Being able to come to conclusions with logical thinking. If you know you have 2 and you get 2, you can argue that you will have 4. This form of proof is therefore based on examinations that have previously been carried out, experiences that have been made, etc.


Empirical evidence

The next step is empirical evidence, or in other words we have actual observational data from studies. There are many different ways to find or obtain data outside the lab. Investigations are very good to come to conclusions or discoveries, when they are set up well. However, one of the characteristics of good research is that the research group is sufficiently large, but at the same time this has the problem that small differences between test subjects are filtered, which may matter.

More and better empirical evidence is the basis for new and better logical evidence, because you have more data from which you can make arguments. In addition, more and better empirical evidence will also provide new anecdotal evidence, because we are being directed more in the right direction and are better able to ask and investigate the right questions.

The problem with empirical evidence is that it is sometimes extremely difficult to properly investigate certain subjects, especially when investigations have to be done on people. As we have mentioned before are always there confounders; matters that influence a result, without the researchers realizing that these things are present. Many of these confounders could be removed by regulating the study conditions as strictly as possible, but this is not possible in human studies; optimally, you would lock up the research groups, keeping light, sound, nutrition, stress, etc all as even as possible between the different research groups and allowing only 1 or a few factors to differ. However, this is never going to be approved by ethics committees, and in addition there is very little chance that people would sign up for this type of study.

Be objective

Evidence-based thinking requires that you consider all the evidence and have no preference or prejudice against parts of the complete picture. A follower of the carnival diet will probably not be completely open to proof that plants are necessary and healthy. A vegan will investigate that tell me that meat is not harmful or may even have benefits with a slanted eye. A good researcher (under that heading I would like to name everyone who is busy with optimizing results from research) tries to ensure that he has as few prejudices as possible and that all research and evidence weigh equally in his vision.

As you know, one research can contradict the other and few things are written in stone or applicable without any exception. That is the difficulty of being a good researcher or being a good coach; we cannot wait until research has given the definitive answer, because it can sometimes take years or decades; waiting and standing still all this time is not an option. You will have to collect all data together and base your approach or vision as well as possible on your own insight. Be creative and innovative.


The take-away of this story so far is that Evidence Based should not be limited to Research Based. Again, this does not mean that studies are not worth anything or important. Research (empirical evidence) is perhaps the most important evidence that we can provide, but it should not stop there. Interpret empirical evidence and continue from there!

Problems with relying solely on research

As I have described above, there are a number of problems with all your conclusions based on research, without having to think logically or try to interpret data. I want to go through these problems with you here.

# 1 Look back

When you base all your approaches and decisions on research, the only thing you can do is look back. You can only base your choices on studies that have already been done. This is going to stand in the way of innovation. In this way you will never be able to stand out from the competition and never do better than average, because you only base your approach on what someone else has already thought of and researched.

# 2 It limits progress

Good research is always limited to the anecdotal and logical evidence that exists; these 2 forms of evidence appeal to people's creativity; it will often happen that the wrong direction is thought of, but from these 2 forms of evidence hypotheses arise that can be tested with research. If no one is questioning the current investigation and everyone regards the current investigation as absolute truth, we are stuck. If everyone decided to only use approaches that are proven by research, in a few years' time we would be nothing or hardly any further in terms of knowledge than we are at the moment, because no new things are being tried.


# 3 Need limited financing

Scientific research is extremely slow; investigations must be well organized, funded, approved by committees, etc.

If this were our only form of burden of proof, development would be extremely slow. In addition, there is very little financial urge for universities to conduct research in the fitness and nutrition area, certainly when it comes to optimizing results. After all, there are no billions of companies who want to invest hundreds of millions, as is the case with the pharmaceutical industry.

# 4 Creativity is limited

Nowadays, many people are more concerned with the wrong of others, instead of getting further or getting the work field further. In forums and also in the coaching business you see that many people who try something new are completely burned down if that 'news' is not in line with what the current evidence says. As a result, fewer and fewer people dare to stick their necks out looking for the unknown, because the chance of losing their face is very high. However, as we have already seen, this can result in the complete fixation of our field.

However, that does not mean that we have to throw everything we know from proven concepts overboard; current evidence, both empirical and logical, should be the basis for new anecdotal evidence and curiosity.

# 5 Research is not always 100% relevant

As mentioned earlier, good research is extremely framed, because it ensures that the results of the research actually say and mean something, with the lowest possible chance that coincidence was the cause. However, this is also a problem; new research therefore always explores a very small piece of the unknown. In addition, physiology is extremely complex and there are always confounders as discussed before; no correction can be made for all of these confounders during an investigation, so that means that the circumstances in which a customer finds you / you may be different from where the test subjects found themselves. The same approach certainly does not always have the same outcome. Perhaps the test subjects slept more or had less stress. Perhaps they had more years of training experience, or certain injuries did not ..

# 6 Look at the context

Many people (if they already use literature to reinforce their arguments) only read the abstract of a study and draw all their conclusions from it. What she misses is that the study results must always be placed in context and results rarely if ever apply by definition to everyone, in all circumstances. A study that says that a squat is a better buttock exercise that a hipthrust is of little value if you don't have a clue as to the length of the subjects' thighs and lower legs, for example. Someone with long thighs will squat far more hip dominance and therefore show more buttock activity than someone with short thighs.

Articles are regularly pulled completely out of context and, in addition, in popular media often only one phrase is copied; this results in weird cups such as 'cold showers for more brown fat' or 'eating chocolate every day makes you lose weight'. Both headers have been in the newspaper for the last year. Both are completely drawn from context.

If not this, then what?

Again, this is absolutely no argument against using literature as the basis of your way of thinking or approach. However, that's where it should begin. Use that data and research to continue with logical reasoning and to try things out. There is more progress outside the lab than in the lab; studies can confirm suspicions, but rarely find completely new things. Most working approaches have been discovered by people who dare to try for themselves or coaches who try new approaches with their clients based on logical thinking.

Read as much as you can and want, but try out new things based on logical reasoning. Dare to be the first in something!

Maybe good advice for everything in life, even outside of sports and nutrition ...

This blog is written by

Jan Willem van der Klis

"My focus is on obtaining and disseminating the best possible knowledge to optimize training and nutrition"

Also interesting!

You achieve goals together!

Register and be the first to receive new discount promotions and information about new products!