Research: defining what is meaningful

In science, what makes it possible to say that a result is "significant"? Initiates will evoke the p-value, but a multidisciplinary team rekindles an old controversy: P-value, they say, has its day.

Traditionally, this value is 0.05 or 5%. It is the "gold standard" of a significant result. For example, if only one of two groups with the same disease received the drug to be tested while the other group received a placebo, the improvement in the health status of the first group should differ On the other by more than 5% to be decreed significant.

One of the problems is that 5% is an easy threshold to cross, as long as other factors have played into the equation - lifestyle change, too small group, and so on. In psychology, we have had to admit in recent years that we have abused the p-value, resulting in a "crisis of reproducibility": a lot of research has been published without having demonstrated that their results were "significant" .

In scholarly terms, the p-value is said to define "the probability of obtaining the same value if the null hypothesis were true": in other words, if the two populations were equal, if the effects of the drug and placebo were equal ... In short, if there was nothing to report in the experiment and if the results were due to chance. The authors who want to reform the p-value propose to increase it from 0.05 to 0.005 (5% to 0.5%).

These 72 authors, who published their proposal on July 22, which is soon to be published in the journal Nature Human Behavior, include statisticians, psychologists and biologists. They revive an old debate: the threshold at which physicists decide to publish is, for a long time, much more severe than that used in biomedical sciences. Genomics - genome analysis - has already taken this turn, made necessary by the astronomical amount of data with which it must juggle.

Consequences of a reform

The subtleties of the p-value are less important to understand than its consequence: if the views of these authors prevail, it will be much more difficult to use the words "significant results". A search that reaches the threshold of 0.5 would simply be called suggestive. Preliminary research would gain less visibility.

While magazines may continue to publish what they want according to their editorial lines, a common standard across disciplines would make it clearer, including in university press releases, which are particularly quick to "Significant" preliminary research from one of their researchers. And a lot of journalistic reports made to the stick have a starting point for a press release.

The pharmaceutical companies are likely to oppose this, because a more stringent approach will mean higher costs: the average number of participants in clinical trials will have to be increased. On this side, everything remains to be evaluated.

But the comments that accompany this proposal are many to point out that British statistician Ronald Fisher, the "inventor" of p-value, had never imagined that it would become a determining element of what defines a "proof" in science.

Already, an article signed by "dozens" of experts from several disciplines is being written, to refute point by point the arguments put on the table. Basically, they reproach the 72 authors for wanting to put a brake on scientific research, on the pretext of wanting to be too careful. Attention to the concept of "statistically significant," they say, is legitimate, but this concept can not be the only burden that science has to bear at this time. The "culture" that drives researchers to publish more and more and faster, weighs much more heavily.