In this paper, Stephens challenges the conventional approach to statistical hypothesis testing, which is prone to overestimation and unnecessary rejection of true null hypotheses. He proposes a new framework called false discovery rate (FDR), which controls the proportion of incorrectly rejected true null hypotheses. The FDR is calculated as the ratio of the number of false positives to the total number of hypothesis tests performed.
Stephens illustrates his concept using an analogy from finance, comparing the traditional approach to testing with a stockbroker who buys and sells randomly. In this scenario, the broker will inevitably make some correct predictions, but also many incorrect ones, leading to a high false discovery rate. Similarly, in statistical hypothesis testing, we may falsely reject a true null hypothesis due to random fluctuations in the data.
The author demonstrates that the FDR approach can significantly reduce this problem by controlling the family-wise error rate (FWER), which is the probability of making at least one type I error among all tests performed. By using the FDR, we can determine the optimal level of significance, which minimizes the expected loss due to false positives while maintaining a specified level of confidence in the null hypothesis.
Stephens also discusses the connection between the FDR and the Bayesian approach to statistical inference, showing that both frameworks share similar goals and methodologies. He provides examples of how the FDR can be applied in various fields, such as genetics, finance, and medical research.
In summary, Stephens’ paper introduces a new perspective on statistical hypothesis testing by focusing on the control of false discoveries rather than the traditional approach of controlling the family-wise error rate. The FDR framework offers a more realistic and practical approach to statistical inference, which can help avoid unnecessary rejections of true null hypotheses and provide more accurate results in scientific research.
Mathematics, Statistics Theory