pingouin.multicomp#
- pingouin.multicomp(pvals, alpha=0.05, method='holm')[source]#
P-values correction for multiple comparisons.
- Parameters:
- pvalsarray_like
Uncorrected p-values.
- alphafloat
Significance level.
- methodstring
Method used for testing and adjustment of p-values. Can be either the full name or initial letters. Available methods are:
'bonf': one-step Bonferroni correction'sidak': one-step Sidak correction'holm': step-down method using Bonferroni adjustments'fdr_bh': Benjamini/Hochberg FDR correction'fdr_by': Benjamini/Yekutieli FDR correction'none': pass-through option (no correction applied)
- Returns:
- rejectarray, boolean
True for hypothesis that can be rejected for given alpha.
- pvals_correctedarray
P-values corrected for multiple testing.
Notes
This function is similar to the p.adjust R function.
The correction methods include the Bonferroni correction (
'bonf') in which the p-values are multiplied by the number of comparisons. Less conservative methods are also included such as Sidak (1967) ('sidak'), Holm (1979) ('holm'), Benjamini & Hochberg (1995) ('fdr_bh'), and Benjamini & Yekutieli (2001) ('fdr_by'), respectively.The first three methods are designed to give strong control of the family-wise error rate. Note that the Holm’s method is usually preferred. The
'fdr_bh'and'fdr_by'methods control the false discovery rate, i.e. the expected proportion of false discoveries amongst the rejected hypotheses. The false discovery rate is a less stringent condition than the family-wise error rate, so these methods are more powerful than the others.The Bonferroni [1] adjusted p-values are defined as:
\[\widetilde {p}_{{(i)}}= n \cdot p_{{(i)}}\]where \(n\) is the number of finite p-values (i.e. excluding NaN).
The Sidak [2] adjusted p-values are defined as:
\[\widetilde {p}_{{(i)}}= 1 - (1 - p_{{(i)}})^{n}\]The Holm [3] adjusted p-values are the running maximum of the sorted p-values divided by the corresponding increasing alpha level:
\[\widetilde {p}_{{(i)}}=\max _{{j\leq i}}\left\{(n-j+1)p_{{(j)}} \right\}_{{1}}\]The Benjamini–Hochberg procedure (BH step-up procedure, [4]) controls the false discovery rate (FDR) at level \(\alpha\). It works as follows:
1. For a given \(\alpha\), find the largest \(k\) such that \(P_{(k)}\leq \frac {k}{n}\alpha.\)
2. Reject the null hypothesis for all \(H_{(i)}\) for \(i = 1, \ldots, k\).
The BH procedure is valid when the \(n\) tests are independent, and also in various scenarios of dependence, but is not universally valid.
The Benjamini–Yekutieli procedure (BY, [5]) controls the FDR under arbitrary dependence assumptions. This refinement modifies the threshold and finds the largest \(k\) such that:
\[P_{(k)} \leq \frac{k}{n \cdot c(n)} \alpha\]References
[1]Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.
[2]Šidák, Z. K. (1967). “Rectangular Confidence Regions for the Means of Multivariate Normal Distributions”. Journal of the American Statistical Association. 62 (318): 626–633.
[3]Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.
[4]Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57, 289–300.
[5]Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29, 1165–1188.
Examples
FDR correction of an array of p-values
>>> import pingouin as pg >>> pvals = [.50, .003, .32, .054, .0003] >>> reject, pvals_corr = pg.multicomp(pvals, method='fdr_bh') >>> print(reject, pvals_corr) [False True False False True] [0.5 0.0075 0.4 0.09 0.0015]
Holm correction with missing values
>>> import numpy as np >>> pvals[2] = np.nan >>> reject, pvals_corr = pg.multicomp(pvals, method='holm') >>> print(reject, pvals_corr) [False True False False True] [0.5 0.009 nan 0.108 0.0012]