Ex. 18.16

Bonferroni method for multiple comparisons. Suppose we are in a multiple-testing scenario with null hypotheses \(H_{0j}, j=1,2,...,M,\) and corresponding \(p\)-values \(p_j, j=1,2,...,M\). Let \(A\) be the event that at least one null hypothesis is falsely rejected, and let \(A_j\) be the event that the \(j\)th null hypothesis is falsely rejected. Suppose that we use the Bonferroni method, rejecting the \(j\)th null hypothesis if \(p_j < \alpha/M\).

(a) Show that \(\text{Pr}(A)\le \alpha\). [Hint: \(\text{Pr}(A_j\cup A_{j'})=\text{Pr}(A_j) + \text{Pr}(A_{j'}) - \text{Pr}(A_j\cap A_{j'})\)]

(b) If the hypothesess \(H_{0j}, j=1,2,...,M\), are independent, then \(\text{Pr}(A)=1-\text{Pr}(A^C)=1-\prod_{j=1}^M\text{Pr}(A_j^C)=1-(1-\alpha/M)^M\). Use this to show that \(\text{Pr}(A)\approx \alpha\) in this case.

Soln. 18.16

(a) We have

\[\begin{eqnarray} \text{Pr}(A) = \text{Pr}\left(\cup_{j=1}^M A_j\right) \le \sum_{j=1}^M\text{Pr}(A_j) = M \cdot \frac{\alpha}{M} = \alpha.\non \end{eqnarray}\]

(b) If follows directly from the fact that \((1-\alpha/M)^M\approx 1-\alpha\) when \(\alpha\) is small or \(M\) is large.