 dotzis16xd

2022-03-24

Let's say I have two samples of results of two bernoulli experiments.
${H}_{0}:{p}_{1}={p}_{2}$
${H}_{1}:{p}_{1}\ne {p}_{2}$
And I want to try to reject ${H}_{0}$ at a confidence level.
I already know a proper way to solve this, but I was wondering, if I have a confidence interval for ${p}_{1}$ and ${p}_{2}$, at the same level of significance. Can I just check if the intervals overlaps each other to test this? Korbin Ochoa

Expert

Step 1
Suppose the confidence intervals ${I}_{j}$ have the property that $p\in {I}_{j}$ with probability $\ge 1-\alpha$.
Then under the null hypothesis, $p\in {I}_{1}\cap {I}_{2}$ with probability $\ge {\left(1-\alpha \right)}^{2}=1-2\alpha +{\alpha }^{2}$, so this is a lower bound for the probability that ${I}_{1}$ and ${I}_{2}$ intersect.
Thus if they don't intersect, you can reject ${H}_{0}$ with confidence level at most $a\alpha -{\alpha }^{2}$.
The true confidence level is presumably better than that, but without more analysis we don't know how much better. Abdullah Avery

Expert

It is true that if the intervals don't overlap, then there is a statistically significant difference. However, the converse is not true. Overlapping intervals do not imply that you cannot reject the null hypothesis.
The best approach to take is the one I'm assuming you've already done: Construct a confidence interval for the difference ${p}_{1}-{p}_{2}$ and check to see if it contains the point 0 or not.

Do you have a similar question?