Let's say I have two samples of results of two bernoulli experiments.H0:p1=p2H1:p1≠p2And I want to...
Let's say I have two samples of results of two bernoulli experiments.
And I want to try to reject at a confidence level.
I already know a proper way to solve this, but I was wondering, if I have a confidence interval for and , at the same level of significance. Can I just check if the intervals overlaps each other to test this?
Answer & Explanation
Suppose the confidence intervals have the property that with probability .
Then under the null hypothesis, with probability , so this is a lower bound for the probability that and intersect.
Thus if they don't intersect, you can reject with confidence level at most .
The true confidence level is presumably better than that, but without more analysis we don't know how much better.
It is true that if the intervals don't overlap, then there is a statistically significant difference. However, the converse is not true. Overlapping intervals do not imply that you cannot reject the null hypothesis.
The best approach to take is the one I'm assuming you've already done: Construct a confidence interval for the difference and check to see if it contains the point 0 or not.