How to interpret a too small chi-square &#x03C7;<!-- χ --> 2 </msup> value? I

excluderho

excluderho

Answered question

2022-06-12

How to interpret a too small chi-square χ 2 value?
I use the Chi-square (i.e., χ 2 ) goodness-of-fit test to measure the distance between chunks of data and a theoretical distribution.
For most of the data being tested the results make sense. I got from time to time unusual results, which correspond to very small values (i.e., between 10 and 100). The sample size is not the problem, there is at least 5 elements for each symbol.
After a careful analysis of the observed data, the frequency of each symbol in the data being tested is very close to the theoretical distribution as the χ 2 value suggest.
But it seems to be too perfect for me. It is unlikely according to the chi-square distribution function that a given value would occur (i.e., z 0.0001 ).
Should I reject the null hypothesis because the data is too good to be true ?
I did not find clear explanations or I missed something.
How to interpret unlikely values in the left tail of the chi-square distribution ?

Answer & Explanation

mar1nerne

mar1nerne

Beginner2022-06-13Added 20 answers

Usually chi-squared tests of goodness-of-fit are one-sided (because the squaring involved in computing the test statistic gives both negative and positive differences the effect of increasing the statistic). Thus one rejects the null hypothesis (data fit the model) if the test statistic is larger than some critical value. (P-value is small.) The test statistic is never negative.
However, these rules do not apply when vetting a pseudorandom number generator to be used in probability simulation because a fit that is "too good to be true" (test statistic near 0, P-value near 1) indicates the generator is giving nonrandom values as much as does a large value of the test statistic.
There are also cases, such as the famous one in @Henry's Comment, in which data fit a model "too closely" and the procedure of data collection or tabulation comes into doubt. If you ask someone to check whether a die is fair by rolling it 600 times, and the answer comes back that each of the six faces showed exactly 100 times, you would be entitled to wonder whether the 600-roll experiment was done faithfully.
"When the P-value is very small, doubt the null hypothesis; when the P-value is very near 1, doubt the model or the data collection."
Note: Interpretation of goodness-of-fit (GOF) tests is often incorrect. In a two-sample test whether Drug A is better than a placebo, the experimenter may be wishing for a rejection of H 0 . (drug has no effect). However, in GOF tests the experimenter is often hoping not to reject H 0 .. This means that it is especially important to assess the power of a GOF test.

Do you have a similar question?

Recalculate according to your conditions!

Ask your question.
Get an expert answer.

Let our experts help you. Answer in as fast as 15 minutes.

Didn't find what you were looking for?