Significance Level
The significance level is the criterion used for rejecting the Null hypothesis. Under the Null hypothesis the test statistic has some distribution. Under the Alternative hypothesis, the test statistic has some distribution that is shifted so that the test statistic is more likely to have smaller (larger) values. Assuming the Null hypothesis is true, the probability that the test statistic would take as large (small) or larger (smaller) value than that observed is determined. This probability is compared to the significance level. If the probability is less than or equal to the significance level, then the Null hypothesis is rejected and the outcome is said to be statistically significant. Traditionally, experimenters have used either the 0.05 level (sometimes called the 5% level) or the 0.01 level (1% level), although the choice of levels is largely subjective. The lower the significance level, the more the data must diverge from the Null hypothesis to be significant. Therefore, the 0.01 level is more conservative than the 0.05 level.