Inferences About a Single Mean: Chapter 8
Defining Expected and Unexpected Results
- The "Alpha" level
- Conducting a z-test, we used +/- 1.96 as the critical values for our decision to reject or not reject null
- When these are the critical values, there is only a 5% chance that any mean you get will be beyond those values, if it is a mean that comes from that population.
- We call this 5% probability our "alpha level" and this means we set our "alpha level" (
) at .05 [Use Greek because we are referring to a sampling dist. of our mean.]
Failing to Reject Versus Accepting the Null Hypothesis
- Notice, in our example that we "rejected the null hypothesis"
- We’re saying that it’s unlikely that the null hypothesis is true, so we reject it.
did not "statistically differ" from 
? Do we "accept the null hypothesis?" NO -- We can’t say that the population mean associated with our sample is exactly equal to some value.
- Instead, we state that we "do not reject" the null hypothesis – we don’t completely accept it as true, but given our probabilistic evidence, we’re also not willing to reject it. "In all probability, there is no difference."
Our decision is always stated in reference to null – reject or not reject.
Type I and Type II Errors
- Two potential "errors" in our decision:
- Type I errors: Reject null, when we shouldn’t have
- Here we conclude that there is a difference, when in reality there is no real difference (the difference was simply due to sampling error).
- Type II errors: Not rejecting null, when we should have
- Here we conclude there is not a difference when in reality there is a difference
- If we decrease our alpha level really low, we increase the chance of making a Type II error.
|
|
H0 is really true |
H0 is really false |
|
We reject H0 |
Type I error ( |
Correct Decision (1 - |
|
We do not reject H0 |
Correct Decision (1 - |
Type II error ( |
Effects of Alpha and Sample Size on the Power of Statistical Tests
- The "power" of a test is defined as the probability that a researcher will correctly reject null.
- In other words, it’s the probability of NOT making a Type II error (
).
- Two main factors influence our level of "power"
- Our alpha level:
- As we increase alpha (i.e., lower our cut off from .05 to .10), we increase power. (But also increase your probability of making a Type I error.) Makes it easier to reject null.
- As we decrease alpha (i.e., raise our cut off from .05 to .01), we decrease power. (And increase chances of Type II error.) Makes it more difficult to reject null.
- Sample size (N):
- As N gets larger, the test becomes more powerful, because
- Conceptually, a larger N decreases the amount of sampling error in our 
, so we can be more confident in our judgments about whether or not to reject null.
Statistical and Real-World Significance
- When we do reject null we say there is a "statistically significant" difference between and
- "Statistical significance" tells us only that any difference we observed is not due to sampling error – it doesn’t tell us how large or meaningful the difference is.
)
) = "power"