Steps

Next: Comparisons Up: NHST Previous: Data Contents

Steps of NHST

NHST Step 1

Identify the populations and restate the research question as $H_0$ and $H_1$
Population 1: UNT undergraduates who are presented words with red font.
Population 2: UNT undergraduates who are presented words with green font.
Population 3: UNT undergraduates who are presented words with blue font.
Null Hypothesis: $H_0: \mu_1 = \mu_2 = \mu_3$
Alternative Hypothesis:
- Note: The Null Hypothesis could be rejected under several conditions.
- Any significant difference among the three means.

NHST Step 2

Determine the characteristics of the comparison distribution.
The comparison distribution is the distribution with 2 and 9 degrees of freedom:
- $df_{b} = k - 1 = 3 - 1 = 2$
- $df_{w} = n_{t} - k = 12 - 3 = 9$

NHST Step 3

Determine the cutoff sample score on the comparison distribution at which $H_0$ should be rejected.
Using the $F$ distribution with a significance level of 0.05; and 2 (numerator) and 9 (denominator) degrees of freedom: $F_{crit} = 4.26$

http://faculty.vassar.edu/lowry/apx_d.html

NHST Step 4

Determine your sample's score on the comparison distribution.
- Compute your sample statistic(s).
Calculate $SOS$ , $df$ , $MS$ , and $F_{calc}$ for Between groups, Within groups, and Total (as a check of the other calculations) given the information provided above.

Calculate $SOS_{b}$ and $df_{b}$

Sum of the $n$ of each group times the squared deviations of each group mean from the grand mean.

$SOS_{b} = \sum{\left[n_j\left(\overline{X}_{j} - \overline{X}_{..}\right)^2\right]}$

$SOS_{b} = n_1\left(\overline{X}_1 - \overline{X}_{..}\right)^2 + n_2\left(\ove... ...erline{X}_{..}\right)^2 + n_3\left(\overline{X}_3 - \overline{X}_{..}\right)^2$

$SOS_{b} = 4\left(22.75 - 17.25\right)^2 + 4\left(11.5 - 17.25\right)^2 + 4\left(17.5 - 17.25\right)^2 = 253.50$

$df_{b} = k - 1 = 3 - 1 = 2$

Calculate $MS_{b}$

Calculation of Mean-Square between, an estimate of the between groups variance:

$MS_{b} = \frac{SOS_{b}}{df_{b}}$

$MS_{b} = \frac{253.50}{2}$

$MS_{b} = 126.75$

Calculate $SOS_{w}$ and $df_{w}$

The sum of the squared deviations of each group's score from its group mean (complete calculations available in the handout).

$SOS_{w} = \sum{\left(X_{ij} - \overline{X}_{j}\right)^2}$

$SOS_{w} = \left(X_{11} - \overline{X}_{1}\right)^2 + \left(X_{21} - \overline{... ..._{41} - \overline{X}_{1}\right)^2 + \left(X_{12} - \overline{X}_{2}\right)^2 +$ ... $+ \left(X_{43} - \overline{X}_{3}\right)^2$

$SOS_{w} = \left(25 - 22.75\right)^2 + \left(22 - 22.75\right)^2 + \left(23 - 22.75\right)^2 + \left(21 - 22.75\right)^2 + \left(10 - 11.5\right)^2 +$ ... $+ \left(16 - 17.5\right)^2 = 18.75$

$df_{w} = n_{t} - k = 12 - 3 = 9$

Calculate $MS_{w}$

Calculation of Mean-Square within (often called Mean-Square error), an estimate of the within groups variance:

$MS_{w} = \frac{SOS_{w}}{df_{w}}$

$MS_{w} = \frac{18.75}{9}$

$MS_{w} = 2.083$

Calculate $SOS_{t}$ and $df_{t}$

The sum of the squared deviations of each score from the grand mean (complete calculations available in the handout).

$SOS_{t} = \sum{\left(X_{ij} - \overline{X}_{..}\right)^2}$

$SOS_{t} = \left(X_{11} - \overline{X}_{..}\right)^2 + \left(X_{21} - \overline... ...41} - \overline{X}_{..}\right)^2 + \left(X_{12} - \overline{X}_{..}\right)^2 +$ ... $+ \left(X_{43} - \overline{X}_{..}\right)^2$

$SOS_{t} = \left(25 - 17.25\right)^2 + \left(22 - 17.25\right)^2 + \left(23 - 17.25\right)^2 + \left(21 - 17.25\right)^2 + \left(10 - 17.25\right)^2 +$ ... $\left(16 - 17.25\right)^2 = 272.25$

$df_{t} = n_{t} - 1 = 12 - 1 = 11$

Build the ANOVA Summary Table and Calculate $F$

Table 2: NHST Example ANOVA Summary Table

Source	$SOS$	$df$	$MS$	$F_{calc}$
Between	253.50	2	126.75	60.84
Within	18.75	9	2.083
Total	272.25	11

Recall, $SOS / df = MS$ and $MS_{b} / MS_{w} = F_{calc}$

$F_{calc} = \frac{MS_{b}}{MS_{w}} = \frac{126.75}{2.083} = 60.84$

Notice that $SOS_{b} + SOS_{w} = SOS_{t}$ and
$df_{b} + df_{w} = df_{t}$

NHST Step 5

Compare and make a decision.

Decide whether to reject $H_0$
Because $F_{calc} = 60.84 > 4.26 = F_{crit}$

We would reject the null hypothesis and conclude that there was a significant difference among the sample means. This study suggests that there is a difference among the three theoretical populations (each represented by a sample here).
However, we do not yet know where the significant difference is located, and there may be more than one significant difference among the three groups.

What's next?

The one-way ANOVA only tells us if there was some significant difference among the groups.
Two possible ways of identifying where the significant difference(-s) lie.
Planned Comparisons, which are planned prior to data collection (when designing the study).
- Used to assess specific hypotheses about specific groups (e.g., Red $>$ Blue; Green $<$ Blue, etc.).
Post-hoc testing, which typically compares all possible pair-wise comparisons of the groups.
- Traditionally, only done when the `omnibus' $F$ is significant.
- Reality; almost always done to explore pair-wise differences.

Next: Comparisons Up: NHST Previous: Data Contents

jds0282 2010-10-21