Effect Size

Next: CI Up: NHST Previous: Post-hoc Contents

Effect Sizes

Effect Size Measures for ANOVA

The ANOVA family of analyses have a variety of effect size measures which can be used.
Recall, there are generally two forms of effect size measures;
- Measures of shared variance such as $R^2$ and $R_{adj.}^2$
- Measures of standardized difference such as Cohen's $d$
Well, it turns out that ANOVA is a member of the General Linear Model (GLM) and therefore, both types of effect size can be utilized in the ANOVA situation.
- GLM will be discussed in more detail in future modules (e.g., regression) but, suffice to say; we have specified a linear model here, which generally refers to a linear relationship between the variables we are analyzing.

Available Effect Size Measures

Cohen's can be computed during individual comparisons (planned comparisons or post-hoc testing).
- In those situations we are essentially comparing two means (similar to the t tests).
The most primitive (slightly biased) effect size used with one-way ANOVA is eta-squared ( $\eta^2$ ).
Another effect size for ANOVA is $R^2$ which we encountered in an earlier module and know it overestimates.
Partial eta-squared is a popular effect size for ANOVA; although it is generally reserved for more complex designs than the one-way situation.
- Same as $\eta^2$ in the one-way situation.
The recommended effect size, for the one-way situation and more complex situations, is Omega squared ()
- Omega squared is virtually unbiased.

Eta squared

Eta squared ( $\eta^2$ ) is very easy to calculate, but as mentioned above it is biased.

Source	$SOS$	$df$	$MS$	$F$
Between	253.50	2	126.75	60.84
Within	18.750	9	2.083
Total	272.25	11

$\eta^2 = \frac{SOS_{b}}{SOS_{t}} = \frac{253.50}{272.25} = .9311$

Which suggests 93.11% of the variance in words recalled (Dependent variable [DV])is attributable to the differences in font color (Indpendent variable [IV]).

$R^2$ for ANOVA

There are two versions of the $R^2$ formula for ANOVA, the one below is used in the one-way situation:

$R^2 = \frac{\left(F\right)\left(df_{b}\right)}{\left(F\right)\left(df_{b}\righ... ...{60.84\left(2\right)}{60.84\left(2\right) + 9} = \frac{121.68}{130.68} = .9311$

No surprise here, 93.11% of the variance in words recalled (DV)is attributable to the differences in font color (IV).
- This is the same effect size we got for $\eta^2$

Omega Squared

Omega squared ( $\omega^2$ ) is the recommended effect size measure because, it takes into account the number of groups (less biased than the ones above).

$\omega^2 = \frac{SOS_{b}\ -\ \left(k\ -\ 1\right)MS_{w}}{SOS_{t}\ +\ MS_{w}} =... ... \left(3\ -\ 1\right)2.083}{272.25\ +\ 2.083} = \frac{249.332}{274.33} = .9089$

Here we see a less biased, more accurate representation of the relationship between the IV and the DV.
90.89% of the variance in words recalled (DV)is attributable to the differences in font color (IV).
Please note; Omega squared should also be calculated on the results of Planned Comparisons if they were performed (examples below).

Omega squared for planned comparisons

For Planned comparison 1 from earlier:

$\omega^2 = \frac{SOS_{b}\ -\ \left(k\ -\ 1\right)MS_{w}}{SOS_{t}\ +\ MS_{w}} =... ...\ \left(2\ -\ 1\right)2.083}{272.25\ +\ 2.083} = \frac{53.037}{274.33} = .1933$

So, 19.33% of the variance in words recalled is attributable to the difference between Red and Blue font color.
For Planned Comparison 2 from earlier:

$\omega^2 = \frac{SOS_{b}\ -\ \left(k\ -\ 1\right)MS_{w}}{SOS_{t}\ +\ MS_{w}} =... ...\ \left(2\ -\ 1\right)2.083}{272.25\ +\ 2.083} = \frac{69.917}{274.33} = .2549$

So, 25.49% of the variance in words recalled is attributable to the difference between Green and Blue font color.

Next: CI Up: NHST Previous: Post-hoc Contents

jds0282 2010-10-21