1                        TESTING SIMPLE EFFECTS IN MANOVA
   2 
   3                                David P. Nichols
   4                          Senior Support Statistician
   5                                   SPSS, Inc.
   6                          From SPSS Keywords, May 1993
   7 
   8 
   9 Factorial designs in analysis of variance and covariance, including designs
  10 with within subjects factors, are very common in many fields of research.
  11 The SPSS MANOVA procedure provides a powerful and flexible set of tools for
  12 performing most of the analyses that are available under the general linear
  13 model framework. A very common problem is that of an experiment in which
  14 interactions have been found and the researcher wants to explore the data
  15 more carefully to determine what statements may be made about main effects
  16 or interactions in the presence of the two-way or higher order interaction
  17 effects. Tests of such simple main effects or simple interaction effects
  18 are generally easily handled in MANOVA through the flexibility in model
  19 specification offered by the DESIGN and WSDESIGN subcommands.
  20 
  21 
  22                    Two-way Between Subjects Models:
  23                     Estimating Simple Main Effects
  24                          
  25 
  26 Let's begin with the simplest case in which we might want to test for
  27 simple effects: a two-way factorial design in which we have found an
  28 interaction effect. If the two factors are both between subjects factors
  29 and have two and three levels respectively, we might have the following
  30 syntax for the factorial analysis:
  31 
  32 MANOVA Y BY A(1,2) B(1,3).
  33 
  34 This one line of syntax will produce the full factorial analysis (MANOVA
  35 always does a full factorial model by default), equivalent to specifying
  36 either
  37 
  38 MANOVA Y BY A(1,2) B(1,3)
  39  /DESIGN
  40 
  41 or
  42 
  43 MANOVA Y BY A(1,2) B(1,3)
  44  /DESIGN=A, B, A BY B.
  45 
  46 If the A by B interaction term is nonzero, the effects of the two factors
  47 A and B are not the same across all levels of the other factor. That is,
  48 it is possible for A to have a positive effect on the dependent variable
  49 at one level of B, no effect at another level and a negative effect at a
  50 third level. Such a situation might lead to an overall main effects test
  51 for A in which no evidence of any A effect was discovered. This is because
  52 the effect of A is confounded with the A by B interaction effect. It is
  53 also possible that A has a positive (or negative) effect at each level of
  54 B, but that this effect is stronger at some levels of B than at others.
  55 In this case it does make sense to talk about an overall positive (or
  56 negative) main effect for factor A, but discussion of the magnitude of
  57 this effect must be conditioned on the particular levels of the B factor
  58 within which these effects do not differ.
  59 
  60 In each of these cases what is called for is to examine the effects of the
  61 A factor separately within each level of the B factor. These effects are
  62 what are known as simple main effects. Specification of such effects in
  63 MANOVA is simple, following a logical algorithm applied to our model
  64 specifications on the DESIGN subcommand. The general algorithm is as
  65 follows: To obtain the proper simple effects estimates and tests of one
  66 factor at (within) each level of a second factor, replace the main effect
  67 of the factor of interest and the two-way interaction involving these two
  68 factors with the simple effects of the factor of interest within each 
  69 level of the other factor. For our example, we would replace the main 
  70 effect of factor A and the A by B interaction with the simple effects of
  71 factor A at each level of factor B:
  72 
  73 MANOVA Y BY A(1,2) B(1,3)
  74  /DESIGN=B,
  75          A W B(1), A W B(2), A W B(3)
  76 
  77 where the W operator is an acceptable shorthand for the WITHIN keyword.
  78 
  79 Important notes to keep in mind here are the following: We have simply
  80 removed the main effect of the A factor and the A by B interaction term
  81 from a full factorial specification and have replaced them with a request
  82 for the simple effects of A within (separately for) each level of factor
  83 B. The effect of this substitution is to repartition the same overall model
  84 into different effects, but to maintain the same total model (total degrees
  85 of freedom, total sums of squares accounted for, same predicted values and
  86 residuals, etc). That is, we are estimating the same B main effect as in
  87 the original full factorial model, and repartitioning the A main effect and
  88 the A by B interaction effect into the simple main effects of A at each
  89 level of B.
  90 
  91 This is important to note for two reasons. First, when working with data
  92 with unequal numbers of observations in the cells of the design (generally
  93 referred to as unbalanced data), the sums of squares for a particular
  94 effect such as A W B(1) will generally not be the same when specified alone
  95 on the DESIGN subcommand as when specified as part of a larger model, due
  96 to the intercorrelation among the factors in an unbalanced design. The
  97 algorithm outlined here is designed to maintain the same overall model
  98 throughout the testing of simple effects so that the simple effects
  99 estimated are logical followups to the results of the overall full factorial
 100 analysis. Second, in releases beginning with version 5.0 the default error
 101 term in MANOVA has been changed from WITHIN CELLS to WITHIN+RESIDUAL. Thus
 102 even in balanced designs, the error term and degrees of freedom used for
 103 testing simple effects would not be the same as in the original analysis
 104 unless the same overall model was estimated or unless the user explicitly
 105 specified /ERROR=WITHIN on the ERROR subcommand.
 106 
 107 So far we have talked only in terms of the simple main effects of A at
 108 each level of B. However, the implications of an interaction effect are
 109 completely symmetric. That is, to say that the effects of factor A are
 110 different at different levels of factor B is equivalent to saying that
 111 the effects of factor B are different at different levels of factor A.
 112 Thus we would probably also want to test the simple main effects of B at
 113 each level of A. To do this we would simply follow the same algorithm,
 114 reversing the role of factor A and factor B. That is, we remove the main
 115 effect of factor B from the full factorial specification, along with the
 116 A by B interaction and substitute the simple main effects of B at each
 117 level of A. Our syntax would thus be:
 118 
 119 MANOVA Y BY A(1,2) B(1,3)
 120  /DESIGN=A,
 121          B W A(1), B W A(2).
 122 
 123 One important point to note is that since the A and B simple main effects
 124 each involve a repartitioning of the interaction term, attempting to fit
 125 both sets of simple main effects on one DESIGN subcommand would introduce
 126 redundant effects and should thus be avoided. Estimation of both sets of
 127 simple main effects in one MANOVA run can be accomplished simply by stacking
 128 two DESIGN subcommands:
 129 
 130 MANOVA Y BY A(1,2) B(1,3)
 131  /DESIGN=B, A W B(1), A W B(2), A W B(3)
 132  /DESIGN=A, B W A(1), B W A(2).
 133 
 134 
 135                    General Between Subjects Models:
 136                     Estimating Simple Main Effects
 137 
 138 
 139 The algorithm outlined above generalizes immediately to cases of higher
 140 order designs. Let's illustrate with the case of a three-way design, with
 141 factors A, B and C. For the sake of brevity we will assume that each
 142 factor has only two levels, since there is no loss of generality in our
 143 discussion and this saves us from writing out more terms in our DESIGN
 144 specifications.
 145 
 146 If in a higher order design we wish to estimate simple main effects, the
 147 procedure is exactly that outlined above, except that we would have other
 148 terms also listed on the DESIGN subcommand. For example, in an A by B by C
 149 design in which we wanted to estimate the simple effects of A at each 
 150 level of B, we would perform the same replacement of main and interaction
 151 effect terms as before, but would maintain the model specifications
 152 involving the C factor. Thus our full factorial syntax
 153 
 154 MANOVA Y BY A B C(1,2)
 155  /DESIGN=A, B, C, A BY B, A BY C, B BY C, A BY B BY C
 156 
 157 would become
 158 
 159 MANOVA Y BY A B C(1,2)
 160  /DESIGN=B, C, A BY C, B BY C, A BY B BY C,
 161          A W B(1), A W B(2).
 162 
 163 Since we are using UNIQUE or regression approach sums of squares, the
 164 order of effects specified makes no difference, assuming that each cell
 165 of the design contains at least one observation (designs involving empty
 166 cells are much more complicated and require careful special handling).
 167 
 168 Many statisticians might object to the foregoing simple effects tests
 169 because they are being conducted in a model in which a higher order
 170 interaction is being estimated which contains the effects in question. 
 171 The logic behind this objection would be that first we should test the
 172 three-way interaction. If this is significant we should then proceed to
 173 test simple, simple main effects and/or simple interaction effects. If
 174 it is not significant, remove the three-way interaction and re-estimate
 175 the model. That is, follow-up tests on simple effects should not be
 176 performed until a final model has been chosen. The algorithm outlined
 177 here is not affected by this approach. We would have first re-estimated
 178 the model without a three-way interaction term, as
 179 
 180 MANOVA Y BY A B C(1,2)
 181  /DESIGN=A, B, C, A BY B, A BY C, B BY C
 182 
 183 and the same substitutions would apply, resulting in
 184 
 185 MANOVA Y BY A B C(1,2)
 186  /DESIGN=B, C, A BY C, B BY C,
 187          A W B(1), A W B(2).
 188 
 189 Others would consider this approach somewhat rigid. That is, though an
 190 interaction effect in a sample was not of sufficient magnitude to provide
 191 evidence at (say) the .05 alpha level of an interaction effect in the
 192 population, the assumption of no interaction effect as opposed to a small
 193 one might be presumptuous. Thus another strategy would be to fit the
 194 simple effects in the context of the overall factorial model, estimating
 195 them in the presence of the estimated questionable interaction effects.
 196 Each user is responsible for coming to her or his own conclusions as to
 197 what procedures should be followed in this case; MANOVA can be made to
 198 analyze the data in either case.
 199 
 200 
 201                    General Between Subjects Models:
 202                 Estimating Simple, Simple Main Effects
 203                     and Simple Interaction Effects
 204 
 205 
 206 If the three-way interaction had been significant in the above model, we
 207 would be faced with a more complicated situation. That is, not only do the
 208 effects of factor A depend on which level of factor B we consider, but
 209 they also depend on the level of factor C in which our A by B designation
 210 of interest is found. The logical step at this point is to examine the
 211 two-way interactions at each level of the third factor (such as A by B
 212 within each level of C) to see if within each level of the third factor
 213 the main effects of the other two factors are invariant. Generalization
 214 of the algorithm discussed in the two-way case results in the A by B and
 215 A by B by C interactions being replaced by the simple interaction effects
 216 of A by B at each level of C. Thus
 217 
 218 MANOVA Y BY A B C(1,2)
 219  /DESIGN=A, B, C, A BY B, A BY C, B BY C, A BY B BY C
 220 
 221 becomes
 222 
 223 MANOVA Y BY A B C(1,2)
 224  /DESIGN=A, B, C, A BY C, B BY C,
 225          A BY B W C(1), A BY B W C(2).
 226 
 227 If the simple interaction effects are nonzero, the next step is to 
 228 estimate the simple, simple main effects of say, factor A at each level
 229 of the two-way breakdown of factors B and C. The simple, simple effects of
 230 A at each level of factors B and C involve a repartitioning of the A main
 231 effect, the A by B, A by C and A by B by C interactions:
 232 
 233 MANOVA Y BY A B C(1,2)
 234  /DESIGN=A, B, C, A BY B, A BY C, B BY C, A BY B BY C
 235 
 236 becomes 
 237 
 238 MANOVA Y BY A B C(1,2)
 239  /DESIGN=B, C, B BY C, 
 240          A W B(1) BY C(1), A W B(1) BY C(2),
 241          A W B(2) BY C(1), A W B(2) BY C(2).
 242 
 243 An equivalent specification would be
 244 
 245 MANOVA Y BY A B C(1,2)
 246  /DESIGN=B, C, B BY C, 
 247          A W B(1) W C(1), A W B(1) W C(2),
 248          A W B(2) W C(1), A W B(2) W C(2).
 249 
 250 As with the more simple two-way case, the factors here are perfectly
 251 symmetric, so we could just as sensibly be using B or C in place of A.
 252 Also, the substitution rules used here generalize to designs with any
 253 number of factors.
 254 
 255 
 256                Models Involving Within Subjects Effects
 257 
 258 
 259 As most users are aware, MANOVA offers the capability of using the
 260 multivariate approach to analyzing data involving within subjects (often
 261 involving repeated measures) effects. The within subjects part of the
 262 model is specified separately from the between subjects part, but in an
 263 analogous manner, via the WSDESIGN subcommand. Thus a two-way completely
 264 within subjects design involving two two-level factors A and B could be
 265 specified as:
 266 
 267 MANOVA V1 TO V4
 268  /WSFACTORS=A(2) B(2)
 269 
 270 which would be the same as
 271 
 272 MANOVA V1 TO V4
 273  /WSFACTORS=A(2) B(2)
 274  /WSDESIGN
 275 
 276 or
 277 
 278 MANOVA V1 TO V4
 279  /WSFACTORS=A(2) B(2)
 280  /WSDESIGN=A, B, A BY B.
 281 
 282 The estimation of simple effects in completely within subjects designs
 283 requires no new concepts; we simply apply the same rules to the WSDESIGN
 284 subcommand that we applied to the DESIGN subcommand. So the simple effects
 285 of A at each level of B would be specified as
 286 
 287 MANOVA V1 TO V4
 288  /WSFACTORS=A(2) B(2)
 289  /WSDESIGN=B,
 290            A W B(1), A W B(2).
 291 
 292 This is also true for the more complicated three-way and higher order
 293 cases.
 294 
 295 
 296          Models Involving Between and Within Subjects Effects
 297 
 298 
 299 Since the between and within subjects parts of the model are specified
 300 separately in MANOVA, the case of a design involving both between and
 301 within subjects factors presents some complications. The specifications
 302 for each part of the model are crossed by default. That is, all between
 303 subjects factors are automatically crossed with all within subjects
 304 factors. Since MANOVA will not allow the specification of between subjects
 305 factors on the WSDESIGN subcommand or within subjects factors on the DESIGN
 306 subcommand, we need a way to tell the procedure that we want to fit the
 307 effects of a factor of one type at each level of one or more factors of
 308 the other type. Fortunately, there is a method for doing this, and the
 309 algorithm involved is generally no more complex than the earlier one, and
 310 in many cases it is even simpler.
 311 
 312 Take the case of a two-way model involving one between subjects factor
 313 (call it A) and a within subjects factor (TIME). The standard syntax for
 314 the full factorial model is
 315 
 316 MANOVA V1 V2 BY A(1,2)
 317  /WSFACTORS=TIME(2)
 318  
 319 which is equivalent to specifying either TIME on the WSDESIGN or A on the
 320 DESIGN subcommand, or both. If we want to estimate the simple effects of
 321 time for each level of A, we use the MWITHIN keyword on the DESIGN
 322 subcommand, and replace the main effect of A with MWITHIN A(1) and 
 323 MWITHIN A(2):
 324 
 325 MANOVA V1 V2 BY A(1,2)
 326  /WSFACTORS=TIME(2)
 327  /DESIGN=MWITHIN A(1), MWITHIN A(2).
 328 
 329 MWITHIN stands for mean within, and it effectively turns the crossing of
 330 A and TIME into the nesting of time within each level of A. This case
 331 requires some special caution in reading the output, since what we are
 332 thinking of as a simple main effect, TIME at each level of A, is listed
 333 on the output as an interaction effect. This analysis produces two tables,
 334 the first of which contains the between subjects part of the analysis:
 335 
 336 * * * * * * A n a l y s i s   o f   V a r i a n c e -- design   1 * * * * * *
 337 
 338 Tests of Between-Subjects Effects.
 339 
 340  Tests of Significance for T1 using UNIQUE sums of squares
 341  Source of Variation          SS      DF        MS         F  Sig of F
 342 
 343  WITHIN+RESIDUAL           60.64      17      3.57
 344  MWITHIN A(1)             441.80       1    441.80    123.85      .000
 345  MWITHIN A(2)             440.06       1    440.06    123.36      .000
 346 
 347  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 348 
 349 What is actually being tested by the MWITHIN A(1) and MWITHIN A(2) terms
 350 here are the null hypotheses that the average value across all time points
 351 (represented to within a constant multiple by transformed variable T1) is
 352 zero within level 1 and level 2 of A, respectively. These are in general
 353 not hypotheses in which we are usually interested. The hypotheses of common
 354 interest are to be found in the within subjects section of the output:
 355 
 356 * * * * * * A n a l y s i s   o f   V a r i a n c e -- design   1 * * * * * *
 357 
 358 Tests involving 'TIME' Within-Subject Effect.
 359 
 360  Tests of Significance for T2 using UNIQUE sums of squares
 361  Source of Variation          SS      DF        MS         F  Sig of F
 362 
 363  WITHIN+RESIDUAL           78.64      17      4.63
 364  MWITHIN A(1) BY TIME       9.80       1      9.80      2.12      .164
 365  MWITHIN A(2) BY TIME      20.06       1     20.06      4.34      .053
 366 
 367  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 368 
 369 Transformed variable T2 represents a normalized difference variable comparing
 370 the two TIME points. Thus a test of MWITHIN A(1) BY TIME represents a test of
 371 the null hypothesis that the TIME differences are zero at level 1 of factor
 372 A, and MWITHIN A(2) BY TIME corresponds to a similar test at level 2 of
 373 factor A.
 374 
 375 The substitution rule in this case is even simpler than in cases in which
 376 all factors are either between or within subjects in nature. That is, all we
 377 had to do was to remove the A effect from the DESIGN subcommand and replace
 378 it with MWITHIN each level of A. The same rule applies when we want to go
 379 the other way, to look at A differences at each TIME point:
 380 
 381 MANOVA V1 V2 BY A(1,2)
 382  /WSFACTORS=TIME(2)
 383  /WSDESIGN=MWITHIN TIME(1), MWITHIN TIME(2)
 384 
 385 produces tests of corresponding null hypotheses with the roles of the two
 386 factors reversed. However, in this case the tables are presented somewhat
 387 differently, as all four hypothesis degrees of freedom in the analysis are 
 388 defined as within subjects effects. In each case we have a constant or
 389 intercept term, followed by the term of interest, labeled essentially as
 390 an interaction term.
 391 
 392 * * * * * * A n a l y s i s   o f   V a r i a n c e -- design   1 * * * * * *
 393 
 394 Tests involving 'MWITHIN TIME(1)' Within-Subject Effect.
 395 
 396  Tests of Significance for T1 using UNIQUE sums of squares
 397  Source of Variation          SS      DF        MS         F  Sig of F
 398 
 399  WITHIN+RESIDUAL           65.29      17      3.84
 400  MWITHIN TIME(1)          408.71       1    408.71    106.42      .000
 401  A BY MWITHIN TIME(1)      10.82       1     10.82      2.82      .112
 402 
 403  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 404 
 405 * * * * * * A n a l y s i s   o f   V a r i a n c e -- design   1 * * * * * *
 406 
 407 Tests involving 'MWITHIN TIME(2)' Within-Subject Effect.
 408 
 409  Tests of Significance for T2 using UNIQUE sums of squares
 410  Source of Variation          SS      DF        MS         F  Sig of F
 411 
 412  WITHIN+RESIDUAL           74.00      17      4.35
 413  MWITHIN TIME(2)          473.68       1    473.68    108.82      .000
 414  A BY MWITHIN TIME(2)      18.95       1     18.95      4.35      .052
 415 
 416  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 417 
 418 In this case T1 is simply V1 and T2 is simply V2. That is, the transformation
 419 applied to the dependent variables was an identity transformation. Thus the
 420 MWITHIN TIME(1) effect tests the null hypothesis that the mean of V1 is zero,
 421 averaged across both levels of A, and MWITHIN TIME(2) tests a similar 
 422 hypothesis concerning V2. As before, these tests of constant or intercept
 423 terms are not generally of interest. The terms labeled as interactions,
 424 A BY MWITHIN TIME(1) and A BY MWITHIN TIME(2) are the effects we want, as
 425 they test the null hypotheses that there are no population differences
 426 between the levels of factor A for V1 and V2, respectively.
 427 
 428 The general substitution rule for designs with both between and within
 429 subjects factors is to substitute MWITHIN each level of a particular main
 430 or interaction effect for that factor or interaction and all effects
 431 encompassed by that term. So if we wanted to estimate the effects of TIME
 432 at each level of the breakdown of an A by B between subjects design, (the
 433 simple, simple main effects of TIME within A by B) we would specify:
 434 
 435 MANOVA V1 V2 BY A B(1,2)
 436  /WSFACTORS=TIME(2)
 437  /DESIGN=MWITHIN A(1) BY B(1), MWITHIN A(1) BY B(2),
 438          MWITHIN A(2) BY B(1), MWITHIN A(2) BY B(2).
 439 
 440 Thus MWITHIN A BY B specifications replace the A by B interaction and the
 441 A and B main effects, which are encompassed within A by B. If we wanted to
 442 estimate TIME effects only within the levels of A, we would specify:
 443 
 444 MANOVA V1 V2 BY A B(1,2)
 445  /WSFACTORS=TIME(2)
 446  /DESIGN=B, A BY B,
 447          MWITHIN A(1), MWITHIN A(2).
 448 
 449 The same logic applies completely when testing simple effects of between
 450 subjects factors at different levels of within subjects factors. The
 451 substitution algorithm here can also, as in the case involving only between
 452 or within subjects factors, be extended to as many factors as necessary. 
 453 
 454 As mentioned earlier, while it is possible in some designs to estimate
 455 more than one set of simple effects at a time, it is safest to do them
 456 individually, as the results of specifying redundant requests are often
 457 meaningless ANOVA tables. This is particularly true with regard to use
 458 of the MWITHIN keyword in releases prior to verion 5.0 of SPSS. In later
 459 releases only one term can be used with MWITHIN, but in earlier releases
 460 use of redundant MWITHIN requests may produce output of questionable
 461 validity that some users will not be able to properly interpret.
 462 
 463 Finally, there is some disagreement in the ANOVA literature about the use
 464 of error terms in designs involving both between and within subjects
 465 factors. Specifically, it is sometimes claimed to be desirable to use a
 466 pooled error term when fitting A within each TIME point, just as a pooled
 467 error term is used when fitting TIME effects within each A level. However,
 468 the simple effects of A at each level of TIME are simply the A effects 
 469 for the original correlated dependent variables. They are therefore not
 470 independent and cannot be pooled to obtain a test statistic with a proper
 471 F-distribution under the null hypothesis. Therefore, in this situation
 472 MANOVA uses a separate error term at each level of TIME, equivalent to a
 473 simple univariate analysis of variance on each dependent variable.
 474 
 475 
 476 
 477 Best matches for simple effects & three-way interaction & repeated measures
 478 Tests of such simple main effects or simple interaction effects are generally 
 479 easily ... Jump to text »
 480 Let's illustrate with the case of a three-way design, with factors A, B and C. 
 481 For ... Jump to text »
 482 data involving within subjects (often involving repeated measures) effects. Jump 
 483 to text »
 484 More matches »« Fewer matches