42018-04-17 11:15:27PeterWatson32018-04-17 11:15:15PeterWatson22018-04-17 11:12:50PeterWatson12018-04-17 11:09:28PeterWatson

(The below is reproduced from Alex Quent's webpage at ). 14 March 2018 •Preface ◦Libraries & functions ◦Model 1: Simple regression model with 1 level ◦Model 2: A random intercept for each subject ◦Model 3: Random intercept and slope for each subject ◦Model 4: Random intercepts for each object and each subject ◦Model 5: Random intercepts for each object, each subject, quadratic term and random slopes for subjects ◾Model 5.1 Model 5.2 Inaccurate estimations of the random intercepts for each subject Conclusion Preface This work documents my attempt to understand the model structure of mixed linear models or multilevel models. I adapted a formula that I found on Wikipedia and work myself through the different models and test whether I am able to retrieve the different parameters with lmerTest. Bodo Winter’s tutorial on the lme was very useful for this. The main motivation for doing all this is to use basically model 5 in my analysis of my first experiment. This work is also part of the process of pre-registering the study with a complete analysis script, which I will upload soon. The design of the fictive experiment that 100 subjects see 20 objects and provide a continuous measure of memory for each objects. The expectancy of the objects (X X ) servers as a predictor for the memory performance (Y Y ). I start with very easy models and add more effects in order to finally arrive at my target model (model 5). 1){ stop("There is no p-value greater than 1") } else if(x < 0.001){ x.converted <- '< .001' } else{ x.converted <- paste(sign,substr(as.character(round(x, 3)), 2,5)) } } else { if(x > 1){ stop("There is no p-value greater than 1") } else if(x < 0.001){ x.converted <- '< .001' } else{ x.converted <- paste(sign,substr(as.character(round(x, 3)), 2,5)) } } return(x.converted) } ]]>Model 1: Simple regression model with 1 level Assume that expectancy and memory performance was aggregated across subjects.In this model, objects’ memorability is predicted by their average expectancy values. The corresponding formula looks like this: Y i =β 0 +β 1 X i +e i Yi=β0+β1Xi+ei •Y i Yi refers to memorability of object i. •X i Xi refers to average expectancy of object i. •β 0 β0 refers to the intercept. •β 1 β1 refers to the linear slope. •e i ei refers to the random errors of prediction. |t|) ## (Intercept) 7.99868 0.02806 285.055 < 2e-16 *** ## x 0.28629 0.02879 9.944 9.73e-09 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.1255 on 18 degrees of freedom ## Multiple R-squared: 0.846, Adjusted R-squared: 0.8375 ## F-statistic: 98.89 on 1 and 18 DF, p-value: 9.731e-09]]>Unsurprisingly, lm returns estimates that are very close the the true values β 0 ^ = β0^= 7.9986752 (vs 8) and β 1 ^ = β1^= 0.2862898 (vs 0.3). Model 2: A random intercept for each subject Now, I don’t aggregated anymore and there is one value for each subject and each object and I allow for different intercepts for each subject, which basically means that I assume that my subjects’ general memory performance varies. Level 1: Y ij =β 0j +β 1 X ij +e ij Yij=β0j+β1Xij+eij Level 2: β 0j =γ 00 +γ 01 W j +u 0j β0j=γ00+γ01Wj+u0j •Y ij Yij refers to memory score for object i and subject j. •β 0j β0j refers to the intercept of the dependent variable in subject j (level 2) in other words the individual difference in general performance across subjects. •β 1 β1 refers to the linear slope. •X ij Xij refers to the level 1 predictor. •e ij eij refers to the random errors of prediction for the level 1 equation. •γ 00 γ00 refers to the overall intercept. This is the grand mean of the scores on the dependent variable across all the subjects when all the predictors are equal to 0. •γ 01 γ01 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W j Wj refers to the level 2 predictor. •u 0j u0j refers to the random error component for the deviation of the intercept of a subject from the overall intercept. |t|) ## (Intercept) 1.799e+01 8.103e-02 9.900e+01 222 <2e-16 *** ## x -1.000e+02 2.236e-03 1.899e+03 -44729 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Correlation of Fixed Effects: ## (Intr) ## x 0.000 anova(model2) ## Analysis of Variance Table of type III with Satterthwaite ## approximation for degrees of freedom ## Sum Sq Mean Sq NumDF DenDF F.value Pr(>F) ## x 18995109 18995109 1 1899.2 2000646776 < 2.2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 # Extract intercept for each subject by coef(model2). ]]>As you can see above (fixed effects), β 0. ^ = β0.^= 17.986858 is approximating the true (17.9836611). The same is true for β 1 ^ = β1^= -99.9995396 (vs -100). The estimated level 1 intercept for subject 1 was also close to the true value (17.8598629 vs 17.8225987). Model 3: Random intercept and slope for each subject In the next model, I am additionally allowing the slope randomly different for each subject. This means that the strength of the relationship between the level 1 predictor X X and the memory scores Y Y varies across subjects. Level 1: Y ij =β 0j +β 1j X ij +e ij Yij=β0j+β1jXij+eij Level 2: β 0j =γ 00 +γ 01 W j +u 0j β0j=γ00+γ01Wj+u0j β 1j =γ 10 +u 1j β1j=γ10+u1j •Y ij Yij refers to memory score for object i and subject j. •β 0j β0j refers to the intercept of the dependent variable in subject j (level 2) in other words the individual difference in general performance across subjects. •β 1j β1j refers to the linear slope that varies across subjects in other words the relationship X X and Y Y is different across subjects. •X ij Xij refers to the level 1 predictor. •e ij eij refers to the random errors of prediction for the level 1 equation. •γ 00 γ00 refers to the overall intercept. This is the grand mean of the scores on the dependent variable across all the subjects when all the predictors are equal to 0. •γ 01 γ01 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W j Wj refers to the level 2 predictor. •u 0j u0j refers to the random error component for the deviation of the intercept of a subject from the overall intercept. •γ 10 γ10 overall slope of the level 2. •u 1j u1j refers to the random error component for the deviation of the slope of a subject from the overall intercept. |t|) ## (Intercept) 17.95924 0.09039 99.04000 198.7 <2e-16 *** ## x -100.01190 0.01102 98.83000 -9073.0 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Correlation of Fixed Effects: ## (Intr) ## x -0.106]]>F) ## x 836595 836595 1 98.831 82319857 < 2.2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 }} ]]>Model 4: Random intercepts for each object and each subject As explained, above each subject rates each of twenty objects. In addition to the varying performance among the participants, one can assume that there are objects, which can be remembered more easily than others. Therefore, I add a random intercept for each object representing at varying baseline memorability. Level 1: Y ij =β 0ij +β 1 X ij +e ij Yij=β0ij+β1Xij+eij Level 2: β 0ij =γ 00 +γ 01 W 1i +u 01i +γ 02 W 2j +u 02j β0ij=γ00+γ01W1i+u01i+γ02W2j+u02j •Y ij Yij refers to memory score for object i and subject j. •β 0ij β0ij refers to the intercept of the dependent variable that is different for each subject and object. •β 1 β1 refers to the linear slope. •X ij Xij refers to the level 1 predictor. •e ij eij refers to the random errors of prediction for the level 1 equation. •γ 00 γ00 refers to the overall intercept. This is the grand mean of the scores on the dependent variable across all the subjects when all the predictors are equal to 0. •γ 01 γ01 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 1i W1i refers to the level 2 predictor for objects. •u 01i u01i refers to the random error component for the deviation of the intercept of a object from the overall intercept. •γ 02 γ02 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 2j W2j refers to the level 2 predictor for subjects. •u 02j u02j refers to the random error component for the deviation of the intercept of a subject from the overall intercept. |t|) ## (Intercept) 1.931e+01 1.762e+00 1.237e+03 10.96 <2e-16 *** ## x -1.000e+02 2.316e-03 1.976e+03 -43184.92 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Correlation of Fixed Effects: ## (Intr) ## x 0.000 anova(model4) ## Analysis of Variance Table of type III with Satterthwaite ## approximation for degrees of freedom ## Sum Sq Mean Sq NumDF DenDF F.value Pr(>F) ## x 18955582 18955582 1 1975.6 1864937637 < 2.2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1]]>Firstly, on level 1 estimates were close to the true values: β 0.. ^ = β0..^= 19.3131619 (vs 19.3127994) as well as β 1 ^ = β1^= -100.0025395 (vs -100). Now looking at the level 2 estimates: the intercept for the object 1 across all subjects is β 01. ^ β01.^ = 19.1269314 (vs 19.1249384), while the intercept for subject 1 across all objects is β 0.1 ^ β0.1^ = -2.8745026 (vs -2.8654707). Model 5: Random intercepts for each object, each subject, quadratic term and random slopes for subjects Model 5.1 I am finally arriving at the model, which I want to use to model my data in the upcoming analysis. This models contains random intercepts for each object and each subject. A second quadratic term is added to test for non-linearities. Both the slope of the linear term and the slope of the quadratic term are random for each subject. At this point it is crucial to note that the predictor X X is random for each subject and object just as if the subjects had to rate the expectancy of each object after their memory was tested. This will be the case of my experiment, but I also collected normative data, where X X varies only across objects representing aggregate values (see model 5.2). Level 1: Y ij =β 0ij +β 1j X ij +β 2j X 2 ij +e ij Yij=β0ij+β1jXij+β2jXij2+eij Level 2: β 0ij =γ 00 +γ 01 W 1i +u 01i +γ 02 W 2j +u 02j β0ij=γ00+γ01W1i+u01i+γ02W2j+u02j β 1j =γ 10 +u 1j β1j=γ10+u1j β 2j =γ 20 +u 2j β2j=γ20+u2j •Y ij Yij refers to memory score for object i and subject j. •β 0ij β0ij refers to the intercept of the dependent variable that is different for each subject and object. •β 1j β1j refers to the linear slope for the relationship in subject j (level 2) between the level 1 predictor and the dependent variable. •X ij Xij refers to the linear level 1 predictor that varies for across objects and subjects. •β 2j β2j refers to the quadratic slope for the relationship in subject j (level 2) between the level 1 predictor and the dependent variable. •X 2 ij Xij2 refers to the quadratic level 1 predictor that varies for across objects and subjects. •e ij eij refers to the random errors of prediction for the level 1 equation. •γ 00 γ00 refers to the overall intercept. This is the grand mean of the scores on the dependent variable across all the subjects when all the predictors are equal to 0. •γ 01 γ01 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 1i W1i refers to the level 2 predictor for objects. •u 01i u01i refers to the random error component for the deviation of the intercept of a object from the overall intercept. •γ 02 γ02 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 2j W2j refers to the level 2 predictor for subjects. •u 02j u02j refers to the random error component for the deviation of the intercept of a subject from the overall intercept. •γ 10 γ10 overall linear slope of the level 2. •u 1j u1j refers to the random error component for the deviation from the linear slope of a subject from the overall intercept. •γ 20 γ20 overall quadratic slope of the level 2. •u 2j u2j refers to the random error component for the deviation from the quadratic slope of a subject from the overall intercept. |t|) ## (Intercept) 16.745775 1.434189 284.140000 11.676 <2e-16 *** ## x -0.006660 0.008967 99.950000 -0.743 0.459 ## x2 10.011245 0.011079 99.070000 903.628 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Correlation of Fixed Effects: ## (Intr) x ## x 0.038 ## x2 0.067 0.000 anova(model5) ## Analysis of Variance Table of type III with Satterthwaite ## approximation for degrees of freedom ## Sum Sq Mean Sq NumDF DenDF F.value Pr(>F) ## x 0.0 0.0 1 99.955 1 0.4594 ## x2 8008.6 8008.6 1 99.073 816544 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 corrResult <- cor.test(coef(model5)$subNo$`(Intercept)`, gamma00 + mean(gamma01*w1 + u01) + gamma02*w2 + u02)]]>The intercept of object 1 across all subjects is β 01. ^ = β01.^= 14.4912111 (vs 14.4971756), while subject’s 1 performance across all objects was β 0.1 ^ = β0.1^= 18.9779272 (vs 20.0973103). Compared to the other estimates, the last estimate is quite far off. However the estimates are still highly correlated with the true values, r = .969, p < .001. However on some simulations they are not, which I realised after running this script a couple of times. I have no idea why this is the case. On level 2, the linear slope for subject 1 was β 11 ^ = β11^= 0.035072 (vs 0.0042065), while the quadratic slope was β 21 ^ = β21^= 9.9665142 (vs 9.988436). Model 5.2 This model is basically the same than the previous except one different. X X varies only across objects like as if it was aggregated values from a different population. This is case in the experiment, in which one could use the normative data instead of participants’ ratings to predict the memory score. Level 1: Y ij =β 0ij +β 1j X i +β 2j X 2 i +e ij Yij=β0ij+β1jXi+β2jXi2+eij Level 2: β 0ij =γ 00 +γ 01 W 1i +u 01i +γ 02 W 2j +u 02j β0ij=γ00+γ01W1i+u01i+γ02W2j+u02j β 1j =γ 10 +u 1j β1j=γ10+u1j β 2j =γ 20 +u 2j β2j=γ20+u2j •Y ij Yij refers to memory score for object i and subject j. •β 0ij β0ij refers to the intercept of the dependent variable that is different for each subject and object. •β 1j β1j refers to the linear slope for the relationship in subject j (level 2) between the level 1 predictor and the dependent variable. •X i Xi refers to the linear level 1 predictor that varies for across objects but not across subjects. •β 2j β2j refers to the quadratic slope for the relationship in subject j (level 2) between the level 1 predictor and the dependent variable. •X 2 i Xi2 refers to the quadratic level 1 predictor that varies for across objects but not across subjects. •e ij eij refers to the random errors of prediction for the level 1 equation. •γ 00 γ00 refers to the overall intercept. This is the grand mean of the scores on the dependent variable across all the subjects when all the predictors are equal to 0. •γ 01 γ01 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 1i W1i refers to the level 2 predictor for objects. •u 01i u01i refers to the random error component for the deviation of the intercept of a object from the overall intercept. •γ 02 γ02 refers to the overall regression coefficient, or the slope, between the dependent variable and the level 2 predictor. •W 2j W2j refers to the level 2 predictor for subjects. •u 02j u02j refers to the random error component for the deviation of the intercept of a subject from the overall intercept. •γ 10 γ10 overall linear slope of the level 2. •u 1j u1j refers to the random error component for the deviation from the linear slope of a subject from the overall intercept. •γ 20 γ20 overall quadratic slope of the level 2. •u 2j u2j refers to the random error component for the deviation from the quadratic slope of a subject from the overall intercept. The intercept of object 1 across all subjects is β 01. ^ = β01.^= 18.1767803 (vs 18.1734038), while subject’s 1 performance across all objects was β 0.1 ^ = β0.1^= 20.1355938 (vs 17.1431682). Compared to the other estimates, the last estimate is quite far off. However the estimates are still highly correlated with the true values, r = .973, p < .001. However on some simulations they are not, which I realised after running this script a couple of times. I have no idea why this is the case. On level 2, the linear slope for subject 1 was β 11 ^ = β11^= 0.0929432 (vs 0.0373702), while the quadratic slope was β 21 ^ = β21^= 9.8407853 (vs 8.9302937). Inaccurate estimations of the random intercepts for each subject There seems to be a problem in both simulation above that the estimate of β 0.j ^ β0.j^ was quite inaccurate sometimes, but this could be due the fact that I used 16 subjects in previous analyses. To test this, I run number of simulation with the same parameters. Even when increasing the number of subjects to 100, there were models that couldn’t be estimated or had every low correlations coefficients. See the histogram below for the distribution.