Using an intraclass correlation to measure consistency across subjects
Fit a multilevel model as below (the null model with no predictors) to the outcome score with one column for outcome, y, and one column indicating participant with at least one repetition on some of the participants.
MIXED Y /FIXED= | SSTYPE(3) /METHOD=REML /PRINT=SOLUTION /RANDOM=intercept | SUBJECT(participant) COVTYPE(VC).
The intraclass correlation which measures the consistency of performance across version of the participants equals
Variance Component for Participant / [(Variance Component for Participant) + (Variance Component of Residual)]
This can also be computed from a one way repeated ANOVA if there are the same number of repetitions (e.g. time) per participant.