Short overview:

User can change parameter values, sample size, and number of random factor levels in a mixed model with 2 fixed factor levels and 2 to 20 random factor levels. Random factor can be nested in or crossed with fixed factor. Random intercepts (no interaction), random slopes (interaction), or ANOVA (no random factor) models can be fit. Data can be simulated from a model with or without interaction, independent of which model is used to fit the simulated data. Estimated treatment means and random factor means can be plotted. Fixed and random effects summary output and conditional and marginal R-squared from the fitted model are reported.

Important details:

• "trt2" is a 0/1 indicator for treatment group. 𝛽 is its "slope", quantifying the size of the fixed effect. This can also be thought of as the difference in treatment means. Selecting "Treatment means" in the "Plot" box will display each mean as a horizontal dashed line.

• "mᵢ" is the random intercept term. Its values are generated from a normal distribution with mean zero and standard deviation σₘ. As you increase σₘ, you will see the means of each level shift up or down. This shift is random for each group; change the random seed and a new set of random intercepts will be generated. Selecting "Random factor means" in the "Plot" box will display the means for each level.

• There are two R-squared values reported: "conditional" calculated error sum of squares around the means for treatment and random intercept. In other words, it conditions on the values of mᵢ, bᵢ (if fitting the random slopes model) and estimated treatment mean. The "marginal" R-squared conditions only on the treatment mean. "Marginal" cannot exceed "conditional".

• Selecting the "Create interaction" box adds a slider for the value of σb. Selecting "Mixed effects ANOVA (with interaction)" in the "select the model" box adds a "bᵢ·trt2" term to the model, where "bᵢ" is the random slope. Its values are generated from a normal distribution with mean zero and standard deviation σb. As you increase σb, you will see the differences between treatment means get larger or smaller. The amount by which the differences in treatment means change is random; change the random see and a new set of random slopes will be generated.
  

The remaining bullet points address aspects of the random slopes (i.e. interaction) model:

• We are accustomed to thinking of slope as a property of a line, and there are no lines here. The "x" variable in this model is "trt", which is binary, and so the "slopes" are the differences between the means of the two treatment groups. If "x" were continuous, we would see a line for each level of the random factor, and if the model generated random slopes, σb would quantify the variability in these slopes.

• A random slopes model typically lists the random intercepts and slopes as distributed multivariate normal, with mean zero and a 2x2 covariance matrix (larger if there is more than one random factor). This app generates the intercepts and slopes independently, and I decided to write the distributions of random intercepts and slopes separately, rather than in a 2x2 matrix with zeros on the off-diagonal. When I initially created this app, it included a covariance slider. I decided to remove this because the visual effect of adjusting covariance seem too complicated for an app that is meant to clarify how mixed models work. If this app had a continuous "x" variable, the effect of adjusting the covariance between intercept and slope would be clear.

• Selecting "Random factor nested in treatment" results in the half of the random factor levels being assigned only treatment 1, and the other half only treatment 2. Under this design, an interaction effect (between treatment effect and random level) is indistinguishable from the random intercept effect, because under each random level we only observe one treatment mean. We can suppose each level would have a mean for the other treatment, and we could suppose that the difference in treatment means would vary randomly across random factor levels, but a fitted random slopes model will not be able to identify this "random slope" variance. Here's how to see this in the app:

  • 1. Select "Create interaction", "Random factor nested in treatment", and select the "with interaction" model.

    2. Bring the slider for the random slope standard deviation (σb) all the way left, and look at the estimated random intercept standard deviation in the regression output. It should be similar to the value set on the slider for σₘ.

    3. Now, increase σb and watch what happens to the estimated random effect standard deviations. You should see the random intercept standard deviation increase, and nothing consistent happen to the random slope standard deviation.

    4. Now, unselect "Random factor nested in treatment" and do this again; now you should see unbiased estimates for both standard deviations.