27 2 Table 2.1 Generalizability theory model: parameters, interpretations and distributions Parameter Interpretation Distribution Note that since students are nested within teachers, the between-teacher variance is confounded with the between-student-groups variance. 2.3.5 Estimation method The combined IRT and GT model is a multilevel model, as introduced by Fox and Glas (2001, 2003). The combined model was estimated in a Bayesian framework using OpenBugs (Lunn et al., 2009). The motive for this choice rather than using traditional standard software for latent variable modelling was that OpenBugs allows the users to completely specify their own model, taking all facets and levels into account. Estimation in OpenBugs was done with a Markov chain Monte Carlo (MCMC) procedure. The script for the estimation of the overall model is provided in Appendix B. The MCMC sampling procedures in the application were run with over 24000 iterations for each of two chains. Each chain had a burn-in of 10,000 iterations. These large numbers were chosen to ensure convergence and small estimation errors. The sampling procedures were checked for convergence by visual inspection of the trace plots, by comparing the estimates of the two chains and by checking whether the MCMC errors were acceptably small (< 5% of the estimated standard deviation; Lunn et al., 2012). 2.4 DATA ANALYSIS 2.4.1 Construct validity To assess the construct validity of the Impact! questionnaire statistically, the fit of the combined IRT and GT model was investigated in three ways. The first investigation tested the probability of the predicted item responses as implied by the formula. For that purpose, the data were divided into three subgroups: a group with low total scores from students (below 11 out of the possible Table 2.1 G ner lizability the ry model: param te s, interpretations a d distributions Parameter Interpretation Distribution with the between-student-group varianc . 2.3.5 Estimation Method 2 2 | | 2 | main effect teacher (0, ) main effect time (0, ) main effect students nested in teachers (0, ) interaction teachers & time (0, ) j j t t i j i j jt jt it j N N N N q s w s t s d s e 2 2 | interaction students & time nested in teachers (0, ) it j N s main effect teacher ain effect time i effect students nested in teach rs i tion teachers & time i tion teachers & time nested in achers
RkJQdWJsaXNoZXIy MjY0ODMw