32 The reliability and construct validity of student perceptions of teaching quality The results of the d-study are visualized in Figure 2.1. As can be seen in the graph, a specific number of students and measurement moments is necessary to obtain a reliable picture of a teacher’s teaching quality. The three lines run almost parallel, indicating that the number of students had little effect on the reliability of the scores. To measure teaching quality with a reliability estimate of at least .80 (which is considered the minimum for making high-stakes decisions; COTAN, 2010) at least three measurements are needed. The graph starts plateauing from three measurement points on, showing that further increase beyond three measurements has less effect on reliability. Local reliability Table 2.3 shows the discrimination ( Table 2.3 shows the discrimination ( -value) and location ( k a-value) and location ( k d-value) parameters of the items (k refers to the item numbers in Appendix A). It also presents the item information values (which are a function of the (which are a function of the -value and the k a-value and the k d-value) at five distinct points on the teacher latent scale (the distribution of teaching quality scores), namely at q = 0.0 (the average of all teachers), and for 0.5 and 1.0 standard deviations below and above the average. The higher the item information value, the higher the contribution of the item to the teacher scale. Table 2.3 Parameters and information values of the items Item parameters Item Information Values K are a function of the -value and the k a k d Ѳ = -1.00 Ѳ = -0.50 Ѳ = 0.00 Ѳ = 0.50 Ѳ = 1.00 1 1.64 -0.41 0.79 0.75 0.75 0.79 0.72 2 2.40 -0.62 1.56 1.30 1.31 1.56 1.15 3 1.99 -0.04 1.05 0.98 0.89 1.00 1.04 4 1.95 0.43 0.88 1.12 1.11 1.11 1.11 5 1.33 0.55 0.47 0.58 0.63 0.63 0.62 6 2.33 -0.61 1.56 1.31 1.32 1.56 1.14 7 1.86 -0.04 0.96 0.96 0.90 0.97 0.94 8 1.78 -0.69 0.81 0.73 0.77 0.84 0.73 9 2.16 -0.41 1.30 1.23 1.20 1.31 1.05 10 1.73 -0.40 0.89 0.89 0.89 0.90 0.75 11 1.62 -0.76 0.73 0.67 0.72 0.77 0.67 12 2.16 0.09 1.19 1.24 1.08 1.20 1.24 13 1.11 -0.32 0.46 0.49 0.49 0.45 0.38 14 1.11 0.97 0.22 0.34 0.49 0.60 0.63 15 1.77 -0.72 0.89 0.84 0.87 0.89 0.71
RkJQdWJsaXNoZXIy MjY0ODMw