21 2 Technical developments nowadays enable eff icient collection and processing of student perceptions of teaching quality. The Impact! tool is an example of such a technological development. It enables students to use a digital device to easily give feedback to their teachers about the degree to which the lesson that just ended met several characteristics of effective teaching. Despite the advantages of using student perceptions for evaluating teaching quality, some critical concerns remain (Camburn, 2012; de Jong & Westerhof, 2001; Ferguson, 2012; Muijs, 2006) with regard to, among other things, the construct validity and reliability of the instruments used for measuring teaching quality. Research on this topic is limited and scientif ic debate continues. The investigation of these psychometric features of the Impact! tool contributes to the debate about the validity of student perception measures of teaching quality. The following research questions are answered in this chapter: Is there support for the construct validity of the Impact! questionnaire? How reliable are student perceptions of teaching quality as measured by means of the Impact! tool? 2.2 THEORETICAL FRAMEWORK 2.2.1 Construct validity The construct validity of a questionnaire reflects the extent to which the questionnaire measures the construct that is intended to be measured (Cronbach & Meehl, 1955; Messick, 1995). To evaluate construct validity, first, one needs to establish whether the questionnaire covers every single element of a construct (Gravetter & Forzano, 2012; Radhakrishna, 2007; Shuttleworth, 2009). For that purpose, the construct that one intends to measure needs to be conceptualized by searching for specific characteristics of that construct and by reviewing relevant research on the construct. Experts in the particular research field can be asked for their ideas about characteristics of the construct. Next, the construct characteristics need to be operationalized in items. A draft version of the questionnaire can then be presented to the target population, to ensure that the items are answerable and understandable for them (Baarda & De Goede, 2006; Camburn, 2012). Secondly, the construct validity of a questionnaire can be evaluated using a statistical approach. Factor-analytic approaches are very familiar. The key concept of factor analysis is that several items have similar patterns of responses because they are all associated with a latent, that is, not directly measured, variable. To determine to what extent a pre-defined theoretical construct can
RkJQdWJsaXNoZXIy MjY0ODMw