RELIABILITY & VALIDITY PRPP-ASSESSMENT IN CHILDREN 169 7 Results Overall for all 65 activities, the mean score of stage 1 performance mastery was 60% (SD 25%), and for stage 2, the total score on the use of information processing strategies was 85% (SD 12%). Table 4 shows detailed information. Children scored relatively high on the Quadrant Recall (93%) and the Quadrant Perceive (91%) while the Quadrant Plan (76%) and Quadrant Perform (79%) received lower scores. Most of the errors in stage 1 were due to inaccurate performance (31%, SD 25%), whereas the other errors (repetition, omission and times) ranged from two to four percent. Intra-Rater Reliability As shown by the in Bland-Altman plots in figure 2, all stage one mastery scores lay within the 95% limits of agreement (figure 2a), in which for 15 out of 22 repeated ratings the difference of scores was zero. For the stage two Sum score, 96% of all observations lay within the 95% limits of agreement (figure 2b). In six out of 22 scores, the difference was zero, and 12 scores differed with less than 5%. For the Error scores (stage 1), and the quadrant scores (stage 2), the percentage observations within the 95% limits of agreement are 93%, and 92%, respectively. A more detailed overview of the intra-rater reliability statistics is presented in table 5. Internal Consistency Based on 65 PRPP-Assessment scores on stage 2 (excluding matches), Cronbach’s alpha was 0.92. In 11 PRPP-Assessments, all 35 descriptors were scored, which yielded a Cronbach’s alpha of 0.89. Inter-rater Reliability For the PRPP-Assessment Stage 1 Mastery Score, ICCagreement was 0.646 with an ICCconsistency of 0.734. Based on the overall high performance of children (figure 2), limited variation was observed between cases on the scores of stage 2 of the PRPP-Assessment, which is also confirmed by the SD of the variance components (table 6). This limited variability resulted in ICCs for Stage 2 Sum score and individual quadrants below the expected 0.6 (table 6). Consequently, the SD variance component of the raters can be used as indicator for the inter-rater reliability: a low score indicates a low variation between scores and therefore, a high inter-rater reliability[54].
RkJQdWJsaXNoZXIy MjY0ODMw