62 | Chapter 3 (institute x year) to keep the rating process feasible. As the portfolio was implemented later in one institute there were too little portfolios available to rate material from the third year, so the decision was made to rate fifteen portfolios for the first two years for this institute. Analyses Phase one: Number of Forms Data were exported from the mirror database into eleven different data files that signified the eleven unique pre-structured forms. Within these data files, each case represented one completed form. In subsequent steps, single cases were aggregated per portfolio, to determine the number of completed forms per portfolio. Different descriptive statistics were calculated using IBM SPSS Statistics 25. An exception was made concerning the form: ‘learning goals and plans’. Since trainees differed to a great extent in the number of learning goals that they formulated on one form (ranging from one to twenty-four), it was considered inconsequential to compare numbers of forms. So, number of learning goals were compared instead. A goal was included if any text was provided within the first of five text boxes (‘description of the learning goal’), relevance of content was not considered at this point. Phase two: Content analysis A quantitative content analysis of portfolios was performed, such an analysis consists of ratings made by the use of a deductively designed coding scheme which is elaborated in a codebook.36-38 The aim of our analysis was to identify the presence of effective SRL behaviour within the documentation kept in the portfolios, thereby focusing on the SRL processes that, according to health professions education literature, can be supported by portfolio use: self-assessment, goal setting and planning, and monitoring.9-11 As there was no existing instrument available that was suitable to rate the presence of these SRL processes within written documentation, we used research literature regarding the different processes to develop our codebook (see Appendix A). For the development of the codebook, we searched the research literature for descriptions of good practice of the included SRL processes, so we could formulate criteria that can be rated for their presence. We decided to focus on two processes that inform self-assessment - reflection and feedback - instead of self-assessment itself, as this process, which is for the most part cognitively performed, is difficult to objectify.39 For the formulation of criteria concerning reflection we used the framework of Hatton and Smith that distinguishes different types of writing with an increasing level of reflection.40,41 With regard to feedback we used the conditions for effective feedback formulated by Gibbs and Simpson.42 Literature by Zimmerman was used to formulate the criteria on effective goal setting and planning.43
RkJQdWJsaXNoZXIy MjY0ODMw