Blueprinting for assessment in undergraduate medical physiology curriculum

INTRODUCTION

Physiology is the biological science concerned with the intra and intercellular properties, functions and processes of living organisms that are beautifully orchestrated for normal functioning and living. For teaching physiology to medical undergraduates, we divide the vast syllabus into systems physiology and then integrate and reintegrate topics for better teaching and student learning. Since centuries medical teaching has been monotonous, teacher centric, didactic type and its assessment primarily focussing on the amount of knowledge recalled by means of stereotyped written questions. Recently, many updates and changes have been introduced in the curriculum with a goal to improve student learning.^[1] It is supposedly a competency/outcome based and an assessment driven medical education programme.^[2]

A blueprint is an outline used to define the content of a given assessment. It specifies weightage to various mapped competencies/topics. It may improve teaching planning and time management. Blueprinting of syllabus provides a basis for sampling examination questions for systematic assessment of students.^[1] A blueprint of the mapped physiology curriculum may thus not only outline the competency but also define their exact weightage to improve overall assessment in physiology education.^[3] In this study, we aimed to prepare a blueprint for assessment in the I^st MBBS human physiology curriculum as per the MCI’s new competency-based curriculum.

MATERIAL AND METHODS

This study was conducted in the department of physiology, from December 2019 to May 2020. Institutional Ethics Approval (Ref. No. GU/HREC/2019/1709) was obtained before the study.

As per the guidelines of the Medical Council of India, medical physiology undergraduate syllabus (various chapters) was first divided into two papers, paper-1 and paper-2. Then, each chapter was considered as per the competencies defined by the MCI.^[2] Learning objectives of all the competencies were decided; also division of the broader competencies and addition of certain core competencies (not already provided) were done where ever necessary. Meticulous mapping and structuring of competencies are of prime importance as high number of unwarranted competencies or low number of core competencies may mask the true weightage of the topic in the chapter and hence of the chapter in the concerned paper.

On finalisation, every competency was scored; between 1 and 3; 1 being minimum and 3 being maximum; for both, its “importance (I)” and “frequency (F).”^[3] Importance of a particular competency depicts the subjective/clinical implication of the topic whereas frequency shows how many times the topic has been/should be asked in the examinations. Physiology being a basic science/pre-clinical subject following criteria^[3] was used to calculate the “I” and “F.” Then, the product of I×F (IF) was calculated to find out the IF value of the given competency [Table 1].

Based on the IF values of all the competencies in a chapter, a cumulative IF (∑ IF) value of the chapter was calculated [Table 2]. Then, IF scores of all chapters were added to calculate the total IF score of the concerned paper [Table 3].

“Weightage” is the division of the IF to total IF. Hence, weightage of every competency in chapter (competency IF/ Chapter IF) and weightage of chapter in paper (chapter IF/ total paper IF) can be calculated, as shown in [Tables 2 and 3], respectively.

Product of weightage and maximum marks gives the maximum marks value of the topic or chapter that can be used to frame questions for formative assessment. For example, if the maximum marks of a paper are 100, then a chapter with weightage 0.2 can be asked for a maximum of 20 marks [Table 4].

Hence, the concept of weightage is simple, which can be obtained by multiplying the weightage with maximum marks to get the maximum marks limit. Hence, based on the requirement, like during system or part completion tests, weightage of competencies/topics can be used to calculate maximum marks that can be allotted to a particular competency; whereas during the summative assessment tests like mid-semester or university examinations, chapter weightage can guide the allocation of marks to each chapter. Then, accordingly, questions can be selected.

Now, when we have calculated the marks from weightage of topic or chapter, we can appreciate that weightage can be used to separately calculate marks for theory and MCQs questions. For example, in a 100 marks paper, theory/ subjective questions carry 80 marks and objective/MCQs carry 20 marks then following [Table 5] method can be used to calculate and allocate marks.

The most important step in the whole process is estimation of “I” and “F” values in clearly defined competences. In the current project, total six faculty members from the department initially allocated “I” and “F” values through focussed group discussion and consensus and then substantiated it by seeking inputs/feedback (link for the feedback form: https://docs.google.com/forms/d/1vXanCAsW84ng3ZD197H3ySP_ HuQ9CcF4MlPBaOjpKuI/edit?usp=sharing) and modifications from 10 distinguished Indian Physiologists (reviewers). Only those reviewers were selected who had at least 10 years of experience in teaching physiology to medical undergraduates and had completed basic training in medical education technology. Finally, we came up with the complete blueprint of the physiology undergraduate syllabus. Their feedback is given the results below.

RESULTS

As per the methodology mentioned above, we have blueprinted the physiology curriculum and the final summary the results are shown in [Table 6].

Detailed results with “I and F” of all competencies and weightage to each competency can be accessed from this link https://drive.google.com/file/d/1JrzchRebzcDcbBhgtW5mzLuW29--e3Hh/view?usp=sharing.

All the reviewers appreciated the authors work and agreed with the concept of blueprinting for assessment in physiology and expressed their willingness to implement the blueprint in their respective institutions. Almost nine reviewers felt that the work done is a “good initiative” and “optimum” and were prepared to use the blueprint “as it is” [Figure 1].

Close

Figure 1:

Reviewer acceptability to use authors work “as it is.”

Export to PPT

Only 1 out of the 10 reviewers had major differences regarding some points (this is addressed in the discussion). Reviewers’ feedback on various questions regarding blueprinting of physiology curriculum based on Likert’s scale of 5 is shown in [Figure 2].

Close

Figure 2:

Reviewer’s feedback on Likert’s scale of 5 (mean ± SD) on blueprinting.

Export to PPT

Based on this blueprint, we had conducted the UG formative (system completion tests) exams. [Table 7] is an example of Paper I according to our university. The pattern will be used for the upcoming pre-university and university examinations.

DISCUSSION

It is said that assessment of skills or knowledge is of equal importance to teaching/learning of the skill or knowledge. Formative and summative assessments are the two most important and indispensable forms of student assessments in medical education.^[4] Blueprinting provided us with the exact list of very important/high weightage topics and thus enabled us to focus and coordinate strategies of formative assessment such as conducting poster making sessions, tutorials and framing clinical scenarios for their better understanding. Blueprinting of physiology curriculum provided the weightage of every topic (based on its importance and implication), it guided in sampling and constructing relevant questions from eligible chapters for summative assessment that eventually enhanced construct relevance, student learning and improved performance. Thus, it formed a functional link between formative and summative assessments.

As a matter of fact, due to the current pandemic “COVID-19,” all the I^st MBBS students have already lost a substantial amount of their college teaching/learning time. These students are trying to cope up as much as possible from the online classes, assessments and guidance from the faculty over the phone or live classroom platforms. Thus, to enable students to plan their timetable in a smart and better way for the upcoming pre-university and university examinations, we had shared the details [Table 6] with them. Students are able to study accordingly and perform better in the online assessments. This was also shown in a study that concluded students and faculty were satisfied as blueprinting helped students to attempt examination better, whereas the faculty who validated the blueprint felt that it helped in distribution of appropriate weightage and questions across the topics.^[5] This complete system also forms the basis of the credit hour system that is being used across the globe for student learning and assessment.^[6]

Out of 10 reviewers that we had to substantiate/validate our blueprint, 1 had differences regarding the competency subdivisions, addition of some competencies and I and F values allotted to certain competencies. Suitable changes in the “I and F” were made where ever the authors felt that the suggested changes are relevant and acceptable. There were certain objections raised by the same reviewer that competency sub-divisions should not be made. The authors do not feel that is relevant because we have already stressed and highlighted the importance of proper mapping and structuring of competencies, as high number of unwarranted competencies or low number of core competencies may mask the true weightage of the topic in the chapter and hence of the chapter in the concerned paper. As provided by the MCI, there are some chapters such as endocrinology and CNS where many major topics are merged in a single competency. Because of this chapter, weightage of endocrinology or CNS was coming to be lesser than that of nerve muscle physiology, which is a severe flaw. Hence, we did not accept the objection and retained sub-divisions that we made.

As is obvious from [Table 7], blueprinting guided the paper setters to plan and construct questions from the eligible chapters. This not only reduced the paper setter’s selection bias but also improved the subject coverage and overall acceptability among students. Better subject coverage improved the validity and reliability into the assessment. The elimination of construct over and under representation enabled us to better assess all learning domains (knowledge, skill and affect) by incorporating specific assessment method (Theory/MCQs/PBL/Reasoning/Ethics and so on) for specific learning objective based on its weightage.^[3]

Since the competencies to be assessed were weighted and assessed accordingly, it was easier to make students realise the importance of outcome-based medical education programme.

CONCLUSION

Blueprinting physiology curriculum improves validity, reliability and acceptability of both formative and summative assessments and thus establishes a balance between teaching and student’s overall learning. Blueprinting medical curriculum can be very helpful in actual execution of the CBME programme as implemented by the MCI.