View/Download PDF
Medical Education
65 (
1
); 60-65
doi:
10.25259/IJPP_183_2020
CROSSMARK LOGO Buy Reprints
PDF

Blueprinting for assessment in undergraduate medical physiology curriculum

Department of Physiology, Geetanjali Medical College and Hospital, Udaipur, Rajasthan, India
Department of Physiology, BJ Government Medical College, Pune, Maharashtra, India
Corresponding author: Anish Singhal, Department of Physiology, BJ Government Medical College, Pune, Maharashtra, India. asksinghal@gmail.com
Licence
This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.

How to cite this article: Kaur M, Kurmi N, Chauhan S, Singhal A, Sharma S, Chaudhary S. Blueprinting for assessment in undergraduate medical physiology curriculum. Indian J Physiol Pharmacol 2021;65(1):60-5.

Abstract

Since long, physiology teaching has been monotonous, teacher centric, didactic type and its assessment primarily focussed on the amount of knowledge recalled by means of stereotyped written questions. A blueprint specifies weightage to various mapped topics (also known as competencies) in the syllabus. For this study, IEC approval was obtained. We aimed to prepare a blueprint for assessment in the undergraduate physiology curriculum and thereby test its utility. We found that blueprinting not only establishes a balance between teaching and learning but also improves validity, reliability and acceptability of assessments. Thus, blueprinting of undergraduate medical curriculum can help in actual execution of the Medical Council of India’s Competency-Based Medical Education programme.

Keywords

Blueprint
Physiology curriculum
Assessment
Weightage

INTRODUCTION

Physiology is the biological science concerned with the intra and intercellular properties, functions and processes of living organisms that are beautifully orchestrated for normal functioning and living. For teaching physiology to medical undergraduates, we divide the vast syllabus into systems physiology and then integrate and reintegrate topics for better teaching and student learning. Since centuries medical teaching has been monotonous, teacher centric, didactic type and its assessment primarily focussing on the amount of knowledge recalled by means of stereotyped written questions. Recently, many updates and changes have been introduced in the curriculum with a goal to improve student learning.[1] It is supposedly a competency/outcome based and an assessment driven medical education programme.[2]

A blueprint is an outline used to define the content of a given assessment. It specifies weightage to various mapped competencies/topics. It may improve teaching planning and time management. Blueprinting of syllabus provides a basis for sampling examination questions for systematic assessment of students.[1] A blueprint of the mapped physiology curriculum may thus not only outline the competency but also define their exact weightage to improve overall assessment in physiology education.[3] In this study, we aimed to prepare a blueprint for assessment in the Ist MBBS human physiology curriculum as per the MCI’s new competency-based curriculum.

MATERIAL AND METHODS

This study was conducted in the department of physiology, from December 2019 to May 2020. Institutional Ethics Approval (Ref. No. GU/HREC/2019/1709) was obtained before the study.

As per the guidelines of the Medical Council of India, medical physiology undergraduate syllabus (various chapters) was first divided into two papers, paper-1 and paper-2. Then, each chapter was considered as per the competencies defined by the MCI.[2] Learning objectives of all the competencies were decided; also division of the broader competencies and addition of certain core competencies (not already provided) were done where ever necessary. Meticulous mapping and structuring of competencies are of prime importance as high number of unwarranted competencies or low number of core competencies may mask the true weightage of the topic in the chapter and hence of the chapter in the concerned paper.

On finalisation, every competency was scored; between 1 and 3; 1 being minimum and 3 being maximum; for both, its “importance (I)” and “frequency (F).”[3] Importance of a particular competency depicts the subjective/clinical implication of the topic whereas frequency shows how many times the topic has been/should be asked in the examinations. Physiology being a basic science/pre-clinical subject following criteria[3] was used to calculate the “I” and “F.” Then, the product of I×F (IF) was calculated to find out the IF value of the given competency [Table 1].

Table 1:: Criteria for scoring a competency for I and F and calculation of IF (I×F).
Possible score per competency Importance (I) Frequency (F) Product (I×F)
1 Little physiological basic/impact or clinical implication (Good to know) Rarely 1
2 Moderate physiological basic/impact or clinical implication (Desirable to know) Relatively common 4} Can be 1,2,3,4,6 or 9
3 High physiological basic/impact or clinical implication (Must know) Common 9
Minimum 1 1 1
Maximum 3 3 9

Based on the IF values of all the competencies in a chapter, a cumulative IF (∑ IF) value of the chapter was calculated [Table 2]. Then, IF scores of all chapters were added to calculate the total IF score of the concerned paper [Table 3].

Table 2:: Method of calculation of competency/topic wise IF and weightage.
Competencies in a chapter Importance (I) Frequency (F) Product (I×F) Weightage (W)
Competency 1 1 1 1 1/20=0.05
Competency 2 2 2 4 4/20=0.2
Competency 3 2 3 6 6/20=0.3
Competency 4 3 3 9 9/20=0.45
Total of IF of a chapter (∑ IF) 20
Table 3:: Method of calculation of chapter wise weightage.
Chapters in a paper Total IF (I×F) Weightage (W)
Chapter 1 20 20/100=0.2
Chapter 2 35 35/100=0.35
Chapter 3 45 45/100=0.45
Total IF of paper 100

“Weightage” is the division of the IF to total IF. Hence, weightage of every competency in chapter (competency IF/ Chapter IF) and weightage of chapter in paper (chapter IF/ total paper IF) can be calculated, as shown in [Tables 2 and 3], respectively.

Product of weightage and maximum marks gives the maximum marks value of the topic or chapter that can be used to frame questions for formative assessment. For example, if the maximum marks of a paper are 100, then a chapter with weightage 0.2 can be asked for a maximum of 20 marks [Table 4].

Table 4:: Method of chapter wise marks distribution based on weightage. (Similarly topic wise marks distribution based on weightage can be calculated).
Chapters in a paper Total IF (I×F) Weightage (W) Marks (out of 100)
Chapter 1 20 20/100=0.2 0.2×100=20
Chapter 2 35 35/100=0.35 0.35×100=35
Chapter 3 45 45/100=0.45 0.45×100=45
Total 100 1 100

Hence, the concept of weightage is simple, which can be obtained by multiplying the weightage with maximum marks to get the maximum marks limit. Hence, based on the requirement, like during system or part completion tests, weightage of competencies/topics can be used to calculate maximum marks that can be allotted to a particular competency; whereas during the summative assessment tests like mid-semester or university examinations, chapter weightage can guide the allocation of marks to each chapter. Then, accordingly, questions can be selected.

Now, when we have calculated the marks from weightage of topic or chapter, we can appreciate that weightage can be used to separately calculate marks for theory and MCQs questions. For example, in a 100 marks paper, theory/ subjective questions carry 80 marks and objective/MCQs carry 20 marks then following [Table 5] method can be used to calculate and allocate marks.

Table 5:: Method of chapter wise marks (separate for theory and MCQs) based on weightage.
Chapters in a paper Total IF (I×F) Weightage (W) Max. theory marks (80) Max. MCQs marks (20) Total marks (out of 100)
Chapter 1 20 20/100=0.2 0.2×80=16 0.2×20=4 20
Chapter 2 35 35/100=0.35 0.35×80=28 0.35×20=7 35
Chapter 3 45 45/100=0.45 0.45×80=36 0.45×20=9 45
Total 100 1 80 20 100

The most important step in the whole process is estimation of “I” and “F” values in clearly defined competences. In the current project, total six faculty members from the department initially allocated “I” and “F” values through focussed group discussion and consensus and then substantiated it by seeking inputs/feedback (link for the feedback form: https://docs.google.com/forms/d/1vXanCAsW84ng3ZD197H3ySP_ HuQ9CcF4MlPBaOjpKuI/edit?usp=sharing) and modifications from 10 distinguished Indian Physiologists (reviewers). Only those reviewers were selected who had at least 10 years of experience in teaching physiology to medical undergraduates and had completed basic training in medical education technology. Finally, we came up with the complete blueprint of the physiology undergraduate syllabus. Their feedback is given the results below.

RESULTS

As per the methodology mentioned above, we have blueprinted the physiology curriculum and the final summary the results are shown in [Table 6].

Table 6:: Summary of blueprinted physiology curriculum (Chapters included in Papers I and II are as per the criteria of GU).
Chapter IF Paper IF Weightage Theory (80) MCQs (20)
Paper I
General physiology 40 429 0.09 7.46 1.86
Nerve muscle physiology 91 429 0.21 16.97 4.24
Endocrine physiology 89 429 0.207 16.60 4.15
Reproductive physiology 38 429 0.09 7.09 1.77
Central nervous system 153 429 0.36 28.53 7.13
Integrated physiology 18 429 0.04 3.36 0.84
Paper II
Haematology 62 324 0.19 15.31 3.83
G I physiology 54 324 0.17 13.33 3.33
CVS physiology 82 324 0.25 20.25 5.06
Respiratory physiology 59 324 0.18 14.57 3.64
Renal physiology 39 324 0.12 9.63 2.41
Integrated physiology 28 324 0.09 6.91 1.73
Practical Chapter IF Total IF Weightage Marks (80)
Clinical 178 250 0.712 56.96
Haematology 60 250 0.24 19.20
Experimental 12 250 0.048 3.84

Detailed results with “I and F” of all competencies and weightage to each competency can be accessed from this link https://drive.google.com/file/d/1JrzchRebzcDcbBhgtW5mzLuW29--e3Hh/view?usp=sharing.

All the reviewers appreciated the authors work and agreed with the concept of blueprinting for assessment in physiology and expressed their willingness to implement the blueprint in their respective institutions. Almost nine reviewers felt that the work done is a “good initiative” and “optimum” and were prepared to use the blueprint “as it is” [Figure 1].

Figure 1:: Reviewer acceptability to use authors work “as it is.”

Only 1 out of the 10 reviewers had major differences regarding some points (this is addressed in the discussion). Reviewers’ feedback on various questions regarding blueprinting of physiology curriculum based on Likert’s scale of 5 is shown in [Figure 2].

Figure 2:: Reviewer’s feedback on Likert’s scale of 5 (mean ± SD) on blueprinting.

Based on this blueprint, we had conducted the UG formative (system completion tests) exams. [Table 7] is an example of Paper I according to our university. The pattern will be used for the upcoming pre-university and university examinations.

Table 7:: Format of our university Paper-I (total 100 marks) and selection of questions based on the chapter weightage.
Section A: 20 marks MCQs (objective type)
Section B: 80 marks theory (subjective) Question No Structure Marks Chapters eligible
1 Clinical type or theory LAQ 10 Endocrine
2 3 SAQs of 5 marks each 15 General Physiology
Nerve muscle
Reproductive Physiology
3 5 SAQs/reasoning type questions of 3 marks each 15 Integrated Physiology
General Physiology
Nerve muscle
Reproductive Physiology
Endocrine
4 (optional to 2 or 3) Either as Q 2 or 3 15 (optional) General Physiology
Nerve muscle
Reproductive Physiology
Endocrine
Integrated Physiology
5 Clinical type or theory LAQ 10 CNS
6 3 SAQs of 5 marks each 15 Nerve muscle
CNS
Endocrine
7 5 SAQs/reasoning type questions of 3marks each 15 Nerve muscle
CNS
8 (optional to 6 or 7) Either as Q 6 or 7 15 (optional) Nerve muscle
CNS
Endocrine

DISCUSSION

It is said that assessment of skills or knowledge is of equal importance to teaching/learning of the skill or knowledge. Formative and summative assessments are the two most important and indispensable forms of student assessments in medical education.[4] Blueprinting provided us with the exact list of very important/high weightage topics and thus enabled us to focus and coordinate strategies of formative assessment such as conducting poster making sessions, tutorials and framing clinical scenarios for their better understanding. Blueprinting of physiology curriculum provided the weightage of every topic (based on its importance and implication), it guided in sampling and constructing relevant questions from eligible chapters for summative assessment that eventually enhanced construct relevance, student learning and improved performance. Thus, it formed a functional link between formative and summative assessments.

As a matter of fact, due to the current pandemic “COVID-19,” all the Ist MBBS students have already lost a substantial amount of their college teaching/learning time. These students are trying to cope up as much as possible from the online classes, assessments and guidance from the faculty over the phone or live classroom platforms. Thus, to enable students to plan their timetable in a smart and better way for the upcoming pre-university and university examinations, we had shared the details [Table 6] with them. Students are able to study accordingly and perform better in the online assessments. This was also shown in a study that concluded students and faculty were satisfied as blueprinting helped students to attempt examination better, whereas the faculty who validated the blueprint felt that it helped in distribution of appropriate weightage and questions across the topics.[5] This complete system also forms the basis of the credit hour system that is being used across the globe for student learning and assessment.[6]

Out of 10 reviewers that we had to substantiate/validate our blueprint, 1 had differences regarding the competency subdivisions, addition of some competencies and I and F values allotted to certain competencies. Suitable changes in the “I and F” were made where ever the authors felt that the suggested changes are relevant and acceptable. There were certain objections raised by the same reviewer that competency sub-divisions should not be made. The authors do not feel that is relevant because we have already stressed and highlighted the importance of proper mapping and structuring of competencies, as high number of unwarranted competencies or low number of core competencies may mask the true weightage of the topic in the chapter and hence of the chapter in the concerned paper. As provided by the MCI, there are some chapters such as endocrinology and CNS where many major topics are merged in a single competency. Because of this chapter, weightage of endocrinology or CNS was coming to be lesser than that of nerve muscle physiology, which is a severe flaw. Hence, we did not accept the objection and retained sub-divisions that we made.

As is obvious from [Table 7], blueprinting guided the paper setters to plan and construct questions from the eligible chapters. This not only reduced the paper setter’s selection bias but also improved the subject coverage and overall acceptability among students. Better subject coverage improved the validity and reliability into the assessment. The elimination of construct over and under representation enabled us to better assess all learning domains (knowledge, skill and affect) by incorporating specific assessment method (Theory/MCQs/PBL/Reasoning/Ethics and so on) for specific learning objective based on its weightage.[3]

Since the competencies to be assessed were weighted and assessed accordingly, it was easier to make students realise the importance of outcome-based medical education programme.

CONCLUSION

Blueprinting physiology curriculum improves validity, reliability and acceptability of both formative and summative assessments and thus establishes a balance between teaching and student’s overall learning. Blueprinting medical curriculum can be very helpful in actual execution of the CBME programme as implemented by the MCI.

Limitations

Although the complete blueprinting is done as per the prescribed methodology, there is chance that allotment of “I and F” may vary from observer to observer/teacher to teacher. Considering our work as one of the ways teachers while implementing this can make minor changes and use the blueprint accordingly in their institutes. The levels of cognitive domain as per Blooms taxonomy were not ventured on and hence can be considered as a minor limitation of the current endeavour.

Declaration of patient consent

Patient’s consent not required as there are no patients in this study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

  1. . Assessment in Undergraduate Medical Education. . Available from: https://www.gmcuk.org/-/media/documents/Assessment_in_undergraduate_medical_education___guidance_0815.pdf_56439668.pdf [Last accessed on 2020 Jun 24]
    [Google Scholar]
  2. . Competency Based Under Graduate Curriculum. . Available from: https://www.mciindia.org/CMS/wp-content/uploads/2020/01/Module_Competence_based_02.09.2019.pdf [Last accessed on 2020 Jun 24]
    [Google Scholar]
  3. , . Blueprinting in assessment In: , , eds. Principles of Assessment in Medical Education (1st ed). New Delhi: Jaypee Brothers Medical Publishers; . p. 205-13.
    [CrossRef] [Google Scholar]
  4. . Best practices in summative assessment. Adv Physiol Educ. 2017;41:110-9.
    [CrossRef] [PubMed] [Google Scholar]
  5. , , , . Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in pathology. Int J Appl Basic Med Res. 2015;5:76.
    [CrossRef] [PubMed] [Google Scholar]
  6. , . Effect of blueprinting methods on test difficulty, discrimination, and reliability indices: Cross-sectional study in an integrated learning program. Adv Med Educ Pract. 2019;10:23-30.
    [CrossRef] [PubMed] [Google Scholar]
Show Sections