Abstract Background Standard setting is one of the most challenging aspects of assessment in high-stakes healthcare settings. The Angoff methodology is widely used, but poses a number of challenges, including conceptualisation of the just-passing candidate, and the time-cost of implementing the method. Cohen methodologies are inexpensive and rapid but rely on the performance of an individual candidate. A new method of standard setting, based on the entire cohort and every item, would be valuable. Methods We identified Borderline candidates by reviewing their performance across all assessments in an academic year. We plotted the item scores of the Borderline candidates in comparison with Facility for the whole cohort and fitted curves to the resulting distribution. Results It is observed that for any given Item, an equation of the form y ≈ C.e Fx where y is the Facility of Borderline candidates on that Item, x is the observed Item Facility of the whole cohort, and C and F are constants, predicts the probable Facility for Borderline candidates over the test, in other words, the cut score for Borderline candidates. We describe ways of estimating C and F in any given circumstance, and suggest typical values arising from this particular study: that C = 12.3 and F = 0.021. Conclusions C and F are relatively stable, and that the equation y = 12.3.e 0.021x can rapidly be applied to the item Facility for every item. The average value represents the cut score for the assessment as a whole. This represents a novel retrospective method based on test takers.Compared to the Cohen method which draws on one score and one candidate, this method draws on all items and candidates in a test. We propose that it can be used to standard set a whole test, or a particular item where the predicted Angoff score is very different from the observed Facility.

Value (mathematics)

10.21203/rs.3.rs-38272/v2

Cite

Citations (0)

Naloxone does not inhibit the injury-induced attenuation of the response to severe haemorrhage

Mitsuo Ohnishi Marina Sawdon Emrys Kirkman R. A. Little

Source

Cite

Citations (0)

A study to investigate the effectiveness of SimMan® as an adjunct in teaching preclinical skills to medical students

BMC Medical Education (2014)

Meenakshi Swamy Marina Sawdon Andrew Chaytor David R. Cox Judith Barbaro-Brown

Following the GMC's report on Tomorrow's Doctors, greater emphasis has been placed on training in clinical skills, and the integration of clinical and basic sciences within the curriculum to promote the development of effective doctors. The use of simulation in the learning environment has the potential to support the development of clinical skills in preclinical medical students whilst in a 'safe' environment, but currently there is little evidence on its effectiveness.Seventy nine year one medical students were divided into two groups. A pre-test was conducted by both groups, after which one group performed chest examination on their peers whilst the other group examined the SimMan® manikin. Both groups subsequently performed a mid-test and crossed over so that the group that conducted peer examination examined the manikin and vice-versa. Finally a post-test was conducted. The students were scored for formative feedback whilst performing examinations. Students completed a feedback questionnaire at the end of the session. Data were analysed using a one-way ANOVA, independent t-test and 2- proportion Z test.When the two groups were compared, there was no significant difference in their pre-test and post-test knowledge scores, whereas mid-test knowledge scores increased significantly (P < 0.001), with the group using SimMan® initially scoring higher. A significant increase in the test scores was seen in both groups after using SimMan® (P < 0.001). Students' confidence increased significantly in differentiating between normal and abnormal signs (P < 0.001). Students highly valued the use of the manikin in the session with 96% of students reporting that it enhanced their learning experience.The study demonstrated a significant improvement in the students' knowledge after examining the manikin and students also reported an increase in their confidence. Students' feedback was generally very positive and they perceived the incorporation of manikin-based examinations useful to prepare them for future patient contact. The use of simulation in this context supports an integrated learning approach when used as an adjunct to peer examination, and can benefit the acquisition of clinical skills in preclinical medical students.

Adjunct

10.1186/1472-6920-14-231

Cite

Citations (34)