Case study: Physiology pre-practical quizzes

Origin

Department of Physiology and Pharmacology
Faculty of Medical and Veterinary Sciences
University of Bristol

Contact

Dr Phil Langton,

Tools used

Also see: online tests.

Objective

To address the problem of poor student preparation and engagement with lab work.

Drag-and-drop quiz illustration

Background

In common with many UK degree programmes in physiology, Bristol invests significant resource into laboratory practical (LP) teaching to ensure that graduates have appropriate laboratory skills. Increased numbers of students and institutional goals that reward activities which compete with teaching has, over a period of years, led to a reduction in the number of write-ups that the students are required to produce and as a consequence the amount of feedback they receive. LPs are mandatory and it has been suggested that LP examination questions tend not to enthuse or engage the majority of students. Thus, many arrive ill-prepared for each LP.

What was done

Accepting the premise that assessment drives learning (1, 2), the department chose mandatory on-line pre-practical quizzes (PPQs) as a solution to the problem of poor student preparation and engagement. Using the Questionmark Perception application, PPQs were constructed for nine LPs as well as two quizzes that focussed on general laboratory skills, numeracy and experimental design. The decision, taken in 2005/6, to develop the PPQs and resulted in a trial in 2006/ 7 in which the PPQs were optional, followed by our first full year in 2007/ 8, both targeting the Physiological Sciences B.Sc programme. In practical terms, PPQs become unavailable an hour before each LP but, importantly, we allow repeated access to each PPQ for one week. However, students were repeatedly made aware that the score from their first attempt would be used to calculate their running average

Outcomes

Compliance can be an issue if an educationally sound activity is made optional with the result that incentives are often sought, such as contribution towards coursework marks (3). For unsupervised, on-line quizzes the opportunity to defraud the system is obvious and there is evidence that it is exploited (3). Our strategy was to make PPQs mandatory with an overall average of 40% required for students to be regarded to have ‘satisfactorily’ completed the coursework element of our Physiological Science first year Unit. Students were reminded that failure to ‘satisfactorily’ complete course can result in the withholding of credit points. Our data, which amounts to five student cohorts and four different degree programmes, suggests this strategy was effective as compliance was excellent; completion rates range from 90% to 96% (n = 774 students). In 2007/8, only four students fell below the required average and, as they passed the end of year examinations, credits were awarded. The median score per cohort has been approximately 70% (range 64 to 76%).

Good compliance may stem from the choice of format; only 13 questions in each, with a variety of question types. Solicited student feedback included comments such as, “just the right amount of questions, did not take too much time – no boredom” and “good format and balance of knowledge and calculation problems”. A small number of questions were drawn directly from the practical schedule, to encourage students to read the schedule; something that the instruction for each test recommends. Student feedback comments were quite clear on this point, “They [PPQs] were very relevant to the practical and required reading of the practical in order to obtain more information” and “[quizzes] did encourage me to read the schedule before the experiment when otherwise I wouldn’t have, in all honesty”. Another popular feature was the provision of immediate and constructive feedback on completion of the PPQ as students reported their satisfaction with, “being given the correct answers at the end” and the “mathematical workings provided”.Some will ask, “Do PPQs help?” Anecdotal evidence from teaching staff is that students were better prepared and although there was no huge right shift of marks in the end of year examinations, there is a significant correlation between the PPQ scores and the end of year marks (Pearson; p<0.0001, r 0.52, r2 0.27) which is consistent with recent findings (4). Others will ask, “How much effort is involved?” That is harder to answer but several hundred hours spread across a year and several staff would be a reasonable estimate.

In conclusion, unsupervised, on-line PPQs can be effective tools to engage and motivate first year students to prepare for LPs.