Offering chocolate cookies found to improve evaluation scores

For many working in higher education, end-of-course-evaluation of teaching (SETs) surveys have become a central part of the academic year.

SETs are widely used and taken extremely seriously by faculties when it comes to evaluating a professor’s teaching prowess. The evaluations can even factor in decision making for academic recruitment, distribution of school funds, and changes made to educational curricula.

As popular as the evaluations are, there are some criticisms that teacher evaluations may not be an accurate assessment of the quality of course material.

Now, a new study found that SETs can be easily manipulated and that simply rewarding students with chocolate cookies can result in higher evaluations.

Researchers from the University Hospital of Muenster in Germany conducted the study.

The researchers wanted to examine the validity of SETs and see if a simple intervention by the teachers being evaluated could influence results.

To test this, the research team split 118 undergraduate third-year medical students into 20 groups.

During the first four sessions of an emergency medicine class (a typical part of a medical student’s curriculum) ten groups were given 500 grams of chocolate cookies. The other ten groups represented the control groups and were not given any sweets.

After the class sessions, all the students completed a 38 question evaluation survey which covered the instructor, course contents, teaching materials, and a self-assessment.

Those groups that had been given cookies during class gave their teacher higher evaluations than those groups who had not gotten any cookies. The groups with the chocolate cookies also rated the teaching materials better and overall scores were higher than the control group.

These worrying results show how easily manipulated course evals can be, and suggest that they may not be the best way to measure course materials and content.

In fact, the study matches other research that has found environmental factors, the attractiveness of the teacher, and grading leniency can all influence SETs results.

“Consequently, a higher student satisfaction does not necessarily correlate with a higher quality of education,” the authors note. “These findings question the validity of SETs when used to make widespread decisions within the faculty.”

Teachers may want to take this study to heart and find ways to control their SETs with treats or easy grading, particularly when SETs play such an integral part of faculty decisions.

“Students’ end-of-course feedback and evaluation of teaching and teachers (SET) has become a standard tool for measuring ‘quality’ of curricular high-grade education courses,” said Manuel Wenk, a leader of the research. “The results of these evaluations often form the basis for far-reaching decisions by the academic faculty, such as changes to the curriculum, the promotion of teachers, the tenure of academic appointments, the distribution of funds and merit pay, and the choice of staff. So this is a totally inadequate tool to measure quality if you can mess with the system that easily!”

—

By Kay Vandette, Earth.com Staff Writer