Survey: no cynicism, but teaching evaluations draw heavy criticism

While there are enough stories about this topic, is there really a problem with teaching evaluations in higher education? Almost three hundred lecturers completed the survey and the findings provide deeper insight for a pair of articles (part 1 and part 2) on teaching evaluations.

The majority of respondents say: the evaluations certainly could be worthwhile if they were conducted differently. Another 29 percent feel they are worthwhile as they now stand. More than one out of six (17.6 percent) disagree and simply find the teaching evaluations to be ‘not worthwhile’.

In total, 291 lecturers completed the survey: 122 university of applied sciences (hbo) lecturers, 166 university (wo) lecturers, and three lecturers who ticked both boxes as they apparently teach at both a wo and at an hbo. Slightly more men than women responded and there were nine people who declined to answer this question.

The survey was kept brief and limited to twelve questions so that lecturers wouldn’t have to spend too much time on it. What is striking is that many respondents provided a comprehensive answer to the final, open-ended question (the respondent’s opinion of teaching evaluations in general). On average these answers comprised fifteen words.

Hbo lecturers are somewhat more likely than their wo university colleagues to say that teaching evaluations could be worthwhile ‘if conducted differently’. Roughly half of university lecturers agree with this statement, while one out of three university lecturers is satisfied with the current state of affairs.

It should be pointed out that in the graphs, a very low number of double responses (such as answering both ‘worthwhile’ and ‘worthwhile if conducted differently’) were omitted. This explains why the percentages don’t add up to 100 percent in some cases.

Women are more likely than men to say the evaluations need to be modified. But at universities, women are significantly less likely than men to say evaluations are not worthwhile.

Lecturers were also asked to indicate their level of experience (novice or experienced) without providing any additional description. Around one out of ten (32 in total) consider themselves to be novice lecturers. These less experienced lecturers are more enthusiastic about the teaching evaluations than their older colleagues. Only one of the 32 said the evaluations were simply not worthwhile. Most (19) felt that something needs to be changed. And then there are two who gave a double response (worthwhile, but would prefer to see them conducted differently).

It was noted that novice lecturers were slightly more likely than experienced lecturers to hold the opinion that their job depended on these evaluations. The same is true for lectures with a temporary contract.

Lecturers teaching in large-scale programmes are more likely to believe their job depended on the evaluations compared to their counterparts teaching in small-scale programmes. The respondents decided for themselves whether their programme was large-scale or small-scale. It’s possible some lecturers work in a large-scale programme, but teach in a small-scale format.

Almost three quarters of the lecturers at universities believe that over the past few years, the evaluations have become increasingly important. A narrow majority of hbo lecturers also believe this to be the case. This says something about the urgency of this issue: lecturers feel that evaluations have an increasingly significant impact on their work.

You could expect that a relatively small group of respondents (just a fraction of all lecturers in the Netherlands) might exhibit bias: perhaps they only completed our survey if the subject stimulated them. We also received assistance from unions in distributing the survey, so it’s possible that slightly more activist-type lecturers were presented with the survey.

But the random sample seems to be reasonably balanced. There are almost the same number of men and women among the respondents, both at universities and at universities of applied sciences. The lecturers also seem to be evenly distributed across large-scale, medium-scale and small-scale programmes. However, the survey was extended to slightly more wo lecturers than hbo lecturers.

Additionally, certain findings are in line with expectations. For example, lecturers in small-scale programmes are not as negative in their opinions as lecturers in large-scale programmes, and novice lecturers are more positive about the evaluations than some of their more experienced lecturers. The trends are clear.

The survey was also translated into English, but there was little interest in the English version. Only six people completed the survey in English: five men and one woman, all of them wo lecturers. Of the six, two were novice lecturers and four were experienced lecturers. In light of this small number of respondents (and perhaps due to different circumstances that apply to international lecturers), these six were not included in the analysis, although the general overview wouldn’t have changed much if they had been. This means the total number of respondents is actually 297, counting these six respondents.

In the open-ended question they point out the problems associated with the surveys: very few students complete them, the results are highly dependent on the questions, and the mood of the students also plays a role. The evaluations “suffer from significant biases, are completed by students who either love or hate the course, and in general have very low response rates”, says one of the respondents.

We would like to thank all the lecturers who took the time to complete our survey.This pair of articles was made possible in part through a working grant for journalism covering topics in education provided by the foundation Onderwijsfonds Cocma.