{"id":91011,"date":"2018-10-20T10:20:48","date_gmt":"2018-10-20T08:20:48","guid":{"rendered":"https:\/\/www.erasmusmagazine.nl\/en\/?p=91011"},"modified":"2025-12-19T11:23:07","modified_gmt":"2025-12-19T10:23:07","slug":"teaching-evaluations-part-1-the-criticism","status":"publish","type":"post","link":"https:\/\/www.erasmusmagazine.nl\/en\/2018\/10\/20\/teaching-evaluations-part-1-the-criticism\/","title":{"rendered":"Teaching evaluations part 1: the criticism"},"content":{"rendered":"

Get rid of them, an overestimated phenomenon, nonsensical and arbitrary, a popularity contest, hardly informative… The answers given in a survey that was held among lecturers speak for themselves. There are some enthusiastic responses (\u201cFine\u201d), but these don\u2019t sound as passionate.<\/p>\n

\"\"

\n\t\n\n<\/svg>\n<\/span>\n\tImage by:<\/span>\n\n\t\t\t\n\t\t\t\t\t\t\tDylan Gillis<\/span>\t\t\t\t\t<\/span>\n\t<\/p>\n<\/figcaption><\/figure>

Nearly all educational programmes in higher education want to know how their students feel about the education they\u2019re receiving: was the lecturer knowledgeable, are they a good teacher, was everything well organised? It might be a worthy aim to ask students about their opinion, but a lot of lecturers take issue with this practice.<\/p>\n

Apart from the abusive tirades that they are sometimes confronted with or the criticisms regarding their appearance, lecturers mainly have a problem with the system itself. Most of them don\u2019t feel that the evaluations are useful in their current form. HOP (Hoger Onderwijs Persbureau<\/em> \u2013 Press Agency for Higher Education) researched the issue, and went in search of the criticisms and rebuttals as well as the solutions that might improve the current system of teaching evaluations.<\/p>\n

Various accounts…<\/h2>

An example? Some educational programmes ask students to comment on the level of English of their lecturers. \u201cIf a lecturer has a Dutch accent, they\u2019re harshly criticised for that \u2013 even if the lecturer\u2019s English is otherwise impeccable,\u201d says Floris van der Burg, university lecturer at Utrecht University. \u201cOn the other hand, an Italian colleague receives enormous appreciation for his cute accent. What\u2019s the point of this type of feedback?\u201d<\/p>\n

Van der Burg isn\u2019t working with dummies either. He teaches at University College Utrecht, a small-scale honours programme that selects students for their talents and motivations. He is highly critical of the evaluations. \u201cI usually get really annoyed reading the evaluations. As I\u2019m the head of the philosophy department, I also read the feedback concerning my colleagues. Sometimes a big fuss is made about a train ticket for an excursion. Or students feel that they should\u2019ve passed a subject they failed, even though they didn\u2019t attend lectures and read none of the relevant books.<\/p>\n\n\t

\n\t\t
\n\t\t\t
\n\t
\n\t\t\u201cIt\u2019s nauseating, I have witnessed lecturers being crushed by these evaluations\u201d\n\t<\/blockquote>\n\n\t<\/figure>\n\t\t<\/div>\n\t<\/div>\n

There are many stories similar to this one. \u201cWe often hear about criticism directed at anonymous teaching evaluations,\u201d says Marijtje Jongsma of academic union VAWO (Vakbond voor de Wetenschap<\/em>). She is a senior lecturer at Radboud University Nijmegen herself. \u201cSometimes students complete surveys while they\u2019re hungover after a night out. They provide a few unconnected observations and you end up having to deal with those. It\u2019s nauseating, I have witnessed lecturers being crushed by these evaluations. As the surveys are anonymous, there\u2019s no chance of defending yourself. I find this truly bizarre and wonder if it is at all beneficial to the quality of education.\u201d<\/p>\n

…in universities as well as universities of applied sciences<\/h2>
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>

Ren\u00e9 van Kralingen<\/span><\/p><\/figcaption><\/figure><\/div>

Educator and higher professional education lecturer Ren\u00e9 van Kralingen, who provides training sessions on various universities of applied sciences, is highly critical of the assessments. \u201cIn the surveys, students are forced to respond to statements such as \u2018the lecturer is enthusiastic, the lecturer is inspiring, the lecturer organises the educational content in an effective manner\u2019. If, for instance, a lecturer encounters a faulty printer just before the lecture starts, they won\u2019t be able to print their handouts. Or perhaps they have to provide an \u2018inspiring\u2019 lecture in an overheated building. There\u2019s not much you can do about those sorts of things, but they will definitely be commented on in the evaluations. Imagine that you\u2019re teaching statistics to students who have hardly finished puberty. Statistics! Especially the students\u2019 assessments of difficult subjects make for sad reading. Students have gained more and more influence and are sometimes too quick to say that something is not right. There are times when I have to refrain myself from just telling them to shut up.\u201d\u00a0<\/strong><\/p>\n

Through the years, the criticisms surrounding the evaluations have occasionally flared up, even if it was only in newspaper columns. In order to find out how widespread the support for those criticisms is, a survey was conducted among nearly three hundred respondents. The findings were quite interesting: lectures are critical, but not cynical about teaching evaluations.<\/p>\n

One of the questions was whether the evaluations were at all worthwhile. More than half of all lectures indicated that the evaluations might be worthwhile if they were conducted differently.<\/em> Thirty percent of lectures are already satisfied with the way things are going. Over one in six (17.6 percent) would prefer it if the evaluations were abolished.<\/p>\n

(Read a complete report of the findings\u00a0here<\/a>, including the differences between men and women, universities and universities of applied sciences, and more.)<\/p>\n

Career consequences…<\/h2>

The possible career consequences of the surveys are a real thorn in the sides of the lecturers. A third of all respondents say that their jobs depend on the survey results, especially if they\u2019ve only recently started teaching. Or they feel that they might not lose their jobs, but that the survey results will prevent them from being promoted.<\/p>\n

These fears are not completely unfounded. \u201cHere at University College Utrecht, you have to achieve a point average of four out of five in your evaluations,\u201d says lecturer Van der Burg. \u201cIt used to be three and a half. If a student is too passive and gets a bad mark, they will give a bad review and harm the course or even the lecturer. Management is not interested in the answers given to open-ended questions, they only look at the point average. If a lecturer is assessed for the first time and gets a 3.2 because a few students felt that they had to work too hard, you can be sure that that lecturer will not be asked to return to teach again.\u201d This might not result in them losing their job at Utrecht University, Van der Burg adds, but it will prevent them from teaching at UCU ever again.<\/p>\n\n\t

\n\t\t
\n\t\t\t
\n\t
\n\t\t\u201cLecturers just have to try and defend themselves.\u201d\n\t<\/blockquote>\n\n\t<\/figure>\n\t\t<\/div>\n\t<\/div>\n

\u201cMaybe we should try assessing the quality of the course managers,\u201d says university of applied sciences lecturer Van Kralingen scornfully. \u201cSome of them have never even taught before, but still they keep on confronting lecturers with results from written evaluations without ever thinking about the consequences. Lecturers just have to try and defend themselves. Course managers also forget to take into account the experience of lecturers: you wouldn\u2019t confront a pilot with assessment results if he or she flies a Boeing 747 for the first time.\u201d<\/p>\n

…quite apart from quality<\/h2>

Christine Teelken (Vrije Universiteit), who has researched the use of educational evaluations, responds in a more reflective manner. Lectures have a certain degree of autonomy, but they have to answer to managers; Teelken is interested in the relation between autonomy and governance in education, and evaluations are significant in that regard. Their significance has even grown in recent times.<\/p>\n

\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>

Christine Teelken<\/span><\/p><\/figcaption><\/figure><\/div>

Teelken: \u201cTen years ago or so, the lecturer would occasionally hand out some forms and that was that. This has changed dramatically in the past five years, as these evaluations are now all conducted online. These days everything is monitored systematically, and the results are sent directly to lecturers and coordinators. The number of students who complete the surveys is dwindling, however. It has gone from eighty to thirty percent. Who are the people who still take the time to complete the surveys? Dissatisfied students, that\u2019s who. They want to vent their views in the evaluations.\u201d<\/p>\n

The main issue here is that the educational quality does not correspond with the student\u2019s experience at the time of the evaluation. \u201cTo me, this is the most interesting finding from my research,\u201d says Teelken. \u201cLecturers believe that the added value of education cannot simply be measured. That value can often only become apparent years after the fact, when students realise why a specific course was useful to them. This is especially true for difficult subjects. The relationship between evaluations and quality is just very weak.\u201d<\/p>\n

Circus of control…<\/h2>

Teelken believes that degree programmes mostly introduce or expand evaluation systems because of the whole \u2018circus of external review\u2019. Once every six years, programmes have to show an external committee that they are in control of the educational quality, so that they are \u2018accredited\u2019 once more by quality controller NVAO (Nederlands Vlaamse- Accreditatieorganisatie<\/em> \u2013 Accreditation Organisation of the Netherlands and Flanders). \u201cProgrammes are highly motivated to maintain a good relationship with the committee. The committee doesn\u2019t focus on student satisfaction per se: its task is mainly to monitor the system surrounding quality control.\u201d<\/p>\n\n\t

\n\t\t
\n\t\t\t
\n\t
\n\t\t\u201cNobody is about to lose their job because of a few evaluations.\u201d\n\t<\/blockquote>\n\n\t<\/figure>\n\t\t<\/div>\n\t<\/div>\n

Teelken distinguishes between the primary feedback loop (students have justified criticisms; the lecturer makes an adjustment in his or her next course) and the secondary feedback loop (the organisation checks whether the lecturer has adjusted the course). \u201cThat second feedback loop is becoming increasingly more important because of the external reviews and it limits the autonomy of lecturers. The work pressure is already high, so lectures are not about to protest. They just feel that they have to put up with it.\u201d<\/p>\n

All in all, the evaluations don\u2019t tell the whole story and many lecturers are afraid that their jobs depend on them \u2013 especially if they\u2019ve just gotten started.
\nSo why have these evaluations in the first place? Because you can also use them in a meaningful way. In fact, rector magnificus Henk Kummeling of Utrecht University can hardly believe that anyone working at his university would be so sharp in their criticism of the use of teaching evaluations. \u201cI am familiar with the age-old question: what\u2019s the point of all those assessments? I hadn\u2019t heard responses like: let\u2019s abolish them once and for all.\u201d<\/p>\n

…or is it not all that bad?<\/h2>
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>

Henk Kummeling<\/span><\/p><\/figcaption><\/figure><\/div>

Nobody is about to lose their job because of a few evaluations, Kummeling reassures his staff. \u201cI am not trying to say that we should take evaluations with a pinch of salt, but I do think that we should put them into perspective. A lecturer\u2019s contract should never be terminated solely on the basis of educational evaluations. They are a part of broader system at best. Evaluations are simply a part of a healthy quality culture in which we ask ourselves if we are on top of things, how a course is doing, if we are on the right track. If a performance culture arises in which lecturers are punished based on evaluation results, the evaluations should seize immediately. That\u2019s not what they\u2019re for and it would be highly impractical.\u201d<\/p>\n

Kummeling would be \u2018very surprised\u2019 if a UU programme would respond to evaluations like that. \u201cIn that case we would have to move quickly and open up a conversation with the relevant dean. Students aren\u2019t the final assessors of how a lecturer functions. Nor can they assess if someone is truly competent or not.\u201d<\/p>\n

Kees Boele, president of the executive board of the HAN University of Applied Sciences, feels the same. He thinks that evaluations are very important: \u201cAs a lecturer you have an inadvertent tendency to create your own little fiefdom. Evaluations aren\u2019t always enjoyable, but they play an important part in keeping you on your toes. I don\u2019t mind admitting that I only figured that out when I started working in a commercial business. One of the company\u2019s partners said to me: \u2018Kees, you are good at your job\u2019 \u2013 I started beaming straight away \u2013 \u2018but you\u2019re not professional\u2019. He was criticising me for never asking for feedback. It was a hard lesson to learn and it still turns my stomach to think about it, but since then I have integrated assessment as standard practice.\u201d<\/p>\n

An excellent tool…<\/h2>

The position that Kummeling and Boele take wins approval from someone who, among other things, earns his crust with educational evaluations: Paul de Weger of the company Amplixs. He believes that evaluations can be very useful. \u201cWe started doing assessments for HU University of Applied Sciences Utrecht five years ago, and are now working with the Amsterdam University of Applied Sciences and Business School Netherlands. We offer bundles of different questionnaires, from which the institutes can take their pick. They might, for instance, want to focus on the enthusiasm of lecturers, their competence or on another theme altogether. A methodologist always monitors the process: what are we looking for and are we asking the right questions? Will we ask first years or students who will be graduating shortly?\u201d<\/p>\n

According to De Weger, the approach should be tailor-made, and no one should ever believe that an evaluation is completely neutral. \u201cWe are well aware that emotions might fuel the student\u2019s answers, but that\u2019s the whole point. If lecturers are usually assessed positively, the students will award significantly higher marks, as they also like the subject itself. That means that there is always a bias. On the other hand, a faulty scale can still register a trend. If my scale says that I weigh three kilos more or less than I actually do, I can still see if I gained or lost weight.\u201d<\/p>\n\n\t

\n\t\t
\n\t\t\t
\n\t
\n\t\t\u201cA faulty scale can still register a trend.\u201d\n\t<\/blockquote>\n\n\t<\/figure>\n\t\t<\/div>\n\t<\/div>\n

It does mean that we have to be precise in our analysis. Are students even capable of commenting on the competence of a lecturer in a meaningful way? They don\u2019t really know anything about that, do they? \u201cThis is true,\u201d says De Weger. \u201cOn the other hand, if you\u2019ve never been to or worked in a Michelin-star restaurant, you are still capable of passing judgment on whether the food is good or not.\u201d<\/p>\n

De Weger understands the criticism that evaluations are mainly a weapon in the hands of management. \u201cI think this is a completely valid objection. They shouldn\u2019t be used for personnel management; the results should be an indication of the quality of education.\u201d And they can really add to that, De Weger feels. \u201cThe fact of the matter is that educational institutes are quite cumbersome organisations. Change-oriented people always hit a wall in educational environments. In those kinds of situations, evaluations can be a practical tool, as they provide insights into what\u2019s going on in people\u2019s minds and hearts. It\u2019s not the most accurate method, but it does provide an indication.\u201d<\/p>\n

…and happy students<\/h2>

How do the students themselves view the evaluations? Their representatives in the Intercity Student Consultation and the Dutch Student Union don\u2019t experience any real issues and prefer to stress the positive aspects of the evaluations. Take another look at the first example in this text regarding the lecturer whose Dutch accent in English is criticised, while the Italian lecturer\u2019s accent in English is judged completely differently. \u201cI can imagine that some people would call that arbitrary.\u201d says Student Union board member John van Harten. \u201cHowever, it might also be an indication that those students cannot really follow the lectures because the accent distracts them too much. That would also be realistic conclusion.\u201d<\/p>\n\n\t

\n\t\t
\n\t\t\t
\n\t
\n\t\t\u201cPerhaps it\u2019s a sign of the times that people start saying negative things about others when they\u2019re anonymous.\u201d\n\t<\/blockquote>\n\n\t<\/figure>\n\t\t<\/div>\n\t<\/div>\n

The ISC shares the views of the Dutch Student Union. \u201cThose we represent really appreciate those evaluations,\u201d says board member Luc Rullens. \u201cWe understand the concerns of the lecturer, but we also see real value in the evaluations. Managers should look at the average and take notice of the outliers.\u201d<\/p>\n

Sticks and stones<\/h2>

What about the personal insults and the negative performance culture that seem to emerge here and there? Lecturers should just be more thick-skinned. \u201cI have also heard about examples of students writing ridiculous things in evaluations or even resorting to swear words,\u201d says Dutch Student Union member Van Harten. \u201cThings shouldn\u2019t be like that, but on the other hand, lecturers should know to ignore those kinds of comments.\u201d<\/p>\n

Rector Kummeling isn\u2019t overly bothered by these examples either: \u201cObviously, you will always get some extremely crude, disconnected remarks about a lecturer, about their appearance or whatever, but you shouldn\u2019t really take those things seriously,\u201d he opines. \u201cThose instances aren\u2019t pleasant, but you have to be aware of working in a professional organisation. Perhaps it\u2019s a sign of the times that people start saying negative things about others when they\u2019re anonymous. These comments might hurt, but should be irrelevant for the lecturers concerned.\u201d<\/p>\n

Still, a majority of the lecturers feel that evaluations could be worthwhile, but in a different way<\/em>. They are advocating change, but what is the result they have in mind? That will be discussed in the second part of this diptych.<\/p>\n

We would like to thank all the lecturers who took the time to complete our survey.<\/em><\/p>\n

This diptych was realised in part thanks to a work experience grant for educational journalism of the Cocma Education Fund.<\/em><\/p>\n

\u00a0<\/em><\/p>\n

Tomorrow<\/em> part 2: the solutions<\/em><\/p>\n\n\t

\n\t\t\t\t\t

De redactie<\/h2>\n\t\t\n\t\t
\n\n\t\t\t\n\t\t\t