In a way, what behavioural economist Jan Stoop does is equivalent to a scientific candid camera show. He involves people in his experiments without them being aware of it. This time he is doing so while wearing a postman’s uniform. “Of course I had to run from a dog once.”
If you really wish to find out how honest people are, you don’t ask them to complete a questionnaire in a lab setting where an investigator wearing a white coat is constantly spying on them. Rather, you serve them a real-life dilemma in an environment where they feel unobserved. This is what behavioural economist Jan Stoop is doing – for instance by delivering supposedly misdelivered letters containing a fair bit of money, then waiting to see if they are returned to sender.
Jan Stoop is an associate professor at the Erasmus School of Economics. He attended Tilburg University, where he received his doctorate in behavioural economics in 2010. Last year he was awarded the Pierson Medal, a prestigious award granted once every three years to a young researcher by the Royal Association of Economists (KVS).
Cowboys in science
In this series of articles, Geert Maarse interviews researchers who take things just a tiny bit farther than their colleagues. Read all about academic adventures and the absolute necessity for this type of field research.
The question to be answered: is lab research as predictive as we believe it to be? In order to check whether this is the case, and if so, to what extent, Stoop is conducting experiments outside the university’s walls, without his respondents being aware that they are part of a study, which in turn means that they will not give socially desirable answers. For his most recent study, Stoop went door to door, dressed like a postman.
How did you come by the postman’s uniform?
“I had some help. It took me no time at all to get one.”
But you did not come by it the official way. Was PostNL aware of this?
“I always act in accordance with the following motto: it is easier to ask forgiveness than to ask for permission. If I had announced my plan beforehand, I’m sure someone would have prevented me from carrying it out. I once tried to discuss my study design with some government agency, only to get caught in a web of civil servants referring me to other civil servants. No one is brave enough to assume responsibility for this sort of thing.”
Why is it necessary that studies be carried out in this fashion?
“The experiment is the gold standard in science. If you can test randomly chosen people under different circumstances, you can really demonstrate a causal effect. We will first do so in a lab, but then we will test the same thing outside the lab, to see if the results are any different.”
So, are they?
“Out of the six hundred envelopes we sent, slightly less than half were returned. That was about the same percentage we found in the lab. So in this case the lab results seemed to be predictive. However, other experiments have shown that this is not always the case. Which is not to say that I’m seeking to prove that lab tests are useless. In the short term, it would be nice if I managed to do so: it would be a cool finding and it would result in the publication of a brilliant article. But it would also mean that an enormous amount of study results would have to be considered useless. As far as science is concerned, it is a good thing if lab results are found to be useful, as experiments carried out at labs are a cheap, quick and efficient way of conducting research.”
In all that time, was there never anyone who thought: Hang on, that’s not my postman?
“I had to run from a dog once. A real classic, that one. And one time there was this woman who drove behind me for several streets. Suddenly she honked her horn and was right next to me. I was scared to death. I thought she was on to me. But in actual fact, she had only come to return the letter I had just ‘misdelivered’. As soon as she was gone, I delivered that letter to the nearest house. Another thing that happened a lot was meeting real postmen, who would then give me a friendly wave, while I would panic so hard I’d nearly have a heart attack. Once I accidentally cycled past a PostNL distribution centre, where about twenty postmen were standing outside smoking cigarettes. Nothing happened, obviously, but there was pandemonium in my head.”
Your study involves people involuntarily becoming subjects in an experiment. Is that even allowed?
“That is a bit of an ethical dilemma. My general rule in such matters is that my average study subject must be better off after having taken part in my experiment. Once upon a time we conducted a study that involved us washing four hundred random, parked cars without obtaining their owners’ permission first, then asking the owners to pay us something for our efforts. I made a genuine effort to get those cars squeaky clean. Who could object to that? And the postal experiment basically saw us giving away money for free.”
This type of experiment is hardly ever conducted in economics research. Why is that?
“Apparently it’s not easy to come up with a study idea like this. John List was one of the first persons to do so. He is a major role model to me. However, judging from the number of papers published on the topic, not nearly enough of these studies are being carried out. They require creativity. And they are incredibly exciting.”
Could it have something to do with the fact that some economists seem to have forgotten that economics is a social science?
“Perhaps. Or alternatively, people may find it hard to get out of their comfort zones. Which is not to say that scientists hop from one project to the next while half asleep. It’s just that they could be conducting more exciting studies. On the other hand, economics is a broad field. My field of study happens to concern itself with social behaviour, which makes it easier to design an isolated experiment. Inflation does not have a button you can press. If your job concerns the interest rate or unemployment, you cannot just go out in the field and conduct an experiment.”
You deal with relatively simple concepts such as honesty. Why?
“It deals with very fundamental questions. I’m convinced that man is born good. However, I’m an economist, so I believe in incentives. Keep nudging a person in a certain direction, and they’ll end up doing something ‘wrong’. I force people to make decisions. They are forced to make ethical decisions, weighing their interests against someone else’s interests. I find that utterly fascinating. Partly because I find myself thinking: what would I do in this situation?”
“Keep nudging a person in a certain direction and they’ll up doing something ‘wrong’”
To me, it is somewhat reminiscent of the social psychological research carried out by people like Diederik Stapel, who published his infamous “meat eaters are dicks” paper. It is cool research whose results any newspaper would be happy to report on.
“It is certainly not my intention to carry out that type of research. In actual fact, it is better for my studies if they are not publicised. I’m in two minds about this. Of course I have an ego. It’s fun to be able to tell your friends that the BBC is interested in your study results. And since I’m funded with public money, I’m required to show society what I’m doing. But too much exposure will negatively affect my future experiments. If everyone in the Netherlands knows that I’m sending envelopes containing money – for instance because I’ve mentioned as much in De Wereld Draait Door – I will no longer be able to use that trick.”
You do all of your fieldwork on your own, from start to finish. Why is that?
“I think my work is insanely cool. There are scientists who work on many projects simultaneously, but when I’m working on a project, I go all out. That’s the way that works best for me. Moreover, I want to be sure that everything is going according to plan. And if something goes wrong, I’ll only have myself to blame.”
Allow me to bring up Diederik Stapel again… Like you, he collected all his data on his own. He turned out to be a fraud. How can we be certain that you are not making it all up?
“You know the funny thing? When I was at Tilburg University, I would have killed to be allowed to work for that man. I adored him – we clicked very well. But anyhow. Sure, I could be making it all up. However, all my steps can be retraced. I have a list of all the houses where I have delivered envelopes. You could visit all those houses and ask if anyone had taken delivery of a misdelivered envelope containing money. When I was carrying out my car wash experiment, I wrote down all the number plates involved. To make sure we weren’t washing the same car twice, but also because I knew that people fake their data these days. I just really don’t understand why people would do such a thing. To me, personally, the outcomes of my study don’t matter. I only want to know how these things work. And I’ll always be like that. I have been involved in the envelope study for three years now, and I just saw that my co-author has passed on new regression results. I’m dying to immerse myself in those.”
PostNL’s reaction by spokesperson Hanne Kluck:
“PostNL recognizes the importance of scientific research too. We find it regrettable that researcher Jan Stoop did not trust the reaction he would receive from PostNL before starting the research. We would have assessed the structure of the experimental field study of Jan Stoop.
We would look favorably on cooperation in the investigation. But we would like to have a conversation beforehand. For example, we would to like to inform our postmen who are working at this location beforehand. We would try to find a way to prevent possible confusion with our customers.”
