AI leads to flood of research funding applications, NWO to revisit lottery system

The rise of ChatGPT and other chatbots is causing a sharp increase in the number of project proposals submitted to research funders worldwide. It is breathing new life into the debate over awarding grants by lottery.

Many scientists depend on grants they apply for from research funders to advance their careers. As a result, some take a scattergun approach: they submit as many project proposals as possible and see if any of them succeed.

The tide turns

But what happens now that AI makes it easier to write grant applications? At the European Research Council (ERC), the number of applications is rising explosively, and the same trend can be seen elsewhere.

This puts pressure on the experts who have to assess the quality of applications. They often do this in their spare time. Some reviewers are also using AI themselves. Applicants then hide secret messages in their work to protect themselves against chatbot assessments.

This raises the question of how worthwhile any of this is. Perhaps science needs to find other ways of distributing grants. In the newspaper Trouw, Nijmegen-based assistant professor Serge Horbach suggests that some of the available grants should be awarded by lottery.

An old debate

It is an old debate that keeps resurfacing. Long before the arrival of AI, critics were already calling it a waste of time for scientists to submit project proposals when most of them would not receive funding.

Research funders understand this criticism and are trying to improve the process. At NWO, applicants now submit a brief application first and only have to write a full proposal if they make it through the first round. But if it becomes easier to submit an application, more and more researchers will give it a try, and little will have changed.

There is also a matter of principle. Critics say the differences in quality are not as clear-cut as research funder NWO makes out. There may often be no meaningful distinction to draw, and AI is also eliminating most of the careless mistakes that made it easy to tell whether someone had devoted enough attention to a proposal.

Could a lottery be the answer? Supporters say assessors can certainly weed out the best and worst proposals, but that the decisions in the middle group are already arbitrary. A lottery would be fairer, or just as fair, and would save time.

Researchers at the Rathenau Institute argued for it as early as 2008, and members of the Young Academy did so in 2014. In 2018, the Netherlands Bureau for Economic Policy Analysis recommended the same. The idea is certainly not confined to the Netherlands. In 2019, the journal Nature published an article about lotteries at various science organisations. Two years ago, UMC Utrecht allocated by lottery so-called starter and incentive grants.

At Dutch research funder NWO, supporters of lotteries have found little support over the years. NWO is waiting to see how experiments in other countries fare, said then-chair Stan Gielen in 2017. A lottery is a lazy solution, argued current chair Marcel Levi in 2022. NWO has occasionally left the door ajar, but has never embraced lotteries.

Taking another look

The organisation will nevertheless look at the issue again soon, a spokesperson says. This will happen as part of a programme on ‘research on research’. This includes studying the impact of research, as well as how research quality is assessed.

The rise in AI-generated research proposals does not yet seem to be having much effect in the Netherlands. “We expect that storm to come”, the spokesperson says. “But for now, AI is not the deciding factor.”

The number of applications to NWO is rising, but there are other explanations for this too. The previous government, for example, cut funding for science, forcing researchers to look more often for external funding.

In practice

But awarding grants by lottery may sound easier than it is. A full lottery is unlikely: research funders will want to reward the best proposals and reject the weakest ones. That means they would still have to scrutinise all the proposals. A lottery might not save much time among the middle group after all.

It could, however, dispel the impression that there is a meaningful distinction to be made – a distinction that is becoming even harder to draw now that researchers can quickly put together a reasonable proposal with AI.

Will NWO ever go ahead with it? The organisation wants to reward the best research proposals, so it would be putting its own raison d’être at stake if it called that mission into question. But who knows what will happen if the storm of AI-generated proposals reaches NWO too.