The RCT Revolution and Anti-Poverty Policies

Though first carried out in medical studies to assess the effectiveness of a new drug, RCTs are now a popular impact evaluation strategy in the social sciences. Through this paper, we begin by introducing what an RCT is and why RCTs are essential to poverty reduction policies. We then go on to access the existing literature on the ongoing RCT revolution, emphasizing anti-poverty policies and concluding with the limitations and way ahead for RCTs. 

What are RCTs? 

RCTs or Randomized Control Trials follow an evidence-based approach where a form of impact evaluation in which the population receiving the policy intervention is chosen randomly from the eligible population. The control group is also chosen at random. It tests the extent to which specific, planned impacts are being achieved (UNICEF). 

RCTs have revolutionized Developmental Economics and poverty reduction policies over the last few decades and are promoted as the global standard of impact evaluation by their proponents like Nobel-winning economists Esther Duflo and Abhijith Bannerjee. These evaluations have been on topics as diverse as the effect of school inputs on learning or vaccination incentives, have attempted to answer critical policy questions. As it involves randomization, it is nearly free of human biases and can estimate the most unbiased impact of any policy. It has revolutionized chiefly the field as it offers quantification of the effect of various policies so that funds can be allocated as efficiently as possible, making them the preferred tool of policymakers in many parts of the world. Key actors of poverty reduction have adopted results from RCTs, both public agencies (e.g., UNICEF) and private donors (e.g., Bill and Melinda Gates Foundation).

Literature Review

Any attempt at drawing a causal inference question such as “What is the causal effect of education on contraceptive usage?” requires answering essentially counterfactual questions: How would those who were not exposed to the program have fared in the presence of the program? We can not directly answer this question, as the participant was either exposed or not exposed to a particular treatment then. Duflo, Glennerster and Kremer tell us that RCTs solve exactly this and claim that when a randomized trial is correctly designed and implemented, it provides an unbiased estimate of a program’s impact and that this estimate is also internally valid.  

Advocates for RCTs like Ben Goldacre (2013) argue that the method is free of ideology, political stances and even theoretical assumptions about the nature of poverty and can estimate a statistically unbiased impact, unlike other methods of impact evaluation. This means that RCTs can identify exactly which program is responsible for which outcome and efficiently utilize funds and resources. Deaton and Cartwright (2017) point out that RCTs do not only ascertain whether a program works or not; they also provide a quantification of its impact. This allows making several programs tackling the same issue comparable, by their cost-effectiveness ratios and has wide-ranging impacts on public policy.

While RCTs seem to have revolutionized the field of Developmental economics, they are not without critics. Concerns arising from RCTs range from ethical concerns of assigning randomization, the production of collateral damages for a greater good, and the instrumentalization of people. Barrett and Carter (2010) provide an insightful and methodologically-oriented discussion on these ethical concerns involved. They claim that the enhanced agency an experimenter enjoys randomizing one or more feature(s) of subjects’ world heightens the opportunity to harm subjects. They point out that researchers seem not to recognize that subjects’ informed consent does not absolve RCTs to steer away from any injury subjects suffer as a direct result of an intervention. 

Bédécarrats, Guérin and Roubaud (2015) think that RCTs are not as unbiased as they claim to be and are subject to various sampling errors. They claim that field partners may refuse to randomly sample beneficiaries for ethical reasons, thereby forcing research teams to opt for selection methods like alphabetical sampling. Yet this could potentially generate an additional bias as many resources are also allocated alphabetically (Deaton, 2010). Quentin & Guérin (2013) who conducted a micro health insurance project in Cambodia, find similar biases. Participants for the randomized trial were drawn from a lottery at a village meeting. They quickly realized that there is a high chance that the people who attend village meetings are more curious and open to innovation, closer to the village leader, more socially integrated, in poorer health, etc. Such uncontrolled factors could have potentially biased the results of randomized trials. 

One of the major concerns critics raise is the lack of external validity of RCTs. Scholars like Peter Dorman (2019) point out that the literature on experimentally designed conditional income transfers, for instance, where every new study, with a new location or time period, seems to alter the bottom line of what works and how. Deaton and Cartwright (2017) seem to be of a similar opinion that RCTs are “not automatically simply generalizable”. Similarly, prominent RCT skeptic, Angus Deaton argues that there is no compelling evidence to prefer unbiasedness championed in RCT over other statistical qualities, particularly, precision. 

However, despite the controversies surrounding it, RCTs have been claimed as the best way to evaluate the impact of a poverty-reduction program in major development institutions. It strives to create anti-poverty policies through standardization, which may be at the cost of diversity of interventions, claims Abdelghafour (2017). 

Upon analyzing the literature surrounding Randomized controlled trials and their impact on anti-poverty policies there is no doubt if they have transformed public policy implementation around poverty. It provides the most unbiased statistical results through randomization and is truly the golden standard that impact evaluation requires. It ensures that only policy with maximum cost-effectiveness is implemented, allowing policymakers to efficiently utilize funds between poverty eradication policies. However, it is a little worrisome that RCTs have been claimed to become too pervasive, rather than being one good tool in a toolbox. Critics of RCTS see using RCTs as steering economists towards asking small questions instead of big ones, like the root causes of poverty. This is a critical misunderstanding that needs to be cleared among economists alike. While RCTs may be helpful tools, they also face concerns such as external validity, their results need to be replicated using other impact evaluation methods in other locations, before implementation of the policy. 

Bibliography

1. Abdelghafour, Nassima. “Randomized Controlled Experiments to End Poverty?” Anthropologie & développement, no. 46-47 (December 1, 2017): 235–62. https://doi.org/10.4000/anthropodev.611. 

2. Piper, K. (2019, December 11). The Nobel went to economists who changed how we help the poor. but some critics oppose their big idea. Vox. Retrieved December 18, 2021, from https://www.vox.com/future-perfect/2019/12/11/20938915/nobel-prize-economics-banerjee-duflo-kremer-rcts 

3. UNICEFInnocenti. (n.d.). Randomized controlled trials (RCTS). UNICEF Innocenti. Retrieved December 18, 2021, from https://www.unicef-irc.org/KM/IE/impact_7.php 

4. Duflo, E., Glennerster, R., & Kremer, M. (2006, December 22). Using randomization in Development Economics Research: A Toolkit. NBER. Retrieved December 18, 2021, from https://www.nber.org/papers/t0333 

5. Hariton, E. (n.d.). Randomised controlled trials - wiley online library. Retrieved December 18, 2021, from https://obgyn.onlinelibrary.wiley.com/doi/pdfdirect/10.1111/1471-0528.15199 

6. Goldacre, Ben. (2013). BUILDING EVIDENCE INTO EDUCATION. 10.13140/RG.2.1.5101.8967. 

7. Deaton, Angus; Cartwright, Nancy (2017). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, (), S0277953617307359–. doi:10.1016/j.socscimed.2017.12.005 

8. Barrett, Christopher and Carter, Michael, (2010), The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections, Applied Economic Perspectives and Policy, 32, issue 4, p. 515-548, https://EconPapers.repec.org/RePEc:oup:apecpp:v:32:y:2010:i:4:p:515-548.

9. Evans, D. K. (2021). Florentbédécarrats, isabelleguérin, Françoisroubaud (eds.) randomized control trials in the field of development: A critical perspectiveoxford university press, 2020, 448 P., $100.00. Population and Development Review, 47(2), 551–554. https://doi.org/10.1111/padr.12410 

10. Quentin, A. & Guérin, I. (2013). Randomized Controlled Trials Tested in the Field: The SKY Health Microinsurance Project in Cambodia. Revue Tiers Monde, 213, 179-200. https://doi.org/10.3917/rtm.213.0179

11. Abdelghafour, N. (2017). Randomized controlled experiments to end poverty? Anthropologie & Développement, (46-47), 235–262. https://doi.org/10.4000/anthropodev.611