Understanding the Average Impact of Microcredit

Main article

The idea that giving small or “micro” loans to poor households in developing countries could help them escape poverty by starting or growing their own businesses, was once considered so compelling that it won Mohammed Yunus the Nobel Prize in 2006. The global microloan portfolio is now worth over 102 billion dollars and is growing yearly (Microfinance Barometer, 2017). Even beyond fostering entrepreneurship, proponents of microcredit argued that offering more loans meant offering more choices to households often left out of the formal financial sector; this could allow them to insure against risk, smooth consumption, or buy large durable items they had trouble saving enough to buy on their own. Despite these appealing theories, however, microcredit did not always seem to deliver positive outcomes for its clients. In 2001, Bolivia experienced a major microcredit crisis; by 2011 several others had occurred, most notably in the Indian state of Andhra Pradesh. Critics of microcredit blamed these crises on the lending practices microfinance institutions (MFIs) who they claimed encouraged clients to take on more debt than they could repay. Even when MFIs have the best intentions, it is possible that having many MFIs operating in one setting could create credit bubbles or overlending to borrowers who won’t benefit from the loans.

This debate motivated researchers to implement several randomized controlled trials (RCTs) and study its impact.  These experiments were designed to test whether microcredit helps poor households by fostering entrepreneurship or allowing them greater freedom in their consumption choices, or whether it could potentially harm them by creating credit bubbles or overlending (Ahmad 2003, Yunus 2006, Roodman 2012).  Seven RCTs of expanding access to microcredit services have now been completed. Yet consensus on the overall result of these studies has been hard to come by. Many in the academic and policy world noted that microfinance institutions actually differ substantially from one another in terms of the loans they offer and the places they offer them. The randomized trials had studied seven different NGOs among hundreds of MFIs, and it was unclear to what extent the results should be considered to apply across all microcredit interventions. This concern is deeply tied to the question of external  validity: the question of whether the impact of a policy may be too different across settings to ever use results in one setting to inform policy in another, or permit general conclusions about the policy as a whole.

So, what can we really learn from any single RCT, or even a set of RCTs, about what microcredit does in other places? Fortunately, it is possible to make progress on the external validity question. By analyzing the results of multiple studies in different contexts together, researchers can turn them into more than the sum of their parts. They collectively provide the opportunity to estimate not only the expected impact of a policy, but also the extent to which this impact is different across different contexts. A statistical technique called Bayesian Hierarchical modeling can be used to understand what can be learned about microcredit from these seven randomized trials.

Overall, the Bayesian hierarchical analysis found:

•  There is little evidence that microcredit generally harms borrowers as was feared by some critics, but there is also little evidence that microcredit transforms poor households into prosperous entrepreneurs.
• The effects of expanding microcredit services in different countries are surprisingly similar.
• Microcredit usually has zero effect for households with no previous business experience. While it has a large average effect for households with business experience, this effect is highly variable across settings and does not generalize.
• Economic variables such as interest rates predict variation in treatment effects better than differences in study protocols.

The bottom line is that the best existing evidence suggests, with reasonable confidence, that the average impact of these loans are small.

What Do RCTs Tell Us About Microcredit?

All seven of the different RCTs of microcredit had found that increased access to microloans at the individual or village level did not have a transformative effect on household business profits, income or consumption (Angelucci et al (2015), Attanasio et al (2015), Augsburg et al (2015), Banerjee et al (2015b), Crepon et al (2015), Karlan and Zinman (2011), and Tarozzi et al (2015)). Yet none of them found large negative effects either, and many of them found some evidence of positive impacts on some other variables. In the course of this policy debate, many scholars noted that the estimated impacts of microcredit on households seemed to be at least somewhat different across the different studies (Pritchett and Sander 2015). In that case, what can we really learn from any single RCT, or even a set of RCTs, about what microcredit does in other places? On what basis could the evidence provided by these randomized trials be used to conclude anything about microcredit in general, that we could expect to replicate in another setting?

It turns out that once we have several studies of a similar policy intervention across different contexts, we can formally measure how much the effect varies across contexts using the data from all the studies taken together. The key idea is that the differences we observe in the estimated effects from the existing set of studies that we have provides us with a good signal of the potential differences in effects across a broader set of places we have not studied. After all, before a study was done in, say, Ethiopia by Tarozzi et al (2016), policymakers in that context were in the same boat that policymakers in many other countries without microcredit studies are now. Understanding how well the effects in the existing studies predict each other can give us information about how well they might be able to predict the effect in future settings.

Bayesian Hierarchical Models

The key challenge for estimating uncertainty in effects across trials is that this variation is confounded by another source of variation: the usual sampling variation we always have within trials. If researchers could go back to the same time and place and run their study again, they would get a different estimate of the treatment effect, both because they randomly sample individuals from the population and because they randomly allocate those individuals (or villages) into treatment groups who got more access to microcredit, and control groups who did not. That variation adds to the genuine variation in effects across sites – so the estimates in the literature will look more different from each other on average than the underlying effects really are.

To get a good estimate of the variation in underlying effects, we need a way to correct for the sampling variation. This can be done using a statistical framework called Bayesian hierarchical modeling (Gelman et al 2004, Rubin 1981, Efron and Morris 1975). The core idea is to build a statistical model that contains multiple levels arranged in a hierarchy: first, there is the data level, where the sampling variation is located within each study. But on top of that, there is the cross-study or “general” level, where the variation across studies is located. The hierarchical model builds structure on both levels that splits out the variation and corrects the general level estimates for the sampling variation at the data level within-studies (Chung et al 2013, Chung et al 2015).

The Bayesian hierarchical method has some other beneficial effects: primarily, it increases the precision of the estimates in each study by borrowing information across studies. Studies in many areas of social science, as well as in medicine and agriculture, often lack precision because it is very costly to administer the intervention or treatment to many people and then to collect follow-up data on both the treatment and control groups: this leads the researchers to collect information on fewer people than they would ideally like to, and that can make the estimates of the effects of the intervention very imprecise or uncertain. Also, many economic variables of interest like business profits are inherently noisy because they are determined by many random factors, and this can lead to even more imprecise estimates. By analyzing everything together, the Bayesian hierarchical model borrows information across studies and improves the study’s power to estimate the effects of the intervention.

I performed a Bayesian hierarchical analysis of the microcredit literature (Meager, 2019), to estimate how much the impact of microcredit varies across studies, and how uncertain we should be about the effect of expanding access to microloans in new settings. I examine the impact of access to microcredit (the “treatment effect”) on household business profit, expenditures and revenues, to evaluate the initial claim that microloans allow poor entrepreneurs to grow their businesses and increase profit (Yunus, 2006). Yet households may benefit from microcredit in other ways, such as increased total consumption, shifting to spending on consumer durables, or decreasing spending on ”temptation” goods due to greater hope for the future (Banerjee, 2013); I examine these variables as well.

The expected impact of microcredit is small and uncertain

In general, the average or expected impact of microcredit on household economic outcomes is small and uncertain. The sign of the estimated average impact suggests beneficial effects on all outcomes, which is encouraging. But there is a reasonable probability of an essentially zero impact due to uncertainty both within and across studies. Figure 1 shows these predicted effects of microcredit on each outcome, and the uncertainty intervals for each, as well as the predicted effect computed using classical meta-analytic techniques which do not account for variation in effects across sites in estimating uncertainty. These classical methods tend to be quite overconfident in the microcredit setting, highlighting the importance of accounting for cross-study variation. Overall, while there is little evidence that microcredit generally harms borrowers as was feared by some of its critics, there is also little evidence of an effect large enough to transform poor households into prosperous entrepreneurs as was initially claimed by its advocates.

The impact of microcredit is similar across contexts

I also directly estimate the variation in effects across studies for each outcome, and find that the differences in effects across studies is smaller than previously thought (Pritchett and Sander 2015, Vivalt 2016). By applying the Bayesian hierarchical method and separating sampling variation from variation across countries, I find that the variation across countries is generally moderate. On average, across metrics and outcomes, approximately 60 percent of the initially observed variation in microcredit treatment effects was actually just within-study sampling variation. The bottom line is that the effects of expanding microcredit services in different countries are quite similar. The results we see here are likely to give a reasonable indication of the future effects of microcredit in other countries. So while there is still some moderate variation in treatment effects, these RCTs appear to be reasonably externally valid. That does not mean however that we could have ignored the heterogeneity in effects across sites: the uncertainty we should have about the effect in the next site is indeed larger than it would be if we assumed the effects were all the same, as figure 1 showed.

Some of the authors of the original studies thought that having previous business experience might be a good indicator that a borrower would be able to use new lines of credit in a productive way. Because I have all the data I can extend that analysis to all sites. As shown in figure 2, I find that microcredit usually has precisely zero effect for those households with no previous business experience. By contrast, the treatment effect on households with business experience is large on average, yet more uncertain and with more variation across sites.

Although the variation in effects across country contexts is not large, there is already no shortage of academics and aid pundits claiming to understand the “key factor” that causes microcredit to have different effects in different settings. But there are two big problems with such claims. First, the correlation between any given contextual factor and the treatment effect size can’t be interpreted causally because many other unobserved factors could be driving the patterns we see. Second, the correlations are computed from seven studies and there are more than seven important contextual variables such as interest rates, loan size, borrower targeting strategies, researcher study design decisions, and many more that varied across the MFIs in the studies. In other words, there are more possible explanations than data points. We must be very cautious about what we can learn in this setting. The best one can do is analyse the covariates using methods that constrain the model to cut down the number of potential explanations. I apply a method called a Ridge regression to assess the relative predictive power of these contextual variables. I find that economic variables such as interest rates predict variation in treatment effects better than differences in study protocols. This suggests that the heterogeneity we do observe reflects genuine differences in effects across settings, although the analysis is somewhat speculative.

Where does microcredit go from here?

Having seen that microcredit does not provide the dramatic, transformative effects claimed by either side of the debate, the next question is: why not? Perhaps part of the answer lies in the terms of the loans: they have to be repaid frequently, often at weekly intervals, so borrowers cannot use much of their loan for risky business ventures which may only pay off in the longer term. In addition, microloans often have very high interest rates to cover the costs of administration, which may mean that the loans are not as desirable for borrowers as the MFIs had hoped: take-up was low in most of the experiments.

Another possibility is that it takes time to build relationships between loan officers and borrowers, so perhaps the initial loans are not made to the people who would be able to use them best; in that case perhaps long-term followups will find something more encouraging. However, it is also possible that credit simply is not the most important constraint that poor households face: after all, they do have extensive informal credit networks amongst themselves, but often do not have other important potential inputs to economic growth such as infrastructure, roads and healthcare. And even if credit access is indeed an important constraint, it is possible that solving one constraint is not enough to help households escape poverty; this thinking has motivated other “big push” or comprehensive interventions such as the BRAC Graduation program.

Despite these possibilities, the best evidence we have suggests with reasonable confidence that the average impacts of these loans are small. While the environment in which the evidence will be eventually be used for making policy decisions is never exactly the same as the environment studied, in the case of microcredit the difference in effects across studies is moderate. There is still some uncertainty about the impact of microcredit in future settings, but the evidence we have suggests it would be beneficial to seek alternative approaches to improve the lives of poor households in the developing world.