I estimate a 36% probability that the household water chlorination rate will meet or exceed 65% in the first two years of Evidence Action’s program in Madhya Pradesh and Andhra Pradesh. While the program has the potential to make a significant impact, several factors make achieving this target challenging.

The biggest hurdle is scale. Moving from small studies—such as the Pickering et al. 2019 study and Evidence Action’s Kenya pilot—to serving tens of millions of people introduces substantial operational complexity. Supply chains, device maintenance, and quality control are all areas where large-scale implementation often falters compared to pilots. History repeatedly shows that scaling up by orders of magnitude rarely preserves the same effectiveness as smaller, controlled interventions.

Local context magnifies these challenges. Madhya Pradesh and Andhra Pradesh are among the lower-ranked states on India’s Human Development Index and face well-documented governance issues, including corruption and bureaucratic inefficiencies. For the program to succeed, state governments must ensure device maintenance, steady chlorination, and consistent engagement with communities—areas where the track record in similar settings is weak.

Behavioral barriers also add uncertainty. Sustaining a 65% chlorination rate requires widespread trust in the safety of chlorinated water, consistent use of the devices, and minimal reliance on unchlorinated sources. While Evidence Action has experience addressing behavioral challenges, scaling these interventions to tens of millions of households significantly increases complexity and variability.

That said, Evidence Action’s strengths offer some mitigation. Their proven track record in other regions and their commitment to monitoring and adaptive problem-solving provide a degree of resilience. If challenges are identified and addressed early, the program could still achieve meaningful outcomes, even if it falls short of 65%.

My forecast of 36% reflects a more skeptical view than GiveWell’s 60% prediction, driven by the well-documented risks of scaling interventions and the challenging governance environment in these states. By highlighting these risks, this forecast emphasizes the importance of robust operational planning and sustained government support to improve program outcomes.

For this particular question, I'm less concerned about potential issues with infrastructure construction scale-up than @dimaklenchin, though maintenance/service issues may have some effect. If I understand right, the question is only asking about the households that had an ILC system installed at their water access point. If there are delays in ILC installation or water pipe construction, fewer households would be served, but it wouldn't directly affect chlorination levels for those who did get ILC installations. But even without that factor, I lean towards negative for this question**.**

The previous two studies on this are limited, but give a starting point. The Evidence Action pilot in Kenya found free chlorine residuals (FCRs) in 68% of households, though rates were higher for one of the two ILC devices they used. The one in Bangladesh (Pickering et al., 2019) found 45% of households had detectable total chlorine residuals (FCRs) in their home drinking water, and don't specifically mention free chlorine residuals for household water (by definition, FCRs < TCRs). I'm not actually clear on where GiveWell is getting the 66% adherence rate they list for Pickering et al. in their spreadsheet - perhaps it's a different measure than the detectable TCR rate, or perhaps it's adjusted somehow? - but the 45% given in the paper seems more relevant to me. Given the that Evidence Action value is close to the 65% threshold and Pickering et al. is well below it, my starting estimate for this is <50%.

To get a bit more information, I wanted to think through the possible differences between those studies, and how they might apply in Andhra Pradesh and Madhya Pradesh. Interestingly, the Bangladesh study found detectable FCRs at a high proportion of the water sources (83% vs. 77% for the Kenya study), and very few households (4%) reported using alternative sources of water, so the difference from the Kenya may be related to the chlorine concentration or storage. A few relevant factors:

• FCRs decrease over time as chlorine evaporates out of stored water. In the Kenya study, water stored <24h had detectable FCRs 71% of cases, vs. 48% for samples stored >24h, and ~20% for samples stored >48h. If households in the Bangladesh study stored water for longer (or if weather conditions were more conducive to evaporation), that may account for part of the difference.
• 17% of the treatment group in Bangladesh reported boiling their water, which evaporates chlorine. In contrast, only ~5% of the households in Kenya boiled their water according to baseline surveys.
• The Bangladesh study was specifically trying to keep the chlorine concentration low to make the taste more palatable. I didn't see specific concentrations listed for the Kenya study, but if it had a higher chlorine concentration, residuals may have remained detectable for longer.

Of these, I suspect boiling and may be the biggest factor in the current study. According to the 2019-21 National Family Health Survey, 9.4% of people in rural Andhra Pradesh and 6.3% of people in rural Madhya Pradesh boiled their water (~14% in urban areas), which is slightly higher than the rates in Kenya. The reduction in FCRs during storage adds a fairly large degree of uncertainty, particularly if measurements are sometimes taken in drier seasons when evaporation may be more rapid.

GiveWell's prediction of 60% is based on two tiny data points: 1) a study with N=50 for control and treatment groups rate of 83% was a mere side point; 2) Evidence Action's pilot with N=67 in Kenia. It is hard for me to figure out from the spreadsheet exactly how many installations in Madhya Pradesh and Andhra Pradesh the 65% water chlorination rate implies, but it is clear that we are talking about tens of million people reached and the installations running in at least hundreds of thousands (or may be much more). Knowing this, my forecast is based on little more than general observations and experiences of how things tend to work in real world and, alas, it is much more pessimistic:

1. The scale up by >1000 in one swoop almost never produces the same result. As the cliche goes, quantity has a quality of its own.
2. The key to the success will be the job done by the government. The government in the states that are close to the bottom in the human development index in the country that perceived as significantly corrupt. Apologies of it sounds too dismissive, but these three elements combined don't sound to me like a great recipe for success. I hope I am wrong. In any case, in the big picture $38M is not such a huge amount of money to try and see how it goes, and learn from the experience if if does not go as expected.

I am at 42% on this question.