Details below, but lessons to be learned from the University of Vermont (UVM) somewhat failed intervention:
- always test on a small sample before scaling behavioral interventions
- always plan for ways to evaluate the effects of the BI before implementation: you shouldn’t be wondering after the fact how you could somehow evaluate your intervention
- unforeseen consequences will happen: make every effort to anticipate the obvious negative ones that can materialize and look for them when testing on a small sample
UVM wanted to cut the volume of waste from plastic bottles generated on the campus. So they banned the sale of bottled water and instead “spent around $100,000 to set up 75 new or retrofitted filling stations for reusable water bottles at the time of the ban, including outside every dining facility.” (link).
It seems no evaluation methods was thought of when the ban went into effect and no experimentation was done either. But nutrition researchers at UVM then evaluated the consequences of the ban by studying beverage shipments to the campus both before and after the ban1.
What they found was that after the ban of bottled water, the shipment of bottled beverages increased significantly and it seems people were consuming more bottled beverages than before. Not only that, but since bottled water was banned, the beverages they consumed were much less healthy.
When the bottled water ban went into effect, per capita number of bottles shipped to campus increased, meaning it did not decrease the number of overall bottles ultimately being discarded. There was then an increase in the number of beverages shipped to campus rated unhealthy by the Nutrition Environment Measures Vending Survey, while healthy beverages sent to campus decreased.
“Because it appears that many bottled water consumers instead decided to purchase other bottled beverages, the best result, nutritionally, would have been for them to select calorie- and sugar-free options, such as seltzer, unsweetened tea, or diet soda,” the professors said in their study’s conclusions. “However, the data suggest that some consumers increased their consumption of calorically sweetened drinks, such as soda and sports drinks, which could add to their liquid calorie and added sugars consumption, thus increasing the risk of weight gain.”
“This … suggests that many consumers who previously drank bottled water replaced bottled water with sugar-free or sugar-sweetened bottled beverages,” the researchers added.—Huffington Post
The failure to accurately take into account the power of salience and the status quo bias seems to be the main reason why this intervention generated negative net results.
Yes, people will not walk a few meters just to get water, but instead will buy a bottle of whatever is in front of them. This is not a surprise but UVM may not have been advised by people familiar with behavioral principles. They should have tested their assumptions however, and observing people at a dining station with no bottled water available would have surely made them aware of the flaws in their design.
Always test on a small sample
Behavioral Interventions are powerful tools that can yield huge impacts even when the changes seem trivial. But their impact is never 100% predictable, even for the best academic or consultants. We can point to probable effects, but more often than not, what we see when testing on a small sample is not what was anticipated. Then it’s a matter of adjusting and testing further to reach the target goal.
UVM should have tested first on 1 dining hall, or a few locations and analyzed the data to determine the effects of a ban. It’s easy to do and the only cost is to wait until they can reap the rewards (PR, etc.) of announcing a full scale implementation.
Always define how you will evaluate the progress you made
When doing any BI, you should plan the intervention itself to generate the data needed to evaluate it. Yes, it’s often possible to do an approximate evaluation after the fact using data generated by the current processes. You should not be waiting to look at this data however, but you should be monitoring any effects continuously to correct course if needed or adjust for optimal efficiency.
Look out for unforeseen consequences
Use scenario planning and data from your small sample tests to try and identify negative externalities of your intervention before implementing them at a full scale. That will save you time and costs.
- Elizabeth R. Berman and Rachel K. Johnson. The Unintended Consequences of Changes in Beverage Options and the Removal of Bottled Water on a University Campus. American Journal of Public Health: July 2015, Vol. 105, No. 7, pp. 1404-1408. doi: 10.2105/AJPH.2015.302593 ↩