Opinion

Innovation and Evaluation: Saluting the Homeless Shelter Pay-To-Stay Experiment

By | Sep 16, 2010

The Union Rescue Mission (URM), a homeless services provider in downtown Los Angeles’s Skid Row, recently announced it would start charging some shelter residents a fee for staying in the agency’s shelter.

I applaud URM’s decision to try something new, not because I think it will necessarily succeed in improving client outcomes, but because it is an experiment.

Experiments drive progress in the social sector just as they do in any other field, helping us to learn how to achieve better outcomes for our clients.  However, in order for experiments to drive progress, we must accurately measure results to see if they reject or confirm our predictions.

URM CEO Andy Bales, in his recent post on this site, gave three compelling reason why charging shelter residents may achieve better outcomes.

  1. Shelter residents who pay may feel more empowered and motivated than their non-paying counterparts, which would drive the paying residents to move on to other kinds of housing more quickly.
  2. Paying shelter residents also have an economic incentive to try to speed up the process of their recovery: more time at URM means less money in their pockets.
  3. People who are willing to pay for URM’s services might think they will get the most benefit from URM.  The fee helps to screen out people who are not committed to URM’s programs.

All three explanations seem like intuitive reasons that charging for services should improve client outcomes, but they may all be wrong.

Evaluation

The Center for Global Development reviewed multiple studies about the effects of user fees on health and education programs in the developing world. The studies found that people who were willing to pay for services were not necessarily those who needed them most. Furthermore, there was no evidence that paying clients put in more effort than those who received services for free.

Intuitive explanations are often wrong, which is why we need to measure outcomes carefully in our experiments.  When we look at the outcomes of clients who pay for shelter at URM, we need to make sure that we can say, to the best of our ability, “what would have happened to these clients if they hadn’t paid for lodging?”

The best way to find out is to borrow an experimentation technique from the physical sciences: the randomized control trial.  Use random chance to determine who of the eligible clients will be asked to pay and who will not.

Randomized Experiments

Random assignment means there is no characteristic that separates those who pay from those who don’t. Enforcing randomization means we can use the outcomes of the non-paying clients to determine the impact of switching to a pay-to-stay system.

Without randomizing, there is no suitable comparison group.  If, for example, URM used client incomes to determine who should pay, it may be that difference in client incomes, rather than status as paying or non-paying resident, leads to different outcomes.

URM should be praised for experimenting.  Innovation can drive our sector toward better solutions for those most hard-off in society. But in order to know which innovation is effective, we have to measure our experiments carefully.

Photo credit: University of Missouri System