News, Policies, & Trends

New York Department of Homeless Services Denies Two-Hundred Families Assistance in Name of Research

By | Oct 11, 2010

We in the social sector not only have a duty to help people experiencing homelessness and poverty, we also have a duty to figure out whether our services actually help.

These two responsibilities, to provide services and evaluate our work, have come into conflict in the City of New York’s Department of Homeless Services (DHS). DHS is under fire for conducting what is known as a randomized control trial with a sampling of the families they serve.

DHS researchers split service seekers into two groups of 200 families. The first group was provided DHS service as normal, but the second group was simply given a list of non-profits that provide similar services. This second group, referred to as a “control group”, was then told they could not receive services from DHS for the duration of the two-year study.

The experiment was designed to identify what, if any, value was added by DHS’s Homebase program. The Homebase program offers assistance with

  • Family or tenant/landlord mediation household budgeting
  • Emergency rental assistance
  • Job training and placement
  • Benefits advocacy (child care, food stamps, tax credits, public health insurance)

According to the DHS website, the department served 8,629 families last fiscal year. Therefore, the control group for this study likely amounts to about 2% of the annual caseload.

Ethical Concerns

There are understandable ethical concerns when conducting research involving vulnerable populations. Speaking to CNN, New York Department of Homeless Services Commissioner Seth Diamond said the clients who participated in the study “had an option; they knew there was a possibility [of being denied services for purposes of the study] and signed a document reflecting that”

While the participants may have signed a consent form, many question the extent to which the consent was truly voluntary.

These are families who came to DHS because they were in desperate situations. Presumably a decision to not sign the consent form was a decision to not receive help from DHS anyway, meaning there was no real choice, rather coerced consent.

This is a serious and important question for our sector, and is not necessarily as black and white as our gut reaction might suggest. Certainly the idea of denying services to someone in need seems wrong, even cruel.

But the hypothesis DHS lays out is that the non-profit referral network might be able to provide the same outcomes as DHS’s Homebase program. And what is to say that people who receive assistance from DHS don’t actually have worse outcomes than people who don’t seek help from the city?

The study seems cruel because we assume that the Homebase program works. Indeed, if the city knew Homebase was effective and denied people help anyway, that would be undoubtedly be cruel. But they don’t.

A Better Way

That said, I am not defending the actions of the New York Department of Homeless Services. My colleague at Idealistics pointed out that when evaluating social service programs, control groups are usually formed by tracking people who did not receive a particular service because the demand exceeded the supply.

What DHS did here is very different.

DHS had the means to provide their service to all who demanded it, but chose not to for the sake of their study. While this style of research is typical in the medical world, where one group of patients is administered an experimental drug then compared to another group that did not receive treatment, it’s not clear that such an approach is appropriate in social sector evaluation.

This is an important issue for the sector to consider. Where do we draw the line in evaluation?

I generally think we do not do enough to evaluate the effectiveness of our programs. Personally, I am surprised to see a situation where we have likely gone too far to evaluate the effectiveness of an intervention.

Was what DHS did unethical? Or do you think conducting a study that impacts a smaller number of families is worth it if it helps us help more people better?

Is there a better way to collect good data, but with less risk to vulnerable populations?

I think so. At the very least, I hope so.

What do you think?

Photo credit: dzingeek

2 Comments

  1. Posted Oct 11, 2013 at 3:24 am | Permalink

    If they want to test something, they need to find willing people. its not good to force someone to undergo an experiment. That's against human rights.

  2. Posted Oct 22, 2013 at 9:04 am | Permalink

    Yeah right, They need to look for people who are willing to undergo the experiment. http://reefergoodness.com/