What bad research looks like

Junk Science in Hawaii

This is a short review of the work done by Dominique Roe-Sepowitz of Arizona State University (ASU) and Khara Jabola-Carolus of the Hawaii State Commission on the Status of Women (HSCSW). This appeared as three separate reports alleging to shed light on sex trafficking here. The first report aimed to demonstrate a high demand for sex trafficking in Hawaii. It has been reviewed and rebutted by two distinguished UH professors.

The research in this phase involved placing a “normative” advertisement online for the selling of sex.  Quoting the research “For this study, we did not intend to look for persons trying to buy sex from a child or an obviously sex trafficked person. “ The methodology and conclusions as to numbers were criticized already, as noted. Finding a lot of callers to an ad for sex does not in itself indicate a demand for sex trafficking. A demand for sex trafficking implies men want to purchase sex from a person they believe to be victimized and not from one they believe to be consensual.  Since no evidence of abuse or minority status was included in the ad no such conclusion can logically be drawn.

  This does not mean that critics of this report deny that there is such demand for children or that there are adults being sex trafficked in Hawaii. The criticism is that the research methodology cannot provide any evidence of it.  The research was further undermined by a lack of input from sex workers who do place ads online and are not victims. Such interviewees are available and might have provided insight into the issues with callers that the researchers simply assumed were all individual sex buyer contacts that could be multiplied to come up with some statewide figure

                This second report represented some improvement over the first, as it did include some actual data of interest.  This was collected through an interview process of 22 people of which 15 were described as sex trafficking victims and 7 as “parents, close family members, or guardians of a child who was a victim of sex trafficking.”  This mixing of first hand and second hand accounts is problematic.  The statistical data should have shown the splits into parts or explained why it was unnecessary to do so.  Using hearsay is seldom a valid way of conducting an interview. There is no reason to assume the point of view of a parent, or other parties, on what has happened, will be the same as the individual themselves. 

                Most of us who are familiar with the industry as service providers, or workers, would not be surprised by the issues and harms related by those persons being interviewed.  How typical they are of all people in the industry is impossible to say as the industry is mostly hidden. However, it is seems that interviewees were selected based on their identification as victims.  So the data, which may be useful in developing policy for a subset, can’t be generalized. No effort was given to interviewing adult consensual workers and to compare those stories with the selected group.    

                As with the first report an unusual amount of space seems to be devoted to conclusions, recommendations, and opinions of the researchers.  Although some points are made that many of us in public health and social services might agree with, even in those cases it is not clear how the actual research done supports them.  Conclusions are supposed to flow from the data.  In this case they seem to have been developed in advance of the survey based on the opinions and values of the research team. The idea that prostitution is simply an evil and a crime against women is widespread within certain sections of the feminist community.  However strongly a researcher may hold to this view they should do their research in a properly neutral manner in which the conclusions are based on the data actually collected. 

                The conclusions start in an executive summary which lists twelve problems that are supposedly creating an environment where sex trafficking exists.  Two or three appear reasonable enough, but for the most part it seems a stretch that a handful of interviews with victims could lead to the broad conclusions listed.  This summary should include ideas that can be backed up with more than a quote from a victim.  Attacks on the police, the tourism industry, and progressive democrats, are included that may represent the personal views of the authors. Including these attacks with the paucity of data is problematic.  Several other points made are simply statements of philosophy or morality that are controversial in themselves, and again not related to the data presented. 

                Two troubling points made are good examples of the leap from interviewed persons to conclusions. These relate to alleged causes of sex trafficking in Hawaii. I quote “A dominant discourse that emphasizes individual choice over collective empowerment and that distracts the public and policymakers from institutional and systematic issues that lead to entry into the sex trades”. I quote “Growing pressure from pro-sex trade groups to focus on short-term solutions that do not recognize male accountability for the objectification of women and the harms that flow from dehumanizing women as both individuals and as members of a group.”  These sorts of complicated statements are unlikely to have come from the trafficking victims themselves whose interviews constituted the research portion of the report.

                Further on after the data and anecdotal statements more bullet pointed items occur, starting with what are labeled “implications” of the study.  These are nine more points of varied utility and without much relation to any evidence presented to support them. The attack on law enforcement is repeated and another on health care providers added on. 

                Subsequently a third report was published claiming 97 sex trafficking victims had been identified in a cohort of names provided by Child and Family Services. Child and Family services stated they had no role in interviewing these people and merely forwarded sealed envelopes to the researches in response to survey questions.  The executive summary in the research claims that interviews were conducted by professional social workers including Child and Family Services. So right off the bat there is a big question as to what was actually done.  Academic questioning of the ASU team on the methodology, phrasing of questioning, and how one was categorized as a sex trafficking victim have gone unanswered. 

                The responses to the criticisms from academics, health care workers, and law enforcement have not replied in any way to the critiques offered or shed any light on the methodology or other problems.  They have simply been focused on defending the goodness of the researchers and calling the critics names.  A lot of this has come from some group styling itself Af3irm.

What Proper Research Looks Like

Many advocacy groups back up their points with research and statistics. It is important to know that there is a strong difference between research done in accordance with scientific standards and that presented by the advocacy groups wishing to suppress prostitution through law enforcement.

In science a researcher starts with a hypothesis, but sets up experiments to eliminate possible alternative conclusions. It is not proper to set up a research regime aimed at ignoring other explanations for the results than those desired by the researcher.  It is unfortunately often easy to set up such faulty research.

Many studies of prostitution rely in large part on interviews. The three most important ways to contaminate such a study with bias involve the way questions are asked, the way answers are interpreted, and ignoring the relationship of the sample group to the whole group being studied. 

The way questions are asked and answered needs to take into consideration factors that may influence the interviewee to answer falsely. This bias is known as “social desirability”. Interviewing people involved in prostitution is particularly problematic since a large part of success and survival within the lives of prostitutes involves lying to johns, pimps, the police, and others. Persons working as prostitutes become adept at telling the listener precisely what they want to hear. Johns are told that the prostitute is doing well. She may claim she is working her way through college or furthering some other pro-social activity through her prostitution income.  This is done to make johns feel better and to facilitate future encounters. When arrested and facing jail prostitutes will portray themselves as victims. They will talk to authorities about the bad things that have happened to them

 An excellent example of how this problem can affect research was found by Nandita Sharma of York University in Toronto. Ms. Sharma did interviews with Chinese women in British Columbia. They had come to Canada illegally using an intermediary described by authorities as a “trafficker”. They had been caught by immigration authorities and were faced with deportation. Prior to the finalization of their deportation proceedings many had spoken about being trafficked in. They told authorities what they wanted to hear. After all possibility of remaining in Canada disappeared Ms. Sharma did follow up interviews. She found that rather than being forced and exploited by the traffickers, the women simply saw them as helpful intermediaries in their attempts to enter Canada.  For the most part they had borrowed money from relatives to pay the traffickers to help them enter Canada. The traffickers wanted all the money up front. There was no continuing debt relationship between the women and the traffickers. The women said they intended to try again. A portion of these women were working as prostitutes. They indicated that they had worked as prostitutes in China and that doing such work upon their arrival in North America was part of their immigration strategy. The lesson demonstrated by Ms. Sharma is that interviews with women when they are trying to please authorities and avoid deportation may yield a very different result than interviews done after the fact.  Questions need to be asked with reference to how the interviewee may view the consequences of their answers. Controls need to be established so that information collected solely from those facing arrest or have gone to an agency to help escape an abusive situation are not used to define individuals who are not in those situations. Without doing so a sampling error occurs which should prevent the researcher from applying the findings from the subset studied to the general population described. 

 Misinterpretation of the answers given to questions is another hackneyed way of distorting research. A few years back I read that an anti-prostitution group had research showing that johns didn’t really hire prostitutes for sex, but they actually hired them out of a desire to control women. The implication is that prostitutes’ customers are unnatural woman haters of some kind and that prostitution is promoting the acceptability of the denigration of women in society. The problem is in misrepresenting what the men meant when they answered the question. Men hire prostitutes to engage in sex that is done to their satisfaction and specification. This is no different than the way purchasers view the sellers of any product on the market. With a prostitute a man need not worry about her sexual or emotional needs. She has agreed to put those aside in exchange for his money. Of course this exhibits a desire to control. It is no more sinister than telling a barber what kind of hair cut you want.  The researchers here simply played a word game involving a variance in understanding of the word “control” between the interviewer and interviewee. This is advocacy not research.

            Another way of twisting the views of those interviewed regards prostitutes’ job satisfaction. Prostitution is a maligned and stigmatized way of life. The motivation to express regret about ones role in such a lifestyle should be obvious. However, no controls or questioning strategy is used to determine if answers are biased by issues of social desirability. Instead remarks as seemingly innocuous as “I’d rather do something else to support myself, but this seems like the only viable option” are interpreted as evidence that women are forced into prostitution. More than half the workforce could make such a statement. Certainly thousands of people in the workforce would make exactly the same claim. To equate the desire to earn money through prostitution because other money making options aren’t available or do not yield adequate income with the idea that one is forced to engage in prostitution by constant fear of violence is to ignore the realities of what is being said.  Unbiased research would include questions that probed not only for all of the perceived negatives aspects of prostitution but also for all the perceived positive aspects of prostitution.

            Actual scientifically acceptable research into the lives of prostitutes in Hawaii is scarce.  A good example is a paper by Glenn Yoshimoto done in 1983. Mr. Yoshimoto does not claim the results of his interviews of Chinatown streetwalkers can be used to draw conclusions about prostitutes outside of this population. That is an important point. He also gets almost 100% of the population he is studying to answer his questions. In his research 90% of the women interviewed supported either the legalization (19%), or decriminalization (71%) of prostitution. These findings are in stark contrast to information put out by many anti-trafficking groups. They claim almost total opposition among the prostitutes they have interviewed to any legal or decriminalized environment. While some research may find that their sample of participants were against prostitution or for it, any good researcher knows that you cannot generalize such findings beyond the specific population sample.

            In presenting these arguments I do not wish to diminish the real problems and abuse that many people face in prostitution and other related areas. I only wish to point out the dangers of making public policy based on propaganda or moralistic ideals in the guise of “research”.  Individuals are not statistics. Policy should be formulated that takes the interests and behavior of individuals into account. Individuals should be held accountable for specific and definable actions, and not be punished by virtue of belonging to a particular class or group, (i.e. prostitutes, johns, youth, traffickers, pimps, or what have you). Such labels should never be a basis for developing law. If pimps or others, abuse prostitutes that abuse should be clearly and rationally definable, (and most already are like rape, kidnapping, extortion, etc), so that such acts can be sanctioned in law and then enforced.