Tuesday, December 3, 2013

Don't put much confidence in 'Intergroup bias toward Group X'

The paper "Intergroup bias toward ''Group X'': Evidence of prejudice, dehumanization, avoidance, and discrimination against asexuals" by MacInnis and Hodson has gotten more attention in asexual spaces than most academic articles, probably because what the authors claim to have found fits well with some people's politics. I'm deeply skeptical that their data show what they say the data show, and so I think that we need to have a serious discussion about the use of this paper in asexual politics. The paper uses scientific methods to get a politically controversial point that isn't remotely merited by the data. When conservatives do this, the paper tends to get the label "junk science." In this case, the supposed findings are more consistent with liberal biases, and so it has received less criticism. However, people of all sorts of political opinions are quite capable of bad science in the service of politics.

To start with, the title itself is misleading, probably intentionally: The title suggests the article provides "Evidence of...discrimination against asexuals." But how could they possibly make claims about discrimination against asexuals without any data from any asexuals? Probably what they have in mind is this: "Comfort with renting to and hiring a member from each sexual orientation were tapped on 11-point scales," which led to scores ranging from 0-10. Responses to this were then labelled (by the researchers) as "discrimination intentions". Both of these words are inappropriate. They didn't ask about intentions--they asked about how comfortable someone would be. Further, I believe that a major difference between discrimination and prejudice is that discrimination involves (unfairly) treating people differently, while prejudice is a matter of beliefs/attitudes/feelings (these are often related, of course). The question was about feelings, not behaviors, so the word "discrimination" is inappropriate. In the abstract, they state "Heterosexuals were also willing to discriminate against asexuals (matching discrimination against homosexuals)." Not only do they make this statement without any questions actually about willingness to discriminate, here's the actual summary of the data (possible scores range from 0-10).
Question Hetero- Homo- Bi- A-
S1-hire 9.5 8.92 8.61 9.09
S1-rent 9.45 9.06 8.76 8.84
S2-hire 9.68 8.86 8.97 9.05
S2-rent 9.63 8.87 8.87 9.00
From Tables 1 & 3.

If someone is asked, "On a scale from 0-10, how comfortable would you be with renting to an asexual?" I confess that I am not overly worried if people tend to answer, "I don't, maybe 9?" There is a significant difference in the means for the different groups, so there is a finding to be explained, but this doesn't remotely justify the claim in the abstract.

So the claims about "discrimination" are totally unsupported by their data. Next, I want to move to what the study is primarily about: attitudes. Attitudes are something they, in fact, did ask about. Now, it is true that differences in attitudes towards heterosexuals vs. bisexuals and homosexuals in research like this do correspond to how people are treated in the real world, but a VERY BIG QUESTION is whether this generalizes to groups that people have (possibly) never even heard of. MacInnis and Hodson state that one reason for study 2 was "to rule out outgroup familiarity as a potential confound." While this typically would mean, "To test whether the results of their first study are (at least in large part) due to this confound" they actually mean it as "to rule it out as a possible confound regardless of what results they end up getting." In fact, results of study 2 strongly suggest that lack of familiarity is a very large part of why the attitude thermometer scores for asexuals were lower than those of bisexuals or homosexuals. However, they try to hide this, and instead set up the absurd standard of whether lack of familiarity can completely account for the finding. (If the standard is "Does r^2=1?" I think I can rather confidently tell you that the answer is "no" without even looking at your data--or even knowing what you're studying--as long as you've got more than a handful of data points.)

If you take a close look at the paper, here are some things to notice:

Although one of the two main purposes of doing study 2 was to include "sapiosexuals" to test the possibility that familiarity accounts for why attitude thermometer scores for asexuals are so low, sapiosexuals are conspicuously missing from Table 3, showing attitude thermometer scores for study 2. They only report the data for sapiosexuals 2 pages later on p. 14, most likely so that readers don't notice that attitudes towards sapiosexuals are lower than attitudes towards homosexuals or bisexuals (though I don't know if it's strong enough, given their sample size, to be statistically significant).

Heterosexuals: 8.78
Bisexuals: 7.02
Homosexuals: 6.92
Sapiosexuals: 6.81
Asexuals: 6.45

I've never seen anyone using this study to claim that there is more (or at least least as much) prejudice towards sapiosexuals as towards homo- or bisexuals. Probably this is because if we accept that relatively low attitude thermometer scores for sapiosexuals is largely a matter of lack of familiarity (which seems likely), then this suggests lack of familiarity is also a large part of the reason for the low attitude scores for asexuals. There's also another piece of data in the paper that suggests lack of familiarity with asexuality is largely the reason for their "finding" (again, under the assumption that this is largely responsible to low attitudes towards sapiosexuals): Table 4 shows that the strongest correlate of attitudes towards asexuals was attitudes towards sapiosexuals (r=.84). Excluding alpha reliability co-efficients, this is, in fact, one of the two strongest correlations in entire table (the other being attitudes towards homosexuals and bisexuals, which are also correlated at r=.84).

However, rather than being up front about these facts, they don't draw attention to the strong correlation between attitudes towards asexuality and sapiosexuality, and they try to hide the attitude data about sapiosexuals somewhere were most readers are unlikely to compare it to attitude data for homo- and bisexuals.

They present two arguments to "rule out" familiarity as accounting for their findings. The first is by showing that although sapiosexuals have a lower familiarity score than asexuals, asexuals have a lower attitude score than sapiosexuals. In fact what this means is that [the particular measure used in their study for] familiarity cannot completely account for the attitude data towards asexuals. I wouldn't expect it to, but I'll bet that it does account for a large part of it. As a second argument, they state that "In further evidence that familiarity per se does not explain dislike of asexuals, relations between [right wing authoritarianism, social dominance orientation] or ingroup identification with asexual evaluations were relatively unchanged after statistically controlling for familiarity with asexuals (partial correlations equal −.36, −.31, −.24, and ps < .02, respectively). Prejudice-prone persons, therefore, were not more prejudiced toward asexuals as a result of mere unfamiliarity" (p.15). (The original correlations were -.34, -0.30, and -.25, respectively)

First of all, the correlation between attitudes about asexuality and each of these three wasn't all that strong compared to other correlates--in fact, in no case did it exceed the correlation between attitudes about asexuals and attitudes about heterosexuals (r = .36). Second, they never actually provide evidence that people in these groups tended to dislike asexuals. They labeled the endpoints of their attitude thermometers as "extremely unfavorable" and "extremely favorable." The mean attitude score for asexuals in the first study was 4.70, and in the second study 6.45. Possible scores range from 0-9, so 4.5 should mean neutral (or "I don't know"???) They then tell us that various things were negatively correlated with attitudes towards asexuals, so it's possible that if we made groups based on these scales, these people might had average attitudes below 4.5, but they don't tell us. The biggest problem, however, is the data given in the 2nd argument leaves open the possibility that attitude scores for asexuality among the less prejudice-prone people was correlated with familiarity. If they're trying to rule out (un)familiarity as a possible confound, it seems odd that they don't provide data on whether familiarity with X is correlated with attitudes about X, at least in the cases of asexuality and sapiosexuality. If I had to guess, it's because doing so wouldn't support their argument...

In summary, I don't think that this study provides good evidence of prejudice against asexuals. More likely, it shows us something about how people answer social psychologists' questionaires like this when asked about groups they've never/barely heard of before. That finding would much less exciting than their claim about asexuals, and I suspect that publishing pressures in social psychology encouraged them to take the route that they did.

2 comments:

Midori said...

Even aside from all of the particular flaws of the study, people are putting way too much confidence in this study. It's just ONE study. You can't get much more than a tentative conclusion, or show a need for more research, when you only have a single study.

People do the same thing with the 1% statistic. There was only that one study that showed about 1% of some population being asexual.

I'd really like to see more research trying to replicate these findings.

Lara Landis said...

From a non-academic standpoint, and instead coming from the citizen journalist viewpoint, my problem was that this city only indicates what people say they would do.

I find myself in the unenviable position of actually hoping some documented case of discrimination against an asexual takes place.

Oddly enough, I got an e-mail from a Philly.com reporter who wanted to interview me. She tried to find out my experiences with discrimination and even corrective rape. To the best of my knowledge, neither of these things has been documented against asexuals.

And of course, I feel like a horrible person for any part of me wishing that these things would break as an actual news event.