When contact changes minds

"When contact changes minds: An experiment on transmission of support for gay equality" is an article by then-UCLA political science graduate student Michael LaCour and Columbia University political science professor Donald Green. The article was published in the academic journal Science in December 2014, and retracted in May 2015 after it emerged that the data in the study had been forged by LaCour.^[1] The article purported to demonstrate that people's minds on the issue of gay marriage could be changed by conversations with gay canvassers, but not with straight canvassers.

Study

The authors investigated whether gay (n = 22) or straight (n = 19) messengers were effective at encouraging voters (n = 972) to support same-sex marriage and whether attitude change persisted and spread to others in voters’ social networks. The results, measured by an unrelated panel survey, show that both gay and straight canvassers produced large effects initially, but only gay canvassers’ effects persisted in 3-week, 6-week, and 9-month follow-ups. LaCour and Green (2014) also found strong evidence of within-household transmission of opinion change, but only in the wake of conversations with gay canvassers.^[2]

The article has been downloaded over 111,000 times and ranks in the top 5% of all research output, as scored using Altmetrics.^[3]

The study's findings made international headlines and received wide media attention including in The New York Times,^[4] The Washington Post,^[5] The Wall Street Journal,^[6] The Economist, The Los Angeles Times, and This American Life.^[7] The study attracted widespread attention, in part, because it seemed to challenge the conventional wisdom and scholarship about social persuasion that suggests that people tend to maintain their points of view, sometimes regardless of what they read or hear to the contrary.^[8]

The Yes Campaign in Ireland stated that LaCour and Green (2014) “provided a template" for campaigners to use one-to-one contact and first person accounts to reach out to more conservative voters, leading to the historic Irish referendum legalizing gay marriage on May 22, 2015.^[9]

Critique

The When contact changes minds study was subsequently discredited after a critique by David Broockman, Joshua Kalla, and Peter Aronow on May 19, 2015, titled "Irregularities in LaCour (2014)",^[10] that concluded that the data had been falsified and no data had actually ever been collected. Broockman et al. found that the survey company LaCour claimed to have used denied performing any work for LaCour and did not have an employee by the name LaCour listed as his contact with the company.^[10] In addition, LaCour had claimed that participants were paid using outside funding, but no organization could be found that had provided the large amount of money required to pay thousands of people.

The "Irregularities" paper also identified the likely method by which LaCour had forged the data. The baseline survey results appeared to have been taken from an earlier dataset called the Cooperative Campaign Analysis Project (CCAP), which LaCour had had access to. The later sets of data appeared to have been simulated from the first using statistical methods to shift the results and by adding normally distributed noise.^[10][11] In addition, the paper noted that canvasser identifiers were missing from the results, making it impossible to verify whether different canvassers produced different results as the original study claimed.

Additional evidence of the study's fictitiousness later emerged, such as evidence that LaCour had tried to retroactively claim that the study had been pre-registered using falsified documents.^[12]

LaCour's coauthor, Donald Green, subsequently requested that the paper be retracted because of “irregularities” in the paper's data and LaCour's failure to give him the raw data on which the paper was based.^[13] Yet Green, as senior author on the Science paper, certified that he had examined the raw/original data on his Science/AAAS Authorship Form and Statement of Conflicts of Interest.^[14] The Science terms include, prominently, the following statement: “The senior author from each group is required to have examined the raw data their group has produced.”^[15]

In his letter to the journal, Green wrote: "I am deeply embarrassed by this turn of events and apologize to the editors, reviewers, and readers of Science."^[16] Andrew Gelman, a professor of statistics and political science at Columbia University, wrote in the Washington Post that Donald Green had accepted LaCour's data "on faith".^[1] On May 28, 2015, the study was retracted by Science,^[17] on the basis that incentives to participate in the survey on which the study was based had been misrepresented, that sponsors had been falsely identified, that the authors could not produce the original data, and (citing the findings of Broockman et al.) that there were "statistical irregularities."^[18]

On May 29, 2015, LaCour uploaded a response to the criticisms.^[19] LaCour admitted to some false statements and apologized for "misrepresenting survey incentives and funding," but denied intentionally falsifying the data itself (though he stated that he could not rule out the possibility that he had "mistakenly mixed up" hypothetical data with collected data). He disputed the timeline of events presented in Broockman et al. (2015). He argued that the failure of Broockman et al. to replicate LaCour and Green (2014) was likely the result of a failure to follow the respondent-driven sampling procedure used in LaCour and Green (2014). He stated that Broockman et al. had selected the "incorrect variable" from CCAP (2012) and then manipulated that variable to make the distribution look more like that in LaCour and Green (2014). LaCour claimed that when the "correct" variable is used, the distributions between the CCAP thermometer and the LaCour and Green (2014) thermometer are statistically distinguishable. LaCour said, "selecting the incorrect variable may have been an oversight, but further manipulating that variable to make the distribution look more like LaCour and Green (2014) is a curious and possibly intentional 'error.'" LaCour also claimed that an independent replication supported the main finding reported in LaCour and Green (2014).

In the New York Times article, LaCour said the study erred in methods, not results.^[20] A subsequent article published by Science indicated that LaCour's response was lacking, failing to address a number of issues while raising new questions about his conduct.^[21] A blog post published by Discover stated that LaCour's rebuttal arguments were "very weak" and failed to refute a central criticism of the Broockman paper.^[22]

After the retraction, the Carnegie Corporation of New York rescinded Donald Green's 2015 Andrew Carnegie Fellowship, revoking a $200,000 award to support Donald Green's research,^[23] and Princeton University rescinded an assistant professorship that had been offered to LaCour.^[24]

The journal Science retracted the article with the concurrence of Green while LaCour did not agree to the retraction.^[18]

Popular press

While Green, David Broockman, and Joshua Kalla conducted numerous interviews with reporters outlining their version of events, LaCour refrained from talking to the media with the exception of a single interview with The New York Times.^[25] Various opinions have been advanced in the media as to why Science published the flawed article by LaCour and Green.^{[26][27][28][29][30]}

The editorial page of The Wall Street Journal speculated that LaCour's argument originally gained acceptance in the scientific community because it "flattered the ideological sensibilities of liberals".^[27] New York Magazine columnist Jesse Singal dismissed the Wall Street Journal editorial as silly and uninformed, and suggested instead that the main reason for the article's publication was its contradiction of prior research.^[26] Gelman, in his Washington Post article, said that the journal Science is sometimes called a tabloid "because of its pattern of publishing dramatic but fishy claims (at least in social science)".^[1]

As to what motivated LaCour's behavior in the first place, his co-author, Donald Green, has expressed bafflement.^[28] New York Magazine points to the pressure that social scientists are under to publish scholarly articles, while stating that "profound pressure to publish certainly can’t explain LaCour’s deception on its own".^[29] Conservative pro-gay-marriage columnist S.E. Cupp wrote that, "The doctored study will only encourage the perception that advocates are going too far."^[30]

A New York magazine opinion piece by sociology doctoral candidate Drew Foster argued that the study exposed problems with the culture of political science research and supervision given to junior academics, along with a competitive culture caused by overproduction of PhD students relative to available political science academic positions.^[31]

Follow-up study

Following their debunking of When Contact Changes Minds, Broockman and Kalla (2016) conducted an experiment similar to the one purported to have been conducted by LaCour and Green.^[32] Households in Florida were sent a survey on social attitudes, and were later visited by canvassers to discuss either recycling or transphobia and transgender rights, with the change in the householders' opinions tracked by follow-up surveys. The study found that extended conversations with canvassers did in fact reduce anti-transgender prejudice and were more effective long-term than attack ads.^[33] However, contrary to one of LaCour's claims, the identity of the canvasser did not seem to matter; both transgender and cisgender canvassers had a similar effect.^[33]

References

Notes

Bibliography

• LaCour, Michael J. & Green, Donald P. (2014). When contact changes minds: An experiment on transmission of support for gay equality. Science 346(6215): 1366.
• LaCour, Michael J. (2014, August). The echo chambers are empty: Direct evidence of balanced, not biased, exposure to mass media. In Annual Meeting of the Midwest Political Science Association.
• LaCour, Michael J. and Vavreck, Lynn (2014). Improving media measurement: Evidence from the field. Political Communication, 31, 408-420. doi:10.1080/10584609.2014.921258