October 22, 2015

"Covert" measure of racism invalid

I have been critical of the IAT for over a decade (See e.g. here) so I am pleased to see that its invalidity seems to be becoming widely accepted.  The text below is from a wide-ranging survey of psychological research findings that have not stood up to scrutiny

Perhaps most consequentially, replications failed to validate many uses of the Implicit Association Test, which is the most popular research tool in social psychology. Its designers say the test detects unconscious biases, including racial biases, that persistently drive human behavior. Sifting data from the IAT, social scientists tell us that at least 75 percent of white Americans are racist, whether they know it or not, even when they publicly disavow racial bigotry. This implicit racism induces racist behavior as surely as explicit racism. The paper introducing the IAT’s application to racial attitudes has been cited in more than 6,600 studies, according to Google Scholar. The test is commonly used in courts and classrooms across the country. 

That the United States is in the grip of an epidemic of implicit racism is simply taken for granted by social psychologists​—​another settled fact too good to check. Few of them have ever returned to the original data. Those who have done so have discovered that the direct evidence linking IAT results to specific behavior is in fact negligible, with small samples and weak effects that have seldom if ever been replicated. One team of researchers went through the IAT data on racial attitudes and behavior and concluded there wasn’t much evidence either way.

“The broad picture that emerges from our reanalysis,” they wrote, “is that the published results [confirming the IAT and racism] are likely to be conditional and fragile and do not permit broad conclusions about the prevalence of discriminatory tendencies in American society.” Their debunking paper, “Strong Claims and Weak Evidence,” has been cited in fewer than 100 studies.

SOURCE

The text above is part of an article that looked at replications.  There have been several attempts made recently to see if a research finding will be repeated if the same experiment is repeated. About two thirds of the reports could not be replicated.  When someone else carried out exactly the same research, the original finding was not repeated.  That is of course very destructive to faith in scientific "findings".

There is however another problem that is equally disquieting:  Researchers keep refusing to make their raw data generally available for others to check the analyses.  Many journals have policies saying that authors MUST make their raw data available to other scientists.  But it still does not happen.  As the report below shows, only 38% of psychologists were willing to make their raw data available to others.  That is however good when compared with climate researchers.  The percentage there seems to be 0%.

Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm

By  Wolf Vanpaemel et al.

Abstract

To study the availability of psychological research data, we requested data from 394 papers, published in all issues of four APA journals in 2012. We found that 38% of the researchers sent their data immediately or after reminders. These findings are in line with estimates of the willingness to share data in psychology from the recent or remote past. Although the recent crisis of confidence that shook psychology has highlighted the importance of open research practices, and technical developments have greatly facilitated data sharing, our findings make clear that psychology is nowhere close to being an open science.

SOURCE

And it's even worse here, where 31 emailed requests for data yielded only 4 positive answers.

"Psychology is bunk" would be a reasonable comment on most of it.  That psychologists are overwhelmingly Leftist does help to explain that.  Leftism is bunk too -- JR






Go to John Ray's Main academic menu
Go to Menu of longer writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page
Go to Selected pictures from John Ray's blogs