November 29, 2007

IQ and factor analysis

Cosma Shalizi is a rather egotistical-sounding young man of apparently Afghan ancestry. He is also an assistant professor in the Department of Statistics at Carnegie Mellon University in Pittsburgh. On his blog, Three Toed Sloth, he has a long post demonstrating that a popular form of mathematical analysis generically called factor analysis cannot be used to demonstrate the existence of IQ (or 'g' as we psychometricans call it).

His criticisms are perfectly correct. Factor analysis cannot prove ANYTHING in fact. It is just a convenient but rather arbitrary way of summarizing a set of correlations. Factor analysis is such a weak technique that I myself in my more than 200 published academic papers have used it only rarely -- even though I normally present my research results in correlational form. Shalizi is also correct in saying that much of the research into IQ has used factor analysis as a way of summarizing findings. That the findings are not DEPENDANT on that descriptive technique is the point he appears to overlook.

The underlying finding for 100 years or more is that ability to solve one sort of puzzle generalizes strongly to the ability to solve other quite different sorts of puzzle. Problem solving ability is general, no matter how you choose to summarize that. And problem-solving ability is what IQ or 'g' measures.

OK. What I have just said above is an excerpt from what I said on October 25. I emailed Shalizi a link to the October 25 post and I hear that others did so too. He has completely ignored it. In fact he has a very inflated recent post here in which he uses his previously demonstrated "wisdom" to attack the article on IQ in Slate by Saletan. Shalizi is so sure of his wisdom that he can only ascribe discreditable motives to Saletan and his editors for publishing what they did. And other bloggers (e.g. here) have taken Shalizi and his conclusions at face value. They seem to think that Shalizi has made lots of large correlations between different types of ability tests suddenly disappear!

A matter I did not address in my previous post is heritability. Once again in this case, Shalizi uses perfectly legitimate concerns about the statistical techniques used in summarizing the findings to claim that no firm conclusions at all are possible. In so doing he is using an old Leftist dodge that I commented on many years ago. He sets the standard so high for what he will accept as satisfactory evidence that nobody could prove anything to him under such standards.

In most of science, maybe ALL of science, you cannot prove things beyond all shadow of doubt -- so someone who does not want to believe something will always be able to say that the evidence is still not good enough. But if we normally behaved that way we would disregard most science. But we don't. And that is because we get a lot further by accepting that science is an enterprise that deals in probabilities rather than in certainties. If you want certainty, you need to turn to either theology or mathematics and you might not even find it there.

So what Shalizi is doing is what is called "intellectual dishonesty" -- judging a matter not on the balance of the evidence for and against it but rather on whether or not the concusions suit him.

And again, when you look at the underlying evidence about heritability that the various statistical techniques try to summarize, there is a lot there that Shalizi just cannot make go away. The IQ scores of identical twins will almost always be strikingly similar, for instance -- much more similar than the IQs of normal brothers and sisters or even of fraternal twins -- and certainly much more similar than the scores of random strangers. The facts on the ground just don't go away, no matter how you choose to summarize them. And those facts show that IQ scores are clearly tied to your genetic inheritance.

Razib makes further points about Shalizi's shilly shallying. Razib shows that, if you look hard enough, Shalizi does in fact to a degree acknowledge the underlying realities in his writings. But you would never guess that from his dismissive conclusions.







Go to John Ray's Main academic menu
Go to Menu of longer writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page
Go to Selected pictures from John Ray's blogs