6 Jun 2024

Good survey methodology

I was talking to my son about the quite stupid study of the Mediterranean diet that I critiqued yesterday.  He challenged me to nominate a feasible methodology that I would accept as credible.  I have done a heap of published survey research so I think I can do that

For a start, I accept that a classical before-and-after study of diet is virtually impossible because of the long lag needed to detect lifespan effects.  But a correlational study can still have considerable credibility if it makes extensive use of statistical controls. The first computer program I ever wrote  way back in 1967 was to do partial correlations so  I have myself long used statistical controls.  When the "Summer of Love" was happening in San Francisco, I was writing FORTRAN code!

But to use statistical controls you have to have measures of the likely confounders, and getting measures of some of the confounders can be pesky.  Both income and IQ are broadly influential and very influential in the case of income.  Poor people routinely do badly on almost all ill health measures.  But to make your findigs credible you usually have  to take at least income  into account, with neuroticism (chronic anxiety as measured for example by the Eysenck N scale) an important third factor)

And the although the difficulties are great, they are not insusuperable.  I have managed them at times.   The Goossen hidden scale of intelligence is particularly useful as long as it is kept up to date.  And the usual demographics should of course be used.  Other confounders examined will depend on the study concerned

And how do we deal with the problem of "faking good" on self-reports?  That is actually one of the easy ones.  I have almost always embedded a social desirability or "lie" scale in my questionnaires

And of course the study must be double blind.  That is as important in survey research as it is in experimental research. So  persuasive  survey research can be done but such research is sadly rare  in epidemiology.  Epidemiologists generally seem to be blissfully  unaware of the precautions psychologists  routinely take in survey research.