*****************************************************************************************

Journal of Social Psychology 1990, 130(3), 397-399.

ACQUIESCENCE AND PROBLEMS WITH FORCED-CHOICE SCALES



J.J. Ray

University of New South Wales, Australia

Abstract

Toner (1987) correctly identifies acquiescent response bias as a distorting influence in the measurement of attitudes. He proposes that the use of forced-choice scales would overcome such problems. It is shown that this advice overlooks several serious problems of forced-choice scales. Use of balanced Likert scales is suggested as an alternative.

Toner (1987) reported research in which acquiescent response bias (the tendency to say "Yes" indiscriminately in answer to questions) could be shown to constitute a serious problem. He suggests that we might generally in the future avoid such problems by using forced-choice scales.

However, if a Likert-type scale is replaced by a forced-choice scale because of a tendency in the subjects to acquiesce, all that is likely to happen is that a "Donkey vote" effect will replace the Yeasaying effect. Both are forms of indiscriminate agreement. If the choices in a forced-choice scale are labelled "a" and "b", the Donkey voter will, at the extreme, simply tick all the "a"s. Both Yeasaying and Donkey voting enable a respondent to convey indifference to the task.

Donkey voting can of course be controlled by alternating the order in which the choices are offered. A forced-choice scale of conservatism (for instance) might key as "a" 50% of the "Leftist" statements and 50% of the "Rightist" statements. Much the same can, however, be done with Likert scales. Wording 50% of the statements in a scale pro-Right and 50% pro-Left will eliminate any systematic effect of acquiescence on a conservatism measure.

But has it not been shown that "balanced" Likert scales often do not work? Are not the supposedly opposed items often in fact found to be nearly orthogonal (Christie, Havel & Seidenberg, 1956)? This is true but note that only a Likert scale enables you to find that out. With a Likert scale the correlation between the positive and negative halves of a scale can be ascertained on any occasion of the scale's use. On some occasions the desired negative correlation may emerge and on some it may not (Ray, 1983). On the latter type of occasion, the validity of the scale for that population must be questioned.

With a forced-choice scale, by contrast, the opposition of meaning between supposedly opposite items cannot be examined. It has to be assumed. And, as we have seen, that is a fairly "heroic" assumption. The person who constructed the scale may have done tests to see that the opposite-seeming items were in fact perceived as opposite by his respondents but there is no way to check on whether this opposition prevails on all subsequent occasions of the scale's use. And the breakdowns of meaning opposition for the supposedly opposite items of a scale are highly unpredictable. It is a breakdown that can happen at any time (Ray, 1985). When a special study is done to check on it, even widely-used forced choice scales tend to be shown as relying on unmet assumptions (e.g. Gatz & Good, 1978).

The inability to allow such checks is not the only problem with forced-choice scales, however. There is another major problem with social desirability responding. If the paired items are not equal in social desirability, there is a big chance that the choice between alternatives will be determined by social desirability considerations rather than in the way the scale-constructor intended. People who construct forced-choice scales are supposed to equate their items in terms of social desirability at the time the scale is constructed but social desirability, unfortunately, changes over time and from sample to sample (Orvik, 1972). One cannot assume that the initial equation in terms of social desirability prevails indefinitely. That this problem is more manageable with Likert scales has been set out at length in Ray (1973) -- where it is also shown that Likert scales are empirically more valid than equivalent forced-choice scales. See also Ray (1980) in that connection.

Finally, forced-choice scales cannot be meaningfully intercorrelated or factor-analyzed (Johnson, Wood & Blinkhorn, 1988).

Clearly, then, forced-choice scales are not a viable solution to the problem Toner addresses. That balanced Likert scales, however, are a viable solution to the problem of acquiescent response bias is suggested by the fact that even the F scale was eventually re-written in a psychometrically satisfactory balanced form (Ray, 1972).


REFERENCES

Gatz, M. & Good, P.R. (1978) An analysis of the effect of the forced-choice format of Rotter's internal-external scale. J. Clin. Psychol. 34, 381-385.

Johnson, C.E., Wood, R. & Blinkhorn, S.F. (1988) Spuriouser and spuriouser: The use of ipsative personality tests. J. Occupational Psychol. 61, 153-162.

Orvik, J.M. (1972) Social desirability for the individual, his group and society. Multivariate Behavioral Research 7, 3-32.

Ray, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Ray, J.J. (1973) Task orientation and interaction orientation scales. Personnel Psychology 26, 61-73.

Ray, J.J. (1980) The comparative validity of Likert, projective and forced-choice indices of achievement motivation. J. Social Psychology, 111, 63-72.

Ray, J.J. (1983) Reviving the problem of acquiescent response bias. Journal of Social Psychology 121, 81-96.

Ray, J.J. (1985) Acquiescent response bias as a recurrent psychometric disease: Conservatism in Japan, the U.S.A. and New Zealand. Psychologische Beitraege 27, 113-119.

Toner, B. (1987) The impact of agreement bias on the ranking of questionnaire response. J. Social Psychology 127, 221-222.




Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here)
Go to John Ray's "Dissecting Leftism" blog (Backup here)
Go to John Ray's "Australian Politics" blog (Backup here)
Go to John Ray's "Gun Watch" blog (Backup here)
Go to John Ray's "Education Watch" blog (Backup here)
Go to John Ray's "Socialized Medicine" blog (Backup here)
Go to John Ray's "Political Correctness Watch" blog (Backup here)
Go to John Ray's "Greenie Watch" blog (Backup here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here)
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated. Backup here)
Go to John Ray's "A scripture blog" (Not now regularly updated)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here)
Go to Selected pictures from John Ray's blogs (Backup here)
Go to Another picture page (Best with broadband)