**************************************************************************************

Australian Psychologist Volume 7 No. 1 March 1972, 40-46.

A New Reliability Maximization Procedure for Likert Scales



J. J. RAY

The University of New South Wales

A method for scale reliability maximization is presented here which is derivable from the intercorrelation matrix of the items. This contrasts with the usual procedure for item selection of seeking correlations of each item with the total score on the protoscale (Guilford, 1954). The new method depends on the equivalence of the Kuder-Richardson and the Spearman-Brown formulas. Both are cases of coefficient "alpha" (Cronbach, 1951; Lord and Novick, 1968, pp. 90 and 118; Morrison, Campbell, and Wolins, 1967).

The advantage of the new method may best be set out after a further description of the more conventional method. In this method one has to re-process the raw data several times because the criterion for item-selection (correlation with the total score on the scale) changes each time a weak item is dropped. The only way to get a new set of total scores for each person is to recalculate from the raw data. With the new method, by contrast, the raw data is processed only once. It must of course be conceded that empirical practice does show that one may remove weak items in small groups (approx. 1/5 of the scale each time) and still get the same optimum results as if the totals had been recalculated after each single removal. Even with this economy however, the conventional method is still laborious and inefficient.

A second point is that a thorough scaling study of a new construct would in any case include some sort of structural analysis of the item pool (principal components, cluster analysis, factor analysis etc.). For any such study an intercorrelation matrix is basic data. The new method, then, allows both the structure study and the reliability maximization to be done at once -- using the same data. With the conventional method a separate study and set of calculations from the raw data has to be done.

A third advantage of the new method that might loom very large in real-life is that it is simple enough to be done quickly by hand once the inter-item correlations are given. This reduces dependence on computers to a minimum.

RATIONALE

In the Spearman-Brown formula, the reliability statistic "alpha" is presented as a ratio between the number of items and the average inter-item correlation. Since all item analysis which aims at increasing reliability does so by rejecting some of the original items, it follows that reliability can only be increased by increasing the average intercorrelation of the items. This immediately leads to a consideration of how we might select that combination of items which has the highest mean inter-item correlation coefficient (mean r). For very small groups of items this might be done by calculating mean r for each possible combination of items at each possible scale length. For a test of conventional length, however, the number of such combinations required would be impossibly large. For this reason, compromise procedures such as the conventional method described above are usually adopted. That such a method is not exact may be illustrated by an example: In a test of 20 items, item 1 correlates .898 with item 2 but near zero with all other items. If intercorrelations of other items were of usual magnitudes (say mean r = .150), item 1 would be immediately rejected by the conventional method -- and yet the reliability of a combination of items 1 and 2 would be .94, a magnitude seldom reached by longer combinations of items. It is not intended to argue here that a two-item test would be desirable but merely to show that the conventional method can only be relied on to produce the highest intercorrelating group of items to the extent to which those items do not show great contrasts in their correlations with other items.

Since one is forced to compromise then, the choice of our index of "item-strength" (i.e. the contribution of each item to overall homogeneity or mean r) is seen to be arbitrary. To date the item-total correlation of each item has served as such an index. A perhaps more obvious index, however, is the mean correlation of an item with each other item (Sum r item). For a 4 item scale, Sum r item for item 1 would be found by summing the correlations between items 1 and 2, 1 and 3 and 1 and 4. This is not to be confused with the statistic mean r mentioned earlier. This would be found by summing the correlations between 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4 and 3 and 4 and then dividing by 6 i.e. n (n - 1)/2.

METHOD

It will be seen that the method to be described is not a short-cut procedure such as the upper-middle-lower thirds procedure and does produce immediately the reliability coefficient for each shortened version of the scale. It has been computerized and there leads to a saving in computing time for all but very long scales administered to very small pools of subjects.



{I will not reproduce the remainder of this article here as it refers to an era when computing time was scarce and expensive. The efficiencies the article enables are therefore no longer relevant. The computer program that operationalizes the method has however been updated for the PC era and is available on request both as an executable and in FORTRAN source code}

REFERENCES

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika, 1951, 16, 297-334.

Guilford, J. P. Psychometric methods. New York: McGraw-Hill, 1954.

Lord, F. M., and Novick, M. R. Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley, 1968.

Morrison, D. G., Campbell, D. T., and Wolins, L. A. Fortran IV program for evaluating internal consistency and single-factoredness in sets of multilevel attitude items. Educational and Psychological Measurement, 1967, 27, 201. (See programme manual.)

Rasch, G. An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 1966, 19, 49-57.




Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here or here)
Go to John Ray's "Dissecting Leftism" blog (Backup here or here)
Go to John Ray's "Australian Politics" blog (Backup here or here)
Go to John Ray's "Gun Watch" blog (Backup here or here)
Go to John Ray's "Education Watch" blog (Backup here or here)
Go to John Ray's "Socialized Medicine" blog (Backup here or here)
Go to John Ray's "Political Correctness Watch" blog (Backup here or here)
Go to John Ray's "Greenie Watch" blog (Backup here or here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here)
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated -- Backup here)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated -- Backup here)
Go to John Ray's "A scripture blog" (Not now regularly updated -- Backup here)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here or here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here)
Go to Selected pictures from John Ray's blogs (Backup here)
Go to Another picture page (Best with broadband)