Uncovering the Pattern of Social Stratification: A Two-Year Test-Retest Inquiry

This is a report of the main findings of a reliability study carried out in association with the 1972 Oxford Social Mobility inquiry. The original sample consisted of some 10,000 men interviewed in England and Wales. The reliability study involved re-interviewing a representative 10 per cent of the sample after a lapse of two years. After initially exploring the reliability of occupation (position on the Hope-Goldthorpe scale) and distinguishing this from its stability over a period of two years, the analysis proceeds to examine the basic set of stratification variables. The raw correlations for the full 1972 sample are adjusted in the light of the reliability findings, and the adjusted values are employed in path analyses. The main substantive conclusion of the paper is that education is more a disrupter than a transmitter of the influence of paternal social standing on respondent's occupation. The main conclusion of the reliability analysis, apart from the estimates of the reliability coeflicients, is that measurement errors are not entirely independent of one another. Finally, some comparisons with a comparable American inquiry are offered. INTRODUCTION: THE CASE FOR A RELIABILITY STUDY Much lip-service is paid to the need for reliability inquiries in empirical sociology, but few satisfactory reliability studies have been carried out. This may be because there is a widespread belief that replication is a dull and technical process, lacking the excitement and glory of substantive sociological inquiry. Nothing could be further from the truth. A sound reliability study has the edge over a one-off survey in both theoretical rigour and substantive interpretability. It is theoretically more rigorous because the most highly-developed theory available to the sociologist is the theory of measurement. And it is substantively more interpretable because only when the relative The British Journal of Sociology Volume XXXVII Number 3 This content downloaded from 157.55.39.58 on Fri, 26 Aug 2016 04:00:03 UTC All use subject to http://about.jstor.org/terms 398 Keith Hope, Joseph Schwartz and Sara Graham reliabilities of variables are known is there any justification for comparing estimates of parameters (e.g. regression/path coefficients) or for disaggregating them into their component parts. These two aspects of reliability work are briefly elaborated in the following introductory remarks. The task of estimating reliabilities is primarily an exercise in research design. Provided the design is appropriate, the task of estimating coefficients is relatively straightforward. Although an enormous number of models may be fitted to a small set of variables, even to as few as six or eight, the logic of the design of a reliability study severely constrains the number of models which may sensibly be employed. Thus a reliability study, unlike most empirical sociological investigations, dictates the form of its own analyses within quite narrow limits. Another way of making this point is to note the great number of parameters which are fixed or otherwise constrained (e.g. to equality within pairs), simply by virtue ofthe logic inherent in the design of the investigation. Of course, the analyses are never entirely determinate; there is always room for inductivist manoeuvre at the margin. Nevertheless, the main point is valid, viz., that the logic of the analysis flows from the logic built into the design of the study. Accordingly, the present paper places rather more weight on presenting details of the survey procedures employed than do most accounts of one-ofT inquiries. Every social survey is the product of a set of social institutions; that is, it is the work of bodies of men and women possessing skills, aims, norms, and understandings which are to some extent shared and in some degree diverse. By contrast, the variable whose reliability is required consists of no more than a column of numbers stored on cards or magnetic tape. What we seek to estimate is the precision with which a person has been assigned to a category or level of the variable, and this task requires both supervision and a conceptual understanding of the social processes by which the assignments take place: in the absence of these, the very nature of the variable becomes indeterminate. In making these observations, we are acknowledging the validity of criticisms which sociologists of an ethnomethodological persuasion often cast at 'positivistic' empiricists. The ethnomethodological critique has obvious analogies with epistemological scepticism and, like its analogue, it may be rendered unanswerable by adoption of an extreme position. But if we consider only its more moderate manifestations in which it admits the existence of criteria of truth and accuracy which are both rationally defensible and satisfiable in principle, if not in practice, then a reliability study may be profferred as a paradigm case of empirical inquiry which brings together strong theory and the study of social understandings. The theory incorporates both sampling theory and the theory of measurement, while the understandings are sociologists' or, in some cases, public social This content downloaded from 157.55.39.58 on Fri, 26 Aug 2016 04:00:03 UTC All use subject to http://about.jstor.org/terms Uncovering the pattern of social stratification 399 constructs such as educational level and occupational prestiget Turning to the second main point of these introductory remarks, we assert the substantive significance of the investigation of reliabiity. In an early and sophisticated paper, Siegel and Hodge (1968: 37) observed that the end result of any assessment of reliability and the effects of measurement errors should be the approximation of the correlation between true scores by correction of the observed correlation between measured variables. The more complex the analyses we wish to perform, the greater the need for the estimation and removal of systematic and random error. The practice of disaggregating regression (path) coefficients into component 'effects' assumes the absence of differential error, for it is only when every variable is equally reliable that the values of components are commensurate with one another. Additivity of components may be ensured by the algebraic rules defining them, but additivity does not guarantee comparability unless the basic variables are all equally reliable. The whole purpose of the multivariate analysis of survey data is vitiated if the different variables are not measured in units of comparable significance. It was in order to deal with this obvious, but much neglected, point that a retest of 10 per cent of the original sample was included in work on the Hope-Goldthorpe occupational prestige scale (Goldthorpe and Hope 1974). It is for the same reason that the British Social Science Research Council was asked to fund a 10 per cent repeat survey of the 1972 Oxford social mobility inquiry, although no reliability study had been incorporated in the original design. It is to the description of this latter reliability study that we now turn.