The Problems of Measuring Attitudes in Social Science Surveys Essay Example

study, and to help explain and to predict human behaviour with ever increasing degrees of accuracy. This is especially true of large scale surveys carried out by questionnaire, as the probability of accurate interpretation even of a small group is minimal, and is greatly exaggerated with larger numbers.Finally, I will conclude that to do less with more is generally the only guarantee of validity and reliability in researching attitudes at any population level.Defining AttitudesThe initial problem with attitude research is that as a concept, the term "attitude" itself has no distinct and uncontested meaning.

Harry S. Upshaw makes the observation that:"The term attitude is used in a variety of ways by social scientists. Most commonly, the term refers imprecisely to the stands people take on controversial issues. Discussions of social attitudes focus on three classes of phenomena... Cognitive [which] refers to an individuals information regarding an issue...behavioural, referring to the acts which an individual performs, advocates, or facilitates with regard to an issue. The third phenomenon is affective, referring to the individuals valuations." (Upshaw: 60, In:The term "attitude" has been broadly defined elsewhere as "a predisposition to behave in a particular way" (Proctor, In Gilbert:117), and even more vaguely as: "our character - what we think and feel about our world and ourselves" (BSA, 11: back cover)In the introduction to an open university paper entitled "Attitudes", Elms defines attitudinal aspects as essentially subjective, the essential criteria that they have in common being that they are all to do with the individual's orientation to the external world.

Thus far, looking at four definitions of the term, it appears that an attitude depends for its existence on an individual,

and an object of attention (the most general object being "the world"), towards which that individual's attention is directed for any length of time. An attitude, inspired by a perceived object of attention, is thus experienced by the individual as thoughts and feelings, and articulated as beliefs and values.Whatever criteria one uses to refine the concept of attitude, any refinement, I would argue, must necessarily be based on the subjective experiential awareness of the individual of the external world. An attitude is, ultimately, an abstraction of individual thoughts and feelings. An attitude is thus a subjective phenomenon, and it this fact presents the social researcher with immediate problems as to the investigation of them:"Beliefs, values and attitudes do share several common attributes. They are all psychological constructs; that is, they cannot be observed directly by another individual, but must be inferred from the individual's introspective reports and (perhaps to a lesser degree) from observations of his behaviour" .General Problems of Attitude ResearchIn getting to this internal, subjective world, the problem is that "most people use only the vaguest criteria in categorising personal concepts such as beliefs, values and attitudes.

Rosenberg considers the nature of the subjective world of the individual in some depth, concluding that if we admit to the necessity of getting to attitudes in order to explain and predict human behaviour in the first place, that no scientific explanation of human action is possible, because:"The immediate upshot of the intensionality of action and its determinants is that...there is no way, even in principle, of providing a description of the beliefs and desires that cause action in which they are independent of one another and

independent of the action they are said to cause.

He goes on to state that every belief and desire is ultimately a function of every other belief and desire. In order to fully understand why one individual acts in a particular way given particular circumstances, we need to get to what Rosenberg refers to as his holism of mind: We must understand the whole picture:"To explain an action with full precision, we must identify the very sentence in which the agent would describe his action and the very sentence that the agent would use to describe his desires and beliefs" (Rosenberg: 52)This is further complicated by the fact that "as soon as we attempt to reflect about the way in which life confronts us in immediate concrete situations, it presents an infinite multiplicity of successively and coexistently emerging and disappearing events, both within and outside ourselves. The absolute infinitude of this multiplicity is seen to remain undiminished even when our attention is focused on a single object..

All the analysis of infinite reality which the human mind can conduct rests on the tacit assumption that only a finite portion of this reality constitutes the object of scientific investigation" (Weber: 72)Given this complex subjective world, it would appear the social scientist must strive to try to get to the conceptual world of the individuals under scrutiny in order to reach a similar explanatory framework of their behaviour as they have. This would be desirable as the more interpretative distortion that arises in the process of explaining action from a different conceptual framework, the less reliable and valid an explanation of the respondents' behaviour that explanation will be. It

follows that the greater the degree of conceptual distortion, the less that the data collected will be able to explain, and the more unreliable it will be as a predictive mechanism.Rosenberg points out that anthropologists have thus likened research to language learning.

Given the fact that subjective worlds are so unique and complex, and are not even perceived with clarity by many individuals, as well as the fact that these subjective worlds are hidden from immediate observation, I would argue that interpretation of these subjective worlds is unavoidable. It follows that the language learning approach, characterised by the anthropological enterprise, is the ideal way to uncover these conceptual worlds with the minimum of distortion.However, The interpretation of significance of certain statements and actions respondents from a secondary perspective will never be the same as the respondent. It will never be 100% reliable or valid. Interpretative distortion will always occur.

As Geertz says:"To set forth symmetrical crystals of significance, purified of the material complexity in which they were located, and then attribute their existence to autogenous principles of order, universal properties of the human mind, or vast, a priori weltanschauungen, is to pretend a science that does not exist and imagine a reality that cannot be found." (Geertz: 20)Unless the researcher can locate that specific sentence and can step into the conceptual framework of the specific individuals whom he is researching, data derived from the research process will lack accuracy and will be less useful in predicting future human behaviour with any degree of accuracy.Specific problems of research involving large populationsThis evidently creates problems for the researcher seeking to collect data on large groups by survey. The anthropologist

has time to get to know the language, and time to write a largely descriptive report of how his subjects make sense of their world and to get to grips with how this conceptual framework relates to their values and beliefs.

The survey researcher, on the other hand, is, by virtue of financial and time constraints, unable to learn the conceptual nuances of the experiential language of his respondents.Following Rosenberg's idea of locating the very sentence that an individual would use to explain his own actions, beliefs and desires in order to reliably explain his actions would mean taking seriously Elm's suggestion and would involve dealing with individual psychological components or elements or thoughts. This would, however, be a daunting task where populations of millions of people are involved. The researcher is then, forced, I believe, to follow Elm's more serious suggestion of distinguishing between attitudes, beliefs and values, basic operationalisation which would be necessary in order for us to measure and compare attitudes and thus to improve the explanation and prediction of human behaviour.As Elms says: "in order to study these matters quantitatively, as well as impressionistically - we need to become more specific and discrete." (Elms: 10) He goes on to argue for the case that we should perceive of attitudes as consisting of two fundamental aspects.

Along with the development of these new methodologies have come a whole host of problems of operationalisation, reliability, validity, and analysis, not least of which is the fact that, as verbal behaviour is also an action, we can't ever prove that saying yes is caused by an underlying attitude.Ultimately, I would argue that the purpose of all attitude

research, that is the whole point of trying to discover what people think and feel about particular aspects of their perceived external world is to enable us to better explain and predict human behaviourThat it is an important exercise to try to ascertain what people know and feel about, and how they evaluate particular issues is indicated by the proliferation of opinion polls and market research surveys that are funded both privately and publicly. The utility of such undertakings lies in their predictive value: that governmental departments or private enterprise can adapt or even abandon certain policies and strategies in the light of the fact that hostile opinion may well lead to hostile action if a particular policy is followed in a particular way.Attitude research essentially became necessary because it was deemed that behavioural observation alone had proved inadequate for explaining patterns in human behaviour. I would thus disagree with Upshaw's conclusion that behaviour is a dimension of an attitudinal state, and would separate behaviour, which can be observed directly, from attitudes, which require careful theoretical and methodological preparation before research is carried out.Behaviour does not come under the conceptual idea of an attitude, and it cannot by definitional necessity.

When I consider attitudes below, I am not talking about behaviour, as attitudes, requiring the development and validation of indirect indicators, start where behaviour ends. Certain research problems that pertain to attitudes simply do not pertain to behaviour, and it follows that there are certain research problems that arise simply because attitudes, by the fact they are by definition, subjective, and thus hidden.Below I will outline and criticise the various strategies that have been suggested to

overcome the problems of interpretation associated with attitude research.Methodological Problems of Attitude Research by QuestionnaireResearch conducted by questionnaire consists of questions consisting of linguistic terms that are derived to measure theoretical constructs. According to Newell, the researcher should "begin with a hypothesis", and "each question must have a direct relevance to one of the variables of the hypothesis". (Newell, in Gilbert: 99).

It follows that the variables that Newell is talking about are likely to be derived from theory, and will not located in the specific world of meaning which Geertz talks about. This means that the amount of interpretative distortion arising from survey research will inevitably be much greater than that arising from research of a more anthropological nature, which seeks to uncover the linguistic and conceptual structure of the world of the respondents.What will be measured, post operationalisation, will never be the actual attitude as experienced by the individual. This, I believe, is what Proctor means when he says that: "an attitude is a hypothetical construct: no one has ever seen or touched one, and its existence or properties must be inferred indirectly.

Such problems arise partly from the fact that the researcher of contemporary industrial society has to deal with deadlines and will usually have a more specific purpose for conducting the research than the anthropologist. The survey researcher is a researcher with an agenda, and, I will argue below, that the very fact that he has goals other than describing the conceptual world of his respondents, creates problems beyond those associated with the unavoidable interpretation of attitudes. I will also suggest that the bigger and more theoretically rich the concepts and the larger

and more diverse the survey population is, the more distortion will arise in the collection of attitudinal data.Conceptual Distortion in Attitude ResearchAs I have already said, in order to measure attitudes over a large sample, it will probably be necessary, due to financial and time constraints, to construct a questionnaire.

This will involve the delimitation of certain aspects of the respondents' subjective worlds. The beliefs, values and attitudes that a questionnaire is designed to tap will vary according to the object of attention that the attitude is held towards, as an attitude does not exist on its own, and is essentially a reaction to the external world.For example, in The British Social Attitude Survey, running since 1983, and conducted by the Social and Community Planning Research Institute, designed to ascertain degrees of variance and changes in British attitudes over time, attitudes are defined rather vaguely in different concluding sections of the text as, for example, "levels of concern", or "degrees of pessimism". For research purposes, when dealing with a sample of 3000, taken to be representative of an entire population, an attitude has to be operationalised discretely, as something that is held by an individual towards something as concrete as public transport policy, the environment, or authoritarianism.

The decision of what to research is made by the researcher, or by those funding the research. This creates the problem that, as the conceptual world of the individual is complex and holistic, then any decision to research a particular isolated linguistic construct that is made separately of that individual will not reveal how that construct relates to the respondents' over all conceptual understanding of the world.Because the subjective world

is so complex, as soon as one sets out to measure a belief or value towards an isolated object, distortion will inevitably occur, because it is highly unlikely that each respondent will understand the concept in the same way as it has been operationalised.

Ultimately, the terms within the question must mean exactly the same thing to each respondent in order for it to be a 100% valid measurement of his level of radicalism. (Proctor, in Gilbert: 118)Witherspoon, in line with Proctor, does not ask people how romantic or materialistic they are, and I assume the reason for operationalising these two constructs into eleven questions is to avoid the problems of definitional flexibility that would invalidate them by allowing them to be interpreted too broadly by respondents. I would argue, however, that some of the terms Witherspoon employs in her eleven Likert scales are so broad as to be asking the respondents to be making social scientific judgements in any case.To take as an example, the question: "We believe too often in science, and not enough in feelings and faith" (Witherspoon, in BSA:110) includes three terms that are extremely abstract. On reading the term "science", the individual may be unconsciously reminded of his boring physics lessons, and may balk at the terms faith and feelings because he simply doesn't know what they mean, and so conclude that he doesn't agree with the statement.

In this case, his response would be taken as a measure of his pro-romantic orientation to the world, when really the respondent was indicating that he thought physics lessons were boring. This is an extreme example, but it makes the point that those terms

that are used as standard, objective measurements of a theoretical construct by the researcher will be interpreted subjectively by the respondents to the questions containing those terms. Such questions will therefor never be objective, standard measures of an attitude, however attitudes are defined.To take an even more extreme example, three out of the eleven questions contain the term nature, which has been consistently regarded as one of the most complex words in the English language. The probability that this term would be interpreted in the same way by each of the 1500 respondents is, I think, fairly minimal.

Evidence for the fact that conceptual confusion has arisen lies in Witherpspoon's conclusion that " Far from being the least romantic about the environment, these strong materialists prove to be the most romantic" (Witherspoon, in BSA 11: 111) and that "there is an incoherency and inconsistency in public valuations of the environment, a situation commonly found when attitudes are only superficially held".I would be more inclined to conclude that, because the linguistic terms used to construct the questions that measure the two concepts of materialism and radicalism are so abstract, there is no guarantee that they are acting as standard measures, and so are invalid indicators of the two constructs."Smith argues that question wording is a significant problem in survey research and suggests that there must be a shared vocabulary between researcher and respondent" (Newell, In Gilbert: p110). With concepts as big as science and faith, however, and with a sample of 3000, chosen to be representative of the entire British population, that there is little hope of ensuring that there is a shared vocabulary between all respondents

and the researcher.