Research Methods Ch. 5-8 – Flashcards

Unlock all answers in this set

Unlock answers
question
Interrogate the construct validity of a study's variables.
answer
Construct validity: a measurement of how well a variable was measured (or manipulated) in a study Measured variables: Variables whose levels are observed and recorded (with no manipulation)
question
Describe the kinds of evidence that support the construct validity of a measured variable.
answer
Reliability: how consistent is the measurement? Validity: is it measuring what it's supposed to measure?
question
Explain why a variable will usually have only one conceptual definition but can have multiple operational definitions.
answer
Conceptual definition is the researchers definition of the variable in question at a theoretical level. Operational definition represents a researcher's specific decision about how to measure or manipulate the conceptual variable.
question
Name three common ways in which researchers operationalize their variables.
answer
Self report Observational Physiological
question
Describe the differences between ordinal, interval, and ratio scales.
answer
Ordinal allows you to say "first, second, third" Interval has equal distance between level but no true zero so can't say "twice as much" etc. Ratio has an absolute zero
question
Reliability is about consistency. Define the three kinds of reliability, noting what kind of consistency each is designed to show.
answer
Test-retest reliability: test is given twice, scores from the two are compared to see if same Interrater reliability: two observers rate some type of behavior and agree with each other Internal reliability: do the different items correlate well with each other
question
For each of the three common types of operationalizations (self-report, observtional, and physiological) indicate which types of reliability would be relevant.
answer
Self-report: test-retest and internal reliability Observational: interrater reliability Physiological: interrater
question
Which of the following correlations is the strongest: r = .25, r = -.65, r = -.01, or r = .43?
answer
R= -.65 because it's closest to -1.0
question
What do face validity and content validity have in common?
answer
They both are subjective ways to assess validity.
question
To establish criterion validity, researchers make sure the scale or measure is correlated with
answer
Some relevant behavior or outcome.
question
Which requires stronger correlations for its evidence: convergent validity or discriminant validity?
answer
Convergent validity
question
Self-report measure
answer
People answer questions about themselves in a questionnaire or interview.
question
Observational measure
answer
Recording observable behaviors of physical traces of behaviors.
question
Physiological measure
answer
Recording biological data.
question
Categorical variable
answer
Variables whose levels are categorical
question
Quantitative variable
answer
Quantitative
question
Ordinal scale
answer
A quantitative measurement scale whose levels represent a ranked order, in which it is unclear whether the distances between levels are equal
question
Interval scale
answer
Quantitative measurement scale that has no "true zero," and in which the numerals represent equal intervals (distances) between levels
question
Ratio scale
answer
Quantitative measurement in which the numerals have equal intervals and the value of zero truly means "nothing"
question
Reliability
answer
The consistency of the results of a measure
question
Validity
answer
The appropriateness of a conclusion or decision.
question
Test-retest reliability
answer
The consistency in results every time a measure is used.
question
Interrater reliability
answer
The degree to which two or more coders or observers give consistent ratings of a set of targets.
question
Internal reliability
answer
In a measure that contains several items, the consistency in a pattern of answers, no matter how a question is phrased.
question
Correlation coefficient r
answer
A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables.
question
Slope direction
answer
The upward, downward, or neutral slope of the cluster of data points in a scatterplot.
question
Strength
answer
A description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn in through them.
question
Cronbach's alpha
answer
A correlation-based statistic that measures a scale's internal reliability.
question
Face validity
answer
Is this a plausible measure of the variable? (does it make sense at a gut level)
question
Content validity
answer
Does it capture all parts of a defined construct?
question
Criterion validity
answer
Both address how well a measure relates to a specific outcome. (Does it predict/correlate)
question
Known-groups paradigm
answer
A method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of that variable.
question
Convergent validity
answer
An empirical test of the extent to which a measure is associated with other measures of a theoretically similar construct.
question
Discriminant validity
answer
Empirical test of the extent to which a measure does not associate strongly with measures of other, theoretically different constructs.
question
Explain how carefully prepared questions improve the construct validity of a poll or survey.
answer
It is crucial that each question be clear and straightfoward to answer so it does not confuse respondents or influence their answers.
question
Describe how researchers can make observations with good construct validity.
answer
When they can avoid three problems: observer bias, observer effects, and reactivity
question
What are three potential problems related to the wording of survey questions? Can they be avoided?
answer
Leading questions, double-barreled questions, negative wording. Avoided by testing different wording... If the results are the same no matter the wording, it clearly doesn't matter... If results differ, may need to report results differently for each question
question
For which topics, and in what situations, are people most likely to answer accurately to survey questions?
answer
They make an effort to think about each question, they don't worry about looking good or bad, simply because they are unable Self-report is the best way.
question
What are some ways to ensure that survey questions are answered accurately?
answer
Ensure anonymity Use weed out items: Include "filler items" (Ex: Interested about racial attitudes, but also ask about politics, gender rolls, and education) Use implicit measures Ask about actions rather than attitudes
question
What is the difference between observer bias and observer effects? How can such biases be prevented?
answer
Train observers well Create clear rating scales (codebooks) Multiple observers
question
What is reactivity? What three approaches can researchers take to be sure people do not react to being observed?
answer
Blend in Wait it out: get used to researchers presence Measure the behavior's result (ex: empty liquor bottles in a residential garbage cans indicates how much alcohol is being consumed in community)
question
Survey
answer
A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the internet.
question
Poll
answer
A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet.
question
Open-ended question
answer
Survey question format that allows respondents to answer any way they like.
question
Forced-choice format
answer
Survey question format in which respondents give their opinion by picking the best of two or more options.
question
Likert scale
answer
Survey question format; a rating scale containing multiple response options that are anchored by the terms 'strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. Likert-type scale does not follow this format exactly.
question
Semantic differential format
answer
Response scale whose numbers are anchored with contrasting adjectives.
question
Leading question
answer
Type of question in a survey or poll that is problematic because its wording encourages only one response, thereby weakening its construct validity.
question
Double-barreled question
answer
Type of question in a survey or poll that is problematic because it asks two questions in one, thereby weakening its construct validity.
question
Negatively worded question
answer
Question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity.
question
Response set
answer
A shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item (aka nondifferentiation)
question
Acquiescence
answer
Answering "yes" or "strongly agree" to every item in a survey or interview
question
Fence sitting
answer
Playing it safe by answering in the middle of the scale fir every question in a survey or interview.
question
Socially desirable responding
answer
Giving answers on a survey that make one look better than one really is.
question
Faking good
answer
Same as socially desirable responding.
question
Faking bad
answer
Giving answers on a survey that make one look worse than one really is.
question
Observational research
answer
The process of watching people or animals and systematically recording how they behave or what they are doing.
question
Observer bias
answer
A bias that occurs when observers' expectations influence their interpretation of the participants' behaviors or the outcome of the study.
question
Observer effect
answer
A change in behavior of study participants in the direction of an observer's expectation. (aka expectancy effect)
question
Masked design
answer
Study design in which the observers are unaware of the experimental conditions to which participants have been assigned (aka blind design)
question
Reactivity
answer
A change in behavior of study participants (such as acting less spontaneously) because they are aware they are being watched.
question
Unobtrusive observation
answer
An observation in a study made indirectly, through physical traces of behavior, or made by someone who is hidden or is posing as a bystander.
question
Experimenter expectations
answer
Experimenter has certain expectations -> expectations alter experimenter's behavior toward participants -; expected response is more likely shown by participants
question
Rosenthall effect
answer
Rosenthal was one of the researchers for both the intellectual bloomers AND the maze bright/dull studies. Study showed that observers not only see what they expect to see; sometimes they even cause the behavior of those they are observing to conform to their expectations.
question
Ethics of behavioral observation
answer
Observing public behavior is considered ethical (no expectation of privacy) Researchers don't report on who they watched specifically Videotaping in public is usually okay too Using 1-way mirrors or private videotaping generally requires permission in advance
question
Explain why external validity often matters for a frequency claim.
answer
The findings need to generalize to a larger population or to other settings
question
Describe which sampling techniques allow generalizing from a sample to a population of interest, and which ones do not.
answer
Allow: Simple Stratified Proportionate Cluster, Multistage Not: Convenience sampling Purposive Snowball Self-selected
question
What are five techniques for selecting a representative sample of a population of interest? Where does randomness enter into each of these five selection processes?
answer
Simple Stratified Proportionate Cluster Multistage During random selection
question
In your own words, describe the difference between random sampling and random assignment.
answer
Random sampling: Researchers draw a sample randomly Random assignment: only used in experiments; researchers randomly assign participants to groups; helps ensure that the groups are the same at the start of the experiment
question
What are four ways of selecting a biased sample of a population of interest? Which subsets are more likely to be selected in each case.
answer
Convenience sampling: who is easiest to access Purposive sampling: Certain kinds of people they want to study Snowball sampling: Participants are asked to recommend few acquaintances Quota sampling: target a number for each category non randomly
question
Why are convenience, purposive, snowball, and quota sampling not examples of representative sampling?
answer
Because they are not random samples so they do not generalize to the population
question
Why do you think researchers might decide to use an unrepresentative sample, even though a random sample would ensure external validity?
answer
When studying association or causal claims. When it actually matters (reviews on shoes online)
question
When will it be most important for a researcher to use a representative sample?
answer
Frequency claim If you want to generalize beyond your sample, it matters. You can't always confirm this (you don't actually take a full census of an entire population that you've first sampled) Sometimes you can.... Election polling is confirmed (or not) by election results.
question
Which of these samples is more likely to be representative of a population if 100,000?
answer
A randomly selected sample of 100 people
question
Explain why a larger sample is not necessarily more representative than a smaller one.
answer
Larger sample = smaller margin of error Statistical term quantifying the degree of error in the study. If 28% of people support a piece of legislation, with a margin of error of 3, then if you did the pool over and over, 95% of the time your result would be between 25-31%.
question
Population
answer
A larger group from which a sample is drawn; the group to which a study's conclusions are intended to be applied.
question
Sample
answer
The group of people, animals, or cases used in a study; a subset of the population of interest.
question
Census
answer
A set of observations that contains all members of the population of interest.
question
Biased sample
answer
A sample in which some members of the population of interest are systematically left out, and as a consequence, the results from the sample cannot generalize to the population of interest (aka unrepresentative sample)
question
Representative sample
answer
A sample in which all members of the population of interest are equally likely to be included (usually through some random method) & therefore the results can generalize to the population of interest.
question
Convenience sampling
answer
Choosing a sample based on those who are easiest to access and readily available, a biased sampling technique.
question
Self-selection
answer
A form of sampling bias that occurs when a sample contains only people who volunteer to participate
question
Probability sampling
answer
The process of drawing a sample from a population of interest in such a way that each member of the population has an equal chance of being included in the sample, usually via random selection.
question
Simple random sampling
answer
The most basic form of probability sampling, in which the sample is chosen completely at random from the population of interest.
question
Cluster sampling
answer
A probability sampling technique in which clusters of participants within the population of interest are selected at random, followed by data collection from all individuals in each cluster.
question
Multistage sampling
answer
A probability sampling technique involving at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters
question
Stratified random sampling
answer
A form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories of interest and then randomly selects individuals within each category.
question
Oversampling
answer
A form of probability sampling; a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups.
question
Systematic sampling
answer
A probability sampling technique in which the researcher counts off members of a population to achieve a sample, using randomly chosen interval.
question
Random assignment
answer
The use of a random method to assign participants into different experimental groups.
question
Purposive sampling
answer
A biased sampling technique in which only certain kinds of people are included in a sample.
question
Snowball sampling
answer
A variation on purposive sampling, a biased sampling technique in which participants are asked to recommend acquaintances for the study.
question
Quota sampling
answer
A biased sampling technique in which a researcher identifies subsets of the population of interest, sets a target number for each category until the quotas are filled.
question
What types of sampling errors could occur through internet research/polling?
answer
Internet research uses a nonrandom sample Participants are self-selected volunteers Participants know how to use computers Participants have access to computers Participants are Internet savvy (maybe)
question
Bivariate correlation
answer
An association that involves exactly two variables
question
Mean
answer
An arithmethic average, a measure of central tendency computed from the sum of all the scores in a set of data, divided by the total number of scores.
question
t test
answer
A statistical test used to evaluate the size and significance of the difference between two means.
question
Effect size
answer
The magnitude of a relationship between two or more variables.
question
Statistical significance
answer
A conclusion that a result from a sample (such as an association or a difference between groups) is so extreme that the sample is unlikely to have come from a population in which there is no association or no difference.
question
Outlier
answer
A score that stands out as either much higher or much lower than most of the other scores in a sample.
question
Restriction of range
answer
A situation involving a bivariate correlation, in which there is not a full range of possible scores on one of the variables in the association, so the relationship from the sample underestimates the true correlation.
question
Curvilinear association
answer
An association between two variables in which is not a straight line; instead, as one variable increases, the level of the other variable increases and then decreases (vice versa)
question
Directionality problem
answer
A situation in which it is unclear which variable in an association came first.
question
Third-variable problem
answer
A situation in which a plausible alternative explanation exists for the association between two variables.
question
Spurious association
answer
A bivariate association that is attributable only to systematic mean differences on subgroups within the sample; the original association is not present within subgroups.
question
Moderator
answer
When the relationship between two variables changes depending on the level of a third variable, the third variable is the moderating variable.
question
At minimum, how many variables are there in an association claim?
answer
Two
question
What characteristic of a study's variables makes a study correlational?
answer
Both variables are measured
question
Sketch two bar graphs: one showing a correlation and one showing a zero correlation.
answer
A bar graph that shows a correlation should have bars at different heights A bar graph with zero correlation would show two bars of the same height.
question
When do researchers typically use a bar graph, as opposed to a scatterplot, to display correlational data?
answer
Categorical variables
question
In one or two brief sentences, explain how you would interrogate the construct validity of a bivariate correlation.
answer
Does it have good reliability? Is it measuring what it's intended to measure (validity)?
question
What are five questions you can ask about the statistical validity of a bivariate correlation? Do all the statistical validity questions apply the same way when bivariate correlations are represented as bar graphs?
answer
Effect size? Statistical significance? Are there subgroups? Outlier? If it looks like a zero correlation, could it actually be curvilinear?
question
Subgroups
answer
Sometimes there can be an apparent association, or lack thereof, but the association is different when you look more closely at at the subgroups. There could be subgroups (males and females, freshmen and seniors, etc) who both follow the pattern of the overall group.
question
Association b/t moderators and external validity
answer
When asking if an association will generalize to another age group, geographic group, etc, we are essentially asking if it could be moderated by that factor.
question
When is a small effect size still important?
answer
A small effect size could be important: taking an aspirin a day lowers heart attack risk, r = 0.03 Small effect size, but high consequences, and many lives saved
question
Correlational studies: What types of questions can these studies answer? What can't these studies tell us?
answer
When gathering data in the early stages of research When manipulating an independent variable is impossible or unethical When studying the relationship between two naturally occurring variables for the purpose of prediction. Correlational relationships can be used for predictive purposes
question
Understand and be able to recognize on a graph positive, negative, and zero correlations
answer
zero correlation is straight line.
question
3rd variable problem?
answer
There may be an unmeasured variable that actually causes variables to covary (change together)
question
Directionality problem?
answer
Not always possible to specify the direction in which a causal arrow points
question
When are these studies used?
answer
When gathering data in the early stages of research When manipulating an independent variable is impossible or unethical When studying the relationship between two naturally occuring variables for the purpose of prediction
Get an explanation on any task
Get unstuck with the help of our AI assistant in seconds
New