Research Methods Exam 3 – Flashcards

Unlock all answers in this set

Unlock answers
question
What is the main reason that researchers do experiments? A) for more validity B) to generalize to more people C) to collect evidence in the real world D) to make causal claims
answer
D) to make causal claims
question
manipulated variable
answer
variables that re completely under the control of the experimenter and are set by the experimenter for the purpose of manipulating the levels of the IV
question
Measured Variables
answer
variables that are simply observed and recorded; this could be the DV (outcome variable), other variables usually at least partly determined by or built-in the the subject, or things that cannot be completely controlled by the experimenter but are of interest to measure (e.g. gender, age)
question
Does experimental manipulation allow for causal claims?
answer
YES
question
control variable
answer
a variable that an experimenter controls for the purpose of holding it constant across the levels of the IV -any variable that an experimenter holds constant on purpose
question
The Elliot et al study (about color and perceived attractiveness) is an example of a ___ experimental design. A) correlational B) within-subjects C) between subjects D) quasi-experimental
answer
C) between subjects
question
In the Elliot et al attraction study, what is the most obvious possible design confound that could affect the internal validity of their conclusion? A) the gender of the participants B) color in general C) lighting in the room D) education of the participants
answer
B) color in general
question
design confound
answer
another variable that accidentally varies systematically along with the IV that could also explain the change in the DV.
question
a threat to internal validity occurs only if a potential design confound varies ___ with the independent variable. A) randomly B) systematically C) cyclically D) spuriously
answer
B) systematically
question
In order to be a confound the variable must change ____ with the independent variable
answer
In parallel (they should be correlated)
question
in order for a confound to be a problem, the variable must be ____ the dependent variable.
answer
capable of influencing
question
when and why do selection effects occur?
answer
when systematically different types of participants are in the two groups. relevant for between-groups designs only. Different groups of people are exposed to different levels of the IV. It is the failure of random assignment
question
Does the Elliot et al attraction study demonstrate temporal precedence? A) no, because t wasn't a longitudinal study B) no, because the manipulated variable came after the dependent variable C) yes, because it was a longitudinal study D) yes, because the manipulated variable came before a difference was observed in the dependent variable
answer
D) yes; the IV was manipulated before the difference was observed in the DV
question
With a pretest/posttest design, how can you tell if random assignment worked?
answer
The first measure of the DV should be the same across groups and the second measure of the DV should be different.
question
In order to meet the strict definition of 'experiment,' ___. A) the researcher must be wearing a white lab coat B) the data must be collected in a sound-proof, semi-darkened booth C) all possible confounds must be eliminated D0 at least one of the variables must be manipulated by the researcher
answer
D) at least one manipulated variable.
question
How do experiments support causal statements?
answer
1. covariance- does the DV change? 2. temporal precedence: measure of DV follows manipulation of IV 3. internal validity: most important. Design confounds, selection effects, etc.
question
counter-balancing
answer
in a repeated measures experiment, presenting the levels of the IV to participants in different sequences to control for order effects
question
order effects
answer
when the exposure to one condition (i.e. level of the IV) changes how participants react to the other condition (i.e. other levels of the IV)
question
what is the most significant threat to internal validity for within subjects designs and how can you address them?
answer
order effects, address them by counterbalancing
question
4 was to counterbalance
answer
full counter balancing, partial counter balancing using random ordering, partial counter balancing using a Latin Square, and partial counter balancing with a balanced Latin Square
question
full counter-balancing
answer
all possible orders are used across different subjects. there are K! possible orders for K conditions
question
partial counter-balancing using random ordering
answer
each subject gets the conditions in a pseudo-random (shuffled) or random order
question
partial counter-balancing using a Latin square
answer
K different orders are created such that each condition occurs in each position once
question
partial counter-balancing with a balanced Latin square
answer
latin square where each condition is also followed by each other condition exactly once
question
between subjects
answer
order held constant because all conditions "first" to a given group and individual differences are hopefully equalized because of random assignment
question
within subjects
answer
individual differences held constant because each person is exposed to each level of the IV and order is equalized on average because of counter-balancing
question
upsides to between-subjects design
answer
-need for a control condition that doesn't get exposed to an experimental condition. -use of a manipulation that could have permanent or long-lasting effect once there is exposure -use of a non-repeatable dependent measure -strong demand characteristics coupled with a need for naive subjects
question
downsides to between-subjects design
answer
-requires many more subjects to achieve equal power -possibility of selection effects if your sample is on the smaller side and/or population is widely variable in the DV
question
upsides to within-subjects design
answer
interest in a small effect, brief experiments, and the same subject population
question
downsides to within-subjects design
answer
increased potential for demand characteristics (cuing the clients in on the primary research question) and variety of possible carry-over effects/order effects
question
Eric designs a study to examine drink preferences of university freshmen. He is planning to have all the freshmen who participate in his study drink a cup of coffee then rate their enjoyment of the coffee, then drink a cup of tea and rate their enjoyment, and finally drink milk and rate enjoyment. 1. What is the IV and the DV? 2. what kind of design is this? 3. Eric tells Theresa about the plans for his study and she says she is concerned about an order effect. How can Eric fix this?
answer
1. IV: type of drink DV: rating of enjoyment 2. within-subjects, repeated measures 3. Counterbalance the groups
question
All of the following are advantages of within-groups except: A) participants in the treatment/control groups will be equivalent B) there is a smaller chance of participants detecting the experimental manipulation C) it gives researchers more power to find differences between conditions D) they require fewer participants
answer
B) there is actually a bigger chance of demand characteristics
question
What are the minimum requirements for a study to be an experiment?
answer
has to have at least 1 manipulation
question
comparison group
answer
the other group in the study to give it something to compare to. For example; if you only measures how much one group of people consumed, you wouldn't know if it was more or less than the other group without a comparison group.
question
placebo group
answer
when the control group is exposed to an inert treatment such as a sugar pill
question
confounds
answer
alternative explanations, potential threats to internal validity
question
systematic variability
answer
varies along with the IV, therefore a potential compound
question
unsystematic variability
answer
random; therefore cannot be considered a confound
question
selection effect
answer
occurs in an experiment when the kinds of participants in one level of the IV are systematically different from the participants in another level of the IV
question
2 ways to avoid selection effects
answer
random assignment and matched groups
question
matched groups
answer
first measure the participants on a particular variable that might matter to the DV; for example IQ might matter to anagram ability. Then they would match participants based on this score to randomly assigned groups
question
posttest only design
answer
participants are randomly assigned to independent variable groups and are tested on the DV once
question
pretest/posttest design
answer
participants are randomly assigned to at least 2 groups and are tested on the key dependent variable twice--once before and once after exposure to the IV
question
concurrent measures design
answer
participants are exposed to each level of the IV at roughly the same time and there is one measure of the DV
question
repeated measures design
answer
participants are exposed to each level of the IV at separate times and the DV is measured after each level
question
power
answer
refers to the ability of a study to show a statistically significant result when an IV truly has an effect on the DV
question
2 types of order effects
answer
practice effects and carryover effects
question
practice effects
answer
also known as fatigue effects; a long sequence might lead participants to get better at the task, or to get tired and bored toward the end
question
carryover effects
answer
some form of contamination carries over from one condition onto the next
question
3 main disadvantages to within subjects design
answer
1. order effects (can be managed by counterbalancing) 2. may to be possible or practical. (can't teach a kid to ride a bike twice and measure which one worked better) 3. demand characteristics (catch on to the primary research question)
question
construct validity
answer
how well were the variables measured and manipulated?
question
manipulation check
answer
extra dependent variable that researchers can insert into an experiment to help them quantify how well a manipulation worked
question
pilot study
answer
simple study, using different groups of people, completed before conducting the study of primary interest
question
what type of validity do manipulation checks and pilot studies address?
answer
construct validity within the manipulation.
question
external validity
answer
to whom or what can the claim generalize to?
question
random sampling
answer
how did the researchers get the participants? were they randomly sampled from the population?
question
what do researchers generally prioritize, internal or external validity?
answer
internal
question
statistical validity
answer
how well do the data support the causal claim?
question
what are the 2 basic concerns for statistical validity?
answer
significance and effect size
question
statistically significant
answer
unlikely that the effect happened by chance
question
effect size
answer
the larger the effect size, the more important, and stronger, the causal effect
question
internal validity
answer
are there alternative explanations for the outcome?
question
3 questions for internal validity
answer
1. were there any design confounds? 2. did the experimenters control for selection effects? 3. did they control for order effects?
question
Using a matched-group design is especially important in which of the following cases? A) when you have a relatively small sample in your study B) when you have at least 3 levels of the IV C) when you do not have a control group D) when you are worried about order effects
answer
A) small sample; because random assignment can fail
question
order effects can be controlled by using which of the following techniques? A) random assignment B) design confounds C) increasing demand characteristics D) counterbalancing
answer
D) counterbalancing
question
In an experiment, the reporter wears the same outfit and answers the questions the same way. these are examples of ___ variables in the study A) confound B) covariate C) control D) mediator
answer
C) control variables
question
maturation
answer
a change in behavior emerging due to a phenomena that naturally occurs with time
question
history
answer
something that happened between the pre test and the post test that could improve performance (other than the manipulation of the IV)
question
regression
answer
extreme at pretest and likely to be less extreme at posttest, no matter what effect the IV had
question
attrition
answer
systematic dropout (some particular type of participants likely to drop out)
question
testing effects
answer
scores improve because the participants do a test more than once (practice) or their scores get worse because of fatigue
question
instrumentation
answer
measurement tool changes from pre to post test
question
Jared is conducting an experiment with ESP (extrasensory perception) training. He begins with a pretest of his 40 participants and divides them into two groups based on their scores. The participants with the 10 lowest scores are given extensive training on how to detect the signals. The participants with the 30 highest scores are given no training. Both groups are retested and the average score of the participants with the training improved, while the average score of the participants without the training actually fell. What threat to internal validity should Jared consider? A) History B) Maturation C) Regression D) Attrition
answer
C) regression to the mean
question
which of the following is true of testing effects? A) It is an example of an order effect B) it can lead to increased/higher scores OR decreased/lower scores C) it can sensitize people to being tested D) all of the above
answer
D) all of the above
question
The addition of a comparison group can address all of the following threats to internal validity EXCEPT: A) history B) attrition C) maturation D) regression
answer
B) attrition (systematic dropouts)
question
which of the following experimental designs is vulnerable to MANY internal validity threats? A) post test only B) pre test only C) concurrent measures D) one group, pretest/posttest
answer
D) one group design. doesn't have comparison
question
observer bias
answer
when there is a behavioral dependent variable and the observer's expectations influence their interpretation of the results
question
demand characteristic
answer
anything that would give the experimental group the expectation to improve due to the awareness of the task
question
placebo effects
answer
the participants aren't explicitly trying to help the experimenters prove their hypothesis, but placebo effects are the expectation to improve
question
what threat to internal validity occurs when there is a greater systematic loss of participants in one condition than in the other? A) history B) maturation C) regression D) attrition
answer
D) attrition
question
on the first exam in psych class, the grades are over than the professor expected. She suspects that multitasking is to blame, so she bans computers and cell phones from her class. when she compares the two scores on the two exams, she finds a significant improvement on the second exam. She looks back over the exam and wonders in it was just an easier exam. What threat to internal validity is she considering? A) instrumentation B) history C) maturation D) testing
answer
A) instrumentation because she is questioning if her exam changed and was easier. this is the way that the DV was measured. testing effects is when the students get better at something because they have been exposed more than once, but the tests were different, so this isn't the case.
question
if results show that video game training does not influence attention in children, what would we call this? A) failed experiment B) alternative hypothesis C) null result D) post-hoc hypothesis
answer
C) null
question
12 threats to internal validity
answer
design confounds, selection effects, order effects, maturation threats, history threats, regression threats, attrition threats, testing threats, instrumentation threats, observer bias, demand characteristics, and placebo effects
question
maturation threat
answer
change in behavior that emerges more of less spontaneously over time. also known as spontaneous remission.
question
how to prevent maturation threats
answer
comparison groups
question
history threats
answer
alternative explanation for the change in DV, due to an external event that effects most members of the treatment group at the same time as the treatment
question
preventing history threats
answer
comparison group
question
regression threat
answer
statistical concept called regression to the mean; when performance at the pretest is extreme, the next time it is likely to be less extreme. -occur only in a pretest/posttest design
question
attrition threats
answer
reduction in participant numbers that occurs when people drop out before the end of the experiment
question
testing threat
answer
specific kind of order effect; change in the participants as a result of taking a test more than once
question
instrumentation threat
answer
occurs when a measuring instrument changes over time
question
instrumentation vs. testing threats
answer
instrumentation: measuring instrument changes testing: participant changes
question
selection-history threat
answer
outside event or factor systematically affects people in the study--but only those in one level of the IV
question
selection-attrition threat
answer
only one of the experimental groups experiences attrition
question
placebo effect
answer
occurs when people receive a treatment and really improve--only because they believe they are receiving a valid treatment
question
double-blind placebo control study
answer
neither the people treating the patients nor the patients themselves know whether they are in the real group or the placebo group
question
null effect
answer
IV didn't make a difference on the DV, there is no significant covariance between the two
question
why could a null effect happen when there actually is an effect?
answer
weak maniplations, insensitive measures, and ceiling/floor effects
question
weak manipulations
answer
is the manipulation strong enough to change the DV
question
insensitive measures
answer
the operationalization of the DV isn't sensitive enough to pick up on the change
question
ceiling effect
answer
all the scores are squeezed together at the high end of the DV
question
floor effect
answer
all the scores cluster at the low end of the DV
question
how can the IV cause ceiling and floor effects
answer
if there is not enough variability in the IV, it may cause a floor effect because it isn't significant enough to change the DV and vice versa
question
noise
answer
too much unsystematic variability within each group
question
measurement error
answer
any factor that can inflate or deflate a person's true score on a dependent measure
question
not enough variability between levels
answer
ineffective manipulation of the IV, insufficient sensitive measurement of the DV, ceiling or floor effects on the IV or DV
question
too much variability between levels
answer
measurement error, individual differences, and situation noise
question
6 threats relevant to the one group, pretest/posttest design
answer
history, attrition, maturation, regression, testing, and instrumental threats
question
Dr. Weber conducted a long-term study on friendship. He noticed that the most introverted people dropped out by the third session. Therefore, his study might have which of the following internal validity threats? A) attrition B) maturation C) selection D) regression
answer
A
question
How is a testing threat to internal validity different from an instrumental threat? A) a testing threat can be prevented with random assignment; an instrumentation threat cannot B) a testing threat applies only to within subjects designs; instrumentation threats are for any design C) testing threat can be prevented with a double blind study instrumentation threats can be prevented with placebo control D) testing threat refers to a change in the participants over time; instrumentation threats refer to a change in the measuring instrument over time
answer
D
question
regression threat applies especially: A) when there are 2 groups in the study; an experimental group and a control group B) when the researcher recruits a sample whose average is extremely low or high at pretest C) in a posttest only design D) when there s a small sample
answer
B
question
Dr. Banks tests to see how many training session sit takes for dogs to learn to sit and stay. She randomly assigns 60 dogs to 2 reward conditions; one in which the reward is miniature hot dogs and one in which the reward is a small piece of steak. Surprisingly, she finds that the dogs learn in about the same amount of time. Given the design, what is the most likely explanation for the null effect? A) dogs loved both treats (reward manipulation has ceiling effect) B) she used too many dogs C) she didn't use a manipulation check D) there were too many individual differences among the dogs
answer
A
question
She modifies her design and conducts a second study. she used the same number of dogs and the same design except now she rewarded one level with hot dogs and one level with apples. She found a big difference. she avoided the null effect because she: A) increased the between group variability B) decreased the within groups variability C) improved the studies internal validity
answer
A
question
when a study has a large number of participants and a small amount of unsystematic variability (low measurement error, low levels of situation noise), then it has a lot of: A)internal validity B) manipulation checks C) dependent variables D) power
answer
D
question
interaction
answer
joint effect of IVs; in a factorial design when the effect of one IC depends on the levels of another IV
question
main effects
answer
effect of each individual IV; overall effect of one IV on the DV, averaging over the levels of the other IV
question
in the case of a factorial design, another term for IV is A) interaction B) main effect C) cell D) factor
answer
D) factor
question
the number of main effects that need to be examined is ___ the number of IVs A) more important than B) equal to C) similar to D) independent of
answer
B) equal to
question
IN a 2x2 factorial design, what are all the possible effects to test? A) one main effect and one 2 way interaction B) one main effect and 2 2 way interactions C) 2 main effects and 1 2 way interaction D) 2 main effects and 2 2way interactions
answer
C) 2 main effects (one for each IV) and 1 interaction
question
independent samples t-test
answer
used when there are 2 experimental conditions and different participants were assigned to each condition (between subjects)
question
paired samles t-test
answer
used when there are 2 experimental conditions and the same participants took part in both conditions of the experiment (within subjects)
question
what can you do if you have too much variability?
answer
change to a thin subjects design, increase the sample size, or do a matched group design o reduce variability within each group
question
what if there isn't enough variability between groups?
answer
manipulation check, pilot study, make sure people can do the tasks but make sure they can't do them too well
question
which of the following study types can result in a null effect? A) pretest/posttest B) correlational C) within groups D) all of the above
answer
D) all of them
question
Dr. is curious is exposing people to violent video games causes them to be more aggressive. He assigns half his participants to play a video game for 5 minutes and theater to play for 7 minutes. He dfinds no relationship. What might be to blame? A) weak manipulation B) insensitive measure C) reverse confound D) ceiling effect
answer
A) weak manipulation
question
all of the following are true of ceiling and floor effects except: A) they can be detected by manipulation checks B) they are only problematic in pretest/posttest designs C) they can be caused by poorly designed DVs D) they can caused by poorly designed IVs
answer
B) they are NOT only problematic in pretest/posttest designs
question
interaction effect
answer
whether the effect of the original IV depends on the level of another IV; difference of differences
question
factorial design
answer
there are two or more independent variables
question
participant variable
answer
variable whose levels are selected (i.e. measured) not manipulated
question
marginal means
answer
arithmetic means for each level of an IV, averaging over the levels of the other IV
question
what is the most important effect in a factorial study
answer
when a study's results show an interaction; this is the overall effect
question
how to detect interactions from a factorial design graph
answer
if the lines are parallel, there is no interaction. if the lines are not parallel, there is probably an interaction
question
2 main reasons to use factorial designs
answer
test limits and to test theories
question
between subjects factorial design
answer
each cell has a different group of participants
question
within groups factorial design
answer
each person is exposed to each cell; each level of the IV
question
mixed factorial design
answer
one IV is manipulated as independent groups and the other is manipulated as within-groups
question
Cara is running a study to examine the effect of music on mood. She randomly assigns participants to three conditions — rock, jazz, and country. She has the participants rate their mood with a short questionnaire, then listen to their assigned music for 20 minutes, and then fill out the mood questionnaire again. What kind of design is she using? A) post test only B) repeated measures C) pretest/postest D) concurrent measures
answer
C) pretest/posttest; it is a between subjects design
question
What is the name for a variable that the experimenter holds constant on purpose? A) dependent variable B) independent variable C) confound variable D) control variable
answer
D) control variable
question
What is the name for the level of the independent variable that is intended to represent a neutral condition? A) control group B) manipulation group C) treatment group D) experimental group
answer
A) control group
question
Participants in a research study are given a list of words to study for 3 minutes and then, following a delay, are asked to recall the list. The length of the delay is manipulated between participants to be either 2 minutes, 5 minutes, or 10 minutes. Which of the following scenarios would present a design confound in this experiment? A) all participants in the 2 min condition are tested at 8am, those in the 5 min condition are tested at noon, and those in the 10 min condition are tested at 4 B) the three groups are run simultaneously in 3 different rooms and the room for each condition is randomly chosen before each group arrives C) the same list is used for each condition and is randomized for each participant D) three different experimenters are in charge of administering the task, and they rotate which condition they are administering
answer
A) time of day could confound the results
question
____________ is an extra dependent variable that can be used to help researchers quantify how well an experimental manipulation worked. A) Latin square B) pilot study C) manipulation check D) counterbalancing
answer
C) manipulation check
question
which of the following is NOT one of the 3 most common threats to internal validity? A) order effects B) floor effects C) selection effects D) design confounds
answer
B) floor effects
question
Emma is planning an experiment to examine whether reading to children increases their vocabulary size. She plans to measure the vocabulary size of a group of 18-month-olds, read to them three times a week for three months, then measure their vocabulary size again. How could Emma change the design of the study to remove a major threat to internal validity? A) she could counterbalance her conditions B) she could make it a double blind placebo control study C) she could add a comparison group D) she could ass a manipulation check
answer
C) comparison group
question
Which of the following is NOT a problem that may contribute to a null effect in a study? A) insensitive measures B) design confound acting in reverse C) large sample size D) weak manipulation
answer
C) large sample size
question
What two types of validity in an experiment can observer bias threaten? A) statistical validity and external validity B) internal validity and external validity C) construct validity and statistical validity D) internal validity and construct validity
answer
D) internal and construct
question
An instructor hypothesizes that doing jumping jacks will improve his students' quiz performance. On Monday, he has his class sit in their chairs for five minutes before completing a multiple-choice quiz on their reading assignment. On Wednesday, he has his class do two minutes of jumping jacks before completing the same quiz that they took on Monday. The students performed better on the quiz on Wednesday. What is a possible threat to internal validity in this study? A) maturation threat B) testing threat C) observer bias D) attrition threat
answer
B) testing threat.
question
IN a factorial design study, which effect is usually considered the most important effect? A) the main effect of the second variable B) the main effect of the first IV C) the interaction D) the overall effect
answer
C) the interaction
question
Koordeman et al. (2011) conducted a study on the effect of alcohol commercials on alcohol consumption in a movie theater. Participants saw one of two types of beverage commercials: alcoholic or nonalcoholic. Their consumption of alcohol during the movie was then measured. Finally, they completed a questionnaire on their usual drinking habits. Koordeman et al. found that young adults who generally consume large amounts of alcohol each week were influenced to drink more alcohol when they saw alcohol commercials, whereas the alcohol consumption of young adults who generally consume small amounts of alcohol each week was not influenced by the type of commercial shown. Which of the following is a factor in this study? A) the movie shown B) type of commercial shown C) young adults who consume small amounts of alcohol each week D) amount of alcohol consumed during the movie
answer
B) type of commercial shown
question
Susan designed a study in which she had a group of younger adults (18-24 year-olds) and older adults (50-65 year-olds). She had both groups recite a poem by memory once in front of an audience of 50 people and once in a room by themselves. She counterbalanced the order of these tasks between participants. She had the participants rate their level of anxiety right before they recited the poem each time. What is the factorial notation for this design? A) 2 x 2 B) 1 x 2 C) 2 x 2 x 2 D) 2 x 4
answer
A) 2x2; IVs are age and people poem was recited in front of. DV was level of anxiety
question
What are the two main reasons researchers use factorial designs? A) to use fewer participants and test the limits of an effect B) test limits of an effect and test theories C) increase construct validity and increase internal validity D) increase construct validity and test theories
answer
B) test limits of an effect and test theories
question
How many independent variables and how many cells are there in a 2 × 3 × 4 study? A) 3 IV, 9 cells B) 2 IV, 9 cells C) 3 IV, 24 cells D) 2 IV, 4 cells
answer
3 IV, 24 cells
Get an explanation on any task
Get unstuck with the help of our AI assistant in seconds
New