Research methods advantages and disadvantages

Naturalistic Observation
Systematically observig people in their natural environment, includes conceptualisation and operationalisation of variables

Naturalistic Observation: Advantages
Insight into real-world behaviour, examine behaviours that can’t be manipulated

Naturalistic Observation: Disadvantages
Time consuming, subjective interpretation

Participant Observation
Researcher interacts with people being observed in order to observe and record their behaviour

Participant Observation: Advantages
Obesrve behaviours that aren’t usually accessible, unique perspective of participation

Participant Observation: Disadvantages
Subjective Interpretation

Contrived Observation
Observing behaviour that has been manipulated by the researcher

Contrived Observation: Advantages
Don’t have to wait for behaviour to occur on its own

Contrived Observation: Disadvantages
Less natural than other forms of observation

Type 1 Error
Reject a true null hypothesis, alpha-level

Type 2 Error
Don’t reject null hypothesis when it is false, beta-level

Effect size
strength of association between variables

Effect size examples
Pearson’s correlation, Cohen’s D

Cohen’s D Formula
difference between means/standard deviation

Confidence Intervals
Where the true population mean is likely to lie

Confidence Intervals: Rules
Sample size increases – CI narrows; Higher confidence required – CI widens

Independent Samples T-tests
Two samples obtained from two populations to be compared

Independent Samples T-tests: Assumptions
Populations follow normal distribution, Equal variance, Samples sizes equal

Independent Samples T-tests: DF
n – 2

Descriptive Uncertainty
Uncertainty of random error; small sd = less uncertainty

Interential uncertainty
More information/less error = smaller p-value

Bonferroni Correction
p-value to the power of n to provide new p-value

Independent Samples T-tests: Disadvantages
Subject to “noise factors”

Observer Error
Human error in measurement process

Observor Error: examples
Maturation, Experimentation, Faciliatated Communication

Environmental Error
Changes between measurements in the environment

Environmental Error: examples

Participant Error
Changes in participants between measurements

Participant Error: Examples
Demand characteristics, expectations/placebo effect, mortality, regression to mean, instrumentation, selection differences, order/practice/fatigue, repeated testing

Controls for Error
Type of experimentation, pilot testing, counterbalancing, deception, debriefing and research communication

Designs that approximate a true experiment

N=1 Designs
Single Case design, for unusual cases or expensive manipulations, or when it isn’t necessary to have a large group

N=1 Design: Types
Reversal (ABA), Multiple Baseline (across subjects, behaviours, or situations)

N=1 Design: Constraints
Permanent or enduring manipulations cannot use reversal design, nor when it would be unethical to reverse a beneficial manipulation

Programme Evaluation
Research on proposed and implented programs to improve something

Programme Evaluation: Needs assessment
Do people need to change?

Programme Evaluation: Programme theory assessment
Do you have valid assumptions about the cause of the problem, and therefore why the program will work?

Programme Evaluation: Process evaluation
a manipulation check; Is the programming doing what it advertised it would do?

Programme Evaluation: Outcome evaluation
Did the programme reduce the problem?

Programme Evaluation: Efficiency assessment
Is the programme worth the cost?

One Group Posttest/One group pretest-posttest
No control group, and many threats to validity (history, maturation, testing, instrumental decay, regression toward mean, differential attrition)

Nonequivalent control group design
No pretest, so is hard to verify change despite the fact that there is a control group as we do not know if the two groups are equivalent or not

Nonequivalent control group pretest-posttest
If the pretest scores are similar, therefore the groups are comparable

Quasi-Experimental: Statistical test
t-tests and ANOVA

Data collected at one point in time on participants of different ages

Cross-sectional: Advantages
Can make general comments about how individuals of different ages perform

Data collected at regular intervals from individuals of a specific age

Cohort: Advantages
Tells you whether there are general changes in time over how a certain age group is thinking or behaving

Data colected from a single cohort at regular intervals over time

Longitudinal: Advantages
Obtain information about causality, and can see individual changes over time

Longitudinal: Disadvantages
Expensive, time-consuming, and limiting to one cohort

Hybrid between cross-sectional study (at time 1), and longitudinal study (follow the cohort through subsequent times)

Sequential: Advantages
Can obtain data from a wide age range quickly, as well as assess change, and not limited to a single cohort

Developmental Research
Assessing change, short and/or long term

Developmental Research: statistical test
MANOVA, Regression, Equation Modelling

Questionnaire Problems
Awareness/memory, response set/bias, question format, sample chosen, analysis done

Yes/No Disadvantages
Not very sensitive

Fill-in-the-blank Disadvantages
Hard to code the answers

Likert Scale Disadvantages
Difficult to have equal distances between points

Aspects to avoid in questionnaires
Complexity, Technicality, Ambiguity, Double-Barreled, Negative, Emotive or Sensitive issues, Leading questions, Privacy invasion