How can we help?

You can also find more resources in our Help Center.

Simple Experiment: Independent Groups Design

▪ Advantages

- Has internal validity (due to random

assignment)

▪ Disadvantages

- Each P provides only one score per condition

▪ Requires large # of Ps

- Individual differences may hide a small

treatment effect

- Has internal validity (due to random

assignment)

▪ Disadvantages

- Each P provides only one score per condition

▪ Requires large # of Ps

- Individual differences may hide a small

treatment effect

Match Pair Design:

PROCEDURE

Form matched pairs

▪ Randomly assign one member of each

pair to the treatment condition, the other

to the control condition

CONSIDERATIONS

Finding an effective matching variable

- Must create pairs that are very similar to each

other in terms of your DV

▪ External Validity

- Advantage: Doesn't restrict participant

population (i.e., can have heterogeneous

group)

- Disadvantage: Results may not generalize to

participants who haven't done the matching

task

Construct Validity may be weakened

because matching may tip off

participants about hyp

▪ Power: Can be a BIG plus

- Reduced random error (AS LONG AS

EFFECTIVE IN MATCHING VARIABLES) results

in larger t-values

- However, have fewer r df (pairs - 1), which may

cause reduced power

Form matched pairs

▪ Randomly assign one member of each

pair to the treatment condition, the other

to the control condition

CONSIDERATIONS

Finding an effective matching variable

- Must create pairs that are very similar to each

other in terms of your DV

▪ External Validity

- Advantage: Doesn't restrict participant

population (i.e., can have heterogeneous

group)

- Disadvantage: Results may not generalize to

participants who haven't done the matching

task

Construct Validity may be weakened

because matching may tip off

participants about hyp

▪ Power: Can be a BIG plus

- Reduced random error (AS LONG AS

EFFECTIVE IN MATCHING VARIABLES) results

in larger t-values

- However, have fewer r df (pairs - 1), which may

cause reduced power

The bigger the t-value the______

more likely the effect is statistically significant

Avantages of Within Subject Designs

▪ Increase Power (two factors)

- Tries to ELIMINATE random error due to

individual differences

▪ More observations (the more you have,

the greater opportunity you have for

random error to balance out)

- More observations = more power

- Tries to ELIMINATE random error due to

individual differences

▪ More observations (the more you have,

the greater opportunity you have for

random error to balance out)

- More observations = more power

Disadvantages of Within Subject Designs

▪Construct validity threatened

- P, who receives at least two levels of IV and

performs DV task at least twice, has multiple

opportunities to figure out hypothesis

▪ Internal Validity may be weakened due

to order effects (4 of them - KNOW

THESE)

- Testing (i.e., practice) effects -> get better on

DV

- Fatigue effects -> get worse on DV

▪ Can be considered a negative practice effect

- Treatment carryover effects

▪ Effect of an earlier treatment lingers

- E.g., drug given earlier may affect performance on

later trial

- Sensitization

▪ Threatens internal validity because they behave

different AFTER they figure out what the hyp is.

- P, who receives at least two levels of IV and

performs DV task at least twice, has multiple

opportunities to figure out hypothesis

▪ Internal Validity may be weakened due

to order effects (4 of them - KNOW

THESE)

- Testing (i.e., practice) effects -> get better on

DV

- Fatigue effects -> get worse on DV

▪ Can be considered a negative practice effect

- Treatment carryover effects

▪ Effect of an earlier treatment lingers

- E.g., drug given earlier may affect performance on

later trial

- Sensitization

▪ Threatens internal validity because they behave

different AFTER they figure out what the hyp is.

Within-Subjects Design: Dealing with order Effects

▪ Minimize each individual threat

- Testing (i.e., practice) effect

▪ Give Ps extensive practice before beginning

- Fatigue/boredom (i.e., negative practice effect)

▪ Make it interesting, undemanding, and short

- Carryover effect

▪ Lengthen times between different treatments

- Sensitization

▪ Hide what you are varying and use unobtrusive DV

▪ Use as few levels as possible to reduce

opportunities for each individual threat

- Testing (i.e., practice) effect

▪ Give Ps extensive practice before beginning

- Fatigue/boredom (i.e., negative practice effect)

▪ Make it interesting, undemanding, and short

- Carryover effect

▪ Lengthen times between different treatments

- Sensitization

▪ Hide what you are varying and use unobtrusive DV

▪ Use as few levels as possible to reduce

opportunities for each individual threat

Counterbalanced Within Subjects Designs:Procedure

▪ Devise set of sequences such that:

- Every condition appears in every position the

same number of times

▪ e.g., C1 is first as many times as it is last

- Every condition precedes every other condition

as many times as it follows that condition

▪ e.g., # C1 comes before C2 = # C2 comes before

C1

▪ Randomly assign Ps to your sequences

- Every condition appears in every position the

same number of times

▪ e.g., C1 is first as many times as it is last

- Every condition precedes every other condition

as many times as it follows that condition

▪ e.g., # C1 comes before C2 = # C2 comes before

C1

▪ Randomly assign Ps to your sequences

Counterbalanced Designs Advantages

Main Advantages

- Ensures that routine order effects are balanced

across conditions

▪ e.g., 2 trx and 8 Ps

4 Ps will get A then B

4 Ps will get B then A

- Opportunity to learn about effect of the withinsubjects variable of order (e.g., trials,

positions, etc.)

- Opportunity to learn about the effect of the

between-subjects variable of sequence

- Ensures that routine order effects are balanced

across conditions

▪ e.g., 2 trx and 8 Ps

4 Ps will get A then B

4 Ps will get B then A

- Opportunity to learn about effect of the withinsubjects variable of order (e.g., trials,

positions, etc.)

- Opportunity to learn about the effect of the

between-subjects variable of sequence

Counterbalanced Designs Disadvantages

May require more subjects

- Therefore, more resources (e.g., time, money,

etc)

▪ Analysis becomes much more

sophisticated

- May be quite difficult for the beginner to

understand

- Therefore, more resources (e.g., time, money,

etc)

▪ Analysis becomes much more

sophisticated

- May be quite difficult for the beginner to

understand

Possible Results from 2x2 Counterbalanced Design

Main effect of Trx

- Between-Groups comparison

▪ Main effect of counterbalancing sequence

(CB)

- Between-Groups comparison

▪ Trx x CB interaction`

- Between-Groups comparison

▪ Main effect of counterbalancing sequence

(CB)

- Between-Groups comparison

▪ Trx x CB interaction`

Counterbalanced WithinSubjects Designs: Summary

▪ Balances out routine order effects

▪ Provides info not only about the effect of

the treatment, but also about the effect

of order and sequence

▪ Provides info not only about the effect of

the treatment, but also about the effect

of order and sequence

Four approaches to the order problem

▪AVOID the problem by using betweensubjects designs instead

▪ Randomize sequence of treatments

▪ Randomly assign Ps to

counterbalanced sequences

▪ Reduce sources of order effects (i.e.,

practice, fatigue, carry-over,

sensitization)

▪ Randomize sequence of treatments

▪ Randomly assign Ps to

counterbalanced sequences

▪ Reduce sources of order effects (i.e.,

practice, fatigue, carry-over,

sensitization)

Within-Subjects Design:

Dealing with Order Effects

Dealing with Order Effects

▪ Minimize each individual threat

- Testing (i.e., practice) effect

▪ Give Ps extensive practice before beginning

- Fatigue/boredom (i.e., negative practice effect)

▪ Make it interesting, undemanding, and short

- Carryover effect

▪ Lengthen times between different treatments

- Sensitization

▪ Hide what you are varying and use unobtrusive DV

▪ Use as few levels as possible to reduce

opportunities for each individual threat

- Testing (i.e., practice) effect

▪ Give Ps extensive practice before beginning

- Fatigue/boredom (i.e., negative practice effect)

▪ Make it interesting, undemanding, and short

- Carryover effect

▪ Lengthen times between different treatments

- Sensitization

▪ Hide what you are varying and use unobtrusive DV

▪ Use as few levels as possible to reduce

opportunities for each individual threat

Mixed Designs

▪Combining between-subjects and withinsubjects design

- One (or more) factors varied BETWEEN groups

- One (or more) factors varied WITHIN groups

▪ Allows for both internal validity, power,

and possibly external validity

- One (or more) factors varied BETWEEN groups

- One (or more) factors varied WITHIN groups

▪ Allows for both internal validity, power,

and possibly external validity

Choosing an Experimental

Design: General Considerations

Design: General Considerations

▪ Pure between-subjects designs may:

- Have more construct validity because harder

for participants to guess the hypothesis

- Have more internal validity because they are

not vulnerable to order effects

- Easier to analyze

▪ Within-subjects designs have more

power

▪ External validity depends on whether the

variable is "within-subjects" or "betweensubjects" in real life

- Have more construct validity because harder

for participants to guess the hypothesis

- Have more internal validity because they are

not vulnerable to order effects

- Easier to analyze

▪ Within-subjects designs have more

power

▪ External validity depends on whether the

variable is "within-subjects" or "betweensubjects" in real life

Choosing Designs: The Two

Conditions Case

Conditions Case

▪Pure between-subjects design

▪ Matched-pairs design

▪ Randomized within-subjects designs

▪ Counterbalanced designs

▪ Matched-pairs design

▪ Randomized within-subjects designs

▪ Counterbalanced designs

Choosing Designs:Multiple IVs

▪ Use a within-subjects factorial design

▪ Use a between-subjects factorial design

▪ Use a mixed design (both within- and

between-subjects design)

▪ Use a between-subjects factorial design

▪ Use a mixed design (both within- and

between-subjects design)