Search
Create
Log in
Sign up
Log in
Sign up
Get ahead with a $300 test prep scholarship
| Enter to win by Tuesday 9/24
Learn more
Research Methods Chapter 9: Factorial Designs
STUDY
Flashcards
Learn
Write
Spell
Test
PLAY
Match
Gravity
Terms in this set (75)
Between- and Within-Study Designs
Study a single independent variable.
Factorial Design
Studies two or more independent variables within a single experiment. Specifically, it includes two or more independent variables and crosses (i.e., combines) every level of each independent variable with every level of all the other independent variables.
Example: Criminal Alibis
Leora Dahl &Heather Price:
Experiment
Cognitive psychologists who were interested in how two factors: the age of an alibi witness and the relationship between the alibi witness and crime suspect, influenced peoples acceptance of an alibi. They studied this issue experimentally by simulating a police investigation during which they exposed participants to different alibi witnesses. Whereas proper experiments had studied peoples perceptions of only adult alibi witnesses, Dahl and Price wanted to compare peoples reactions to both child and adult alibi witnesses.
Leora Dahl &Heather Price:
Hypothesis
Drawing on findings from research on child and eye witness testimony, Dahl and Price predicted that people would be more likely to believe a child alibi witness than an adult alibi witness. They also wanted to examine how an alibi witness's age and relationship to a crime suspect jointly influenced peoples willingness to believe an alibi.
Leora Dahl &Heather Price:
Experiment Procedure
Dahl and Price (2012) asked 134 Canadian undergraduates to assume the role of a police officer investigating a 50-year-old robbery suspect. Each participant received a simulated police file that contained details of the robbery and included an eyewitness account. The participant than searched a computer database of potential suspects with a history of prior arrests and decided whether to select one suspect for further investigation. Unknown to the participants, the database was constructed to create a good match between one participants profile and the information in the police file of the robbery. Only 10 participants failed to select this suspect, and for them the experiment was discontinued. The participants were then randomly assigned to watch one of several videos showing the interview of a male Caucasian alibi witness. Dahl and Price used these videos to experimental manipulate two independent variables.
Leora Dahl &Heather Price:
Independant Variables
Dahl and Price used these videos to experimental manipulate two independent variables. First, to manipulate the age of the alibi witness, some videos featured a 6-year-old alibi witness and other videos featured a 25-year-old witness. Second, to manipulate the relationship between the alibi witness and crime suspect, in some videos the alibi witness was said that he was the suspects son, and in other videos he indicated that he was the suspects neighbour. In the videos showing a child-neighbour alibi witness, the child stated that his grandmother had been sick and his mother was away that day caring for her, and this is why he had been under his neighbours (the suspects) care. Beyond these differences, the actors who played the roles of the alibi witness and interviewer in the videos all followed the same script.
Leora Dahl &Heather Price:
Factorial Design of Experiment
Combining two independent variables, which have two levels, creates an experimental design that has four conditions. This is
the most basic factorial design
possible. The phrase "manipulated the video" appears beneath each independent variable to remind you that the alibi witness is an experimentally manipulated stimulus: an aspect of the social environment, to which participants were exposed.
Each of the four conditions has an equal number of participants
.
Leora Dahl &Heather Price:
Dependant Variables
Dahl and Price measured several dependent variables. After watching the video, participants rated the degree to which they found the alibi witness and the alibi itself credible. Both before and after watching the alibi video, participants rated the probability that the suspect was guilty and indicated whether they should support this arrest.
Leora Dahl &Heather Price:
Experimental Results
The results indicated that overall, participants gave low credibility ratings to the alibi and alibi witness. Dahl and Price noted that some prior studies had obtained a similar finding, possible reflecting broad skepticism on the part of the public about alibis. Still, as the researchers expected, participants judged a child witness to be more credible than an adult witness.
Leora Dahl &Heather Price:
Result Analyses
Before watching the alibi witness video, on average, participants believed that there was a 69% probability that the suspect had committed robbery. Moreover, the average probability rating did not differ significantly across four experimental conditions. Although, rating differences between conditions emerged after the participant viewed the alibi. Overall, those who viewed a child alibi witness, decided that it was significantly less likely that the suspect had committed the crime (55% mean probability rating) than did participants who viewed the adult alibi witness (70% mean probability rating).
Leora Dahl &Heather Price:
Experiment Result Statistical Analyses
There was no statistically significant difference overall between the ratings of participants who were exposed to a son alibi witness (65% mean probability rating) and those exposed to a neighbour alibi witness (60% mean probability rating). But this does not mean that the alibi's witness's relationship to the suspect failed to influence participants judgements: It did influence them, but its effect depended on whether the witness was a child or an adult. When the alibi witness was 25-years-old, participants were significantly more likely to judge the suspect as guilty if the witness was the suspects son rather than a neighbour. This finding was consistent with the results of prior alibi experiments, which found that biological relatedness between an alibi witness and a suspect decreases peoples acceptance of an alibi. In contrast, when the alibi witness was 6-years-old, the son-neighbour distinction did not, overall, significantly influence participants' judgements of the suspect's probable guilt. Prior experiments had not exposed participants to child alibi witnesses, so this finding, if replicated, adds new information to our understanding of how alibi witnesses may affect peoples judgments.
Leora Dahl &Heather Price:
Additional Note
When participants made their final decision about whether to arrest the suspect, 41% stated they would, and this result did not differ significantly across the four conditions or from their overall arrest decisions before watching the alibi videos. At least in this study, the pattern of findings concerning judgements of probable guilt did not carry over the participants' final yes/no decision on arresting the suspect.
Between-Subjects Factorial Design
A factorial design in which each subject engages in only one condition.
Within-Subjects Factorial Design
A factorial design in which each subject engages in every condition.
Mixed-Factorial Design
A factorial design that includes at least one between-subjects variable and at least one within-subjects variable.
2 x 2 Factorial Design
The fact that two numbers are involved (i.e., first number x second number) tells us that there are just two independent variables (for example, temperature and humidity). The fact that the first number is a "2" tells us that the each independent variable has two levels. Applying this, Dahl and Price employed a 2 (alibi witness age: 6 or 25 years of age) x 2 (witness suspect relation)
between-subjects factorial design
.
Individual Cells of a Factorial Design
Usually described by name and may be labelled by a number or letter. Another approach is to designate each independent variable by a different letter, such as A and B in a 2 x 2 design. The two levels of A and B are A1 & A2 and B1 and B2. For a 2 x 3 design, the third level of the independent variable B would be defined as B3, and the diagram would contain a third column on the far right with two new cells below it labelled A1B3 and A2B3. Note that you can also choose to place a third variable A along the top and variable B along the side.
Advantages of Factorial Designs
They are better able to capture real-life causal complexity than are designs that manipulate only one independent variable. There are many theories in psychology that predict interactions among causal factors, and thus factorial designs provide scientists with a way to test the validity of such theories. Researchers also use factorial designs to test hypotheses about interactions that are not based on theory. And finally, there are times when researchers have no formal hypothesis in mind and use factorial designs simply to explore whether independent variables interact. Regardless of scientific motivation, interaction affects can readily be examined in an experiment that has two or more independent variables.
Advantages of Factorial Designs:
Main Effect
Occurs when an independent variable has an overall effect on a dependent variable.
- A main effect for room temperature would mean that overall, as room temperature changes, this influences task performance.
- A main effect for humidity would mean that overall, as humidity changes, this influences task performance.
--> How do these two variables jointly affect behaviour?
Advantages of Factorial Designs: Examining Interactions Between Independent Variables
Studying both variables in the same experiment allows us to determine whether the way one independent variable influences behaviour differs, depending on the levels of a second independent variable.
Ex. Drug Interaction: the effect of one drug differs, depending if a second drug is also being used.
Advantages of Factorial Designs: Interaction (or Interaction Effect)
In behavioural research it occurs when the way in which an independent variable influences behaviour differs, depending on the level of another independent variable. Interaction effects are also common in psychological research, and the most important advantage of factorial designs is
the ability to test whether unique combinations of two or more independent variables affect our behaviour in ways that cannot be predicted simply by knowing how each variable individually affects behaviour
.
Three Key Findings in the Temperature-Humidity Experiment Example
1. Participants made more errors overall when the room temperature was 95 degee F (35 degrees Celsius) rather than 68 degree F (20 degrees Celsius).
This represents a main effect of temperature
.
2. This effect of temperature is must stronger when humidity is high rather than low. This represents an
interaction
: The strength of the relation between temperature and performance differs considerably, depending on the humidity level.
3. We can also describe this interaction from the viewpoint of how humidity affects performance. Exposure to high versus low humidity has no significant affect on performance at 20 degrees celsius, but a sizeable effect at 35 degrees celsius. Thus,
the effect of humidity on performance differs, depending on room temperature
.
Dahl & Price Experiment Findings
A significant interaction between the age of the alibi witness and witness-suspect relationship: When the alibi was an adult son, participants guilt-probability ratings were on average higher than when the witness was an adult neighbour. In contrast, participants ratings in the child-son and child-neighbour conditions did not differ. Thus, the effect of the witness-suspect relationship on participants judgements differed, depending on whether the witness was a child or adult.
Advantages of Factorial Designs:
Examining Moderator Variables
One of the most common strategies for testing a hypothesized moderator variable, or for exploring whether a variable might be a moderator variable, is to incorporate it into a factorial design and examine whether it produces an interaction effect.
Moderator Variable
A variable that alter he strength or direction of the relation between an independent ad dependent variable. In other words, the effect of the independent variable on behaviour
depends on
the level of the moderator variable, and this is precisely what an interaction involves.
Examining Moderator Variables Example
Suppose we hypothesize that talking on a cell phone will have a stronger negative effect on driving performance when traffic density is high than when traffic density is low. To test this hypothesis, we create a 2 (cell phone use-no use) x 2 (high-low traffic density)
within-subjects factorial design
. Overall, the order of conditions will be counterbalanced so that each of the four driving conditions will occur equally often in each route segment. Using this design, we can examine whether traffic density has an overall effect (i.e., main effect) on driving performance, and whether cell phone use has an overall effect (i.e., main effect) on performance. Our primary interest, however, is determining if an interaction occurs: will the effects of using a cell phone on driving performance differ, depending on the level of traffic density? Note that in our hypothesis, we're conceptualizing traffic density as a moderator variable. Operationally, however, it is being incorporated into our 2 x 2 design as an independent variable that we manipulate.
Additional Advantages of Factorial Designs
1. They are more often efficient to conduct, as compared to conducting a series of separate experiments, each of which examines only one independent variable. Rather than setting up equipment and initiating procedures for recruiting and scheduling participants for each individual experiment, all these administrative and procedural aspects of conducting an experiment can be done just once when a factorial design is used.
2. Researchers frequently wish to examine whether situational factors have different effects on different types people.
Example: Would the effects of room temperature and humidity on task performance be the same for men and women, or older versus younger adults?
-> Factorial designs easily lend themselves to examining these types of questions.
Limitations of Factorial Designs
The key limitation of using factorial designs is the fact that as the number of independent variables increases, and the total number of levels within each independent variable increases,
the total number of conditions in an experiment can rapidly increase beyond manageable proportions
. As the total number of conditions increases,
practical issues
such as the number of participants that need to be recruited and/or the amount of time that each participant must devote to the experiment, as well as the amount of time and effort that the researchers need to expend, may make it difficult if not practically impossible to conduct the experiment. As the preceding exercise in multiplication makes clear, practical limitations often necessitate a choice between adding independent variables to factorial designs or adding more levels to a smaller number of independent variables. Other issues, also factor into the creation of a factorial design.
Limitations of Factorial Designs Example
If we want to include 5 temperature levels and 5 humidity levels in a factorial design, this would produce 25 experimental conditions. Multiplying our variables between subjects, we would now have to recruit many more participants. Manipulating our variables within subjects, each participant would have to perform the task over and over many times under different temperature-humidity combinations. Suppose we scale back to our original two levels of humidity, but want to add more independent variables: noise intensity (low, hight); noise pattern (constant, periodic); and noise type (traffic noise, people talking). Our 2 x 2 x 2 x 2 x 2 design would have 32 conditions, and if we prefer to have 3 or 4 levels of some independent variables, we could easily approach or exceed 100 conditions. As the number of independent variables and conditions increases, interpreting the results often becomes more difficult and mentally taxing, because of all the possible outcomes that can occur.
Designing a Factorial Experiment:
Examining Nonlinear Effects
When a researcher seeks to determine whether an independent variable has a nonlinear influence on behaviour, that variable must be designed to have three or more levels.
Designing a Factorial Experiment: Examining Nonlinear Effects:
Example: On a 2 x 2 Factorial Design
Suppose we hypothesize that as room temperature increases from 20 to 35 degrees celsius, this will have a nonlinear effect on peoples task performance. We want to determine whether this nonlinear effect will be under the same conditions of low and high humidity. If we only included these two temperature levels, then we would only be able to asses linear relations. It immediately apparent that the yellow and orange lines are not parallel, and this lack of parallelism suggests the presence of an interaction. As room temperature increases, errors increase both when humidity is low and high, but the temperature change has a larger effect when humidity is high. We would use statistical analyses to determine whether this finding is statistically significant and, if it is, conclude that an interaction is present. Still, even with an interaction, with only two levels of temperature for which to plot data, the graph implies that increases in temperature have a linear effect on performance at each level of humidity. Suppose we included 4 levels of temperature, we can again see the interaction: The increase in temperature has a stronger effect on task performance when humidity is high, but now the
nonlinear effects of temperature are revealed at both levels of humidity
. Regardless of humidity, the increase in room temperature has little effect on performance. Beyond that point,
errors increase at higher temperature levels more rapidly when humidity is high than when it is low
. If we also wish to examine whether humidity has a nonlinear influence on performance, we will need to create at least three humidity levels.
Designing a Factorial Experiment:
Incorporating Subject Variables
In examples of factorial designs discussed so far, all the independent variables within each experiment have been
situational factors: aspects of the social or physical environment
, that the experimenter manipulated. Now lets add a new important variable to the mix:
Subject Variables
. The scientific and practical question is: How do different types of individuals respond to the same situational factors? OR: Does the effect of a situational factor in behaviour differ, depending on the type of individual? These questions ask whether a subject characteristic interacts with a situational factor.
Subject Variables
Represent characteristics of people or nonhuman animals who are being studied.
Designing a Factorial Experiment: Incorporating Subject Variables:
Examples
Does the effect of talking on a cell phone on driving performance differ, depending on peoples level of driving experience? Do success and failure outcomes have the same psychological impact on people who have high versus low self-esteem? Does the way in which people respond to consuming various doses of alcohol depend on their expectations of how alcohol will affect them?
Designing a Factorial Experiment: Incorporating Subject Variables:
The Person x Situation Factorial Design
In designing an experiment to address such questions, the most common approach is to create a
person x situation
(also called
person x environment
)
factorial design
: An experiment design that incorporates at least one subject variable along with at least one manipulated situational variable. The word
person
is used liberally here (
organism
would be more inclusive), because experiments with nonhuman animals may also combine subject variables (such as the animals age or sex) with the manipulated situational variables. Also note that although the word "person" appears first in the term "person x situation", when researchers describe the design of their study, they may list subject variables before or after situational variables.
The Person x Situation Factorial Design: 2 x 2 Design Example
In the simplest possible case, we would have one subject variable with two levels and one manipulated independent variable with two levels. For example, in an experiment examining the effects of cell phone use on driving performance, we would have our original manipulated independent variable (cell phone use-no cell phone use) and then add a subject variable, such as driver experience (experienced-inexperienced). This would produce a 2 x 2 design with four driving conditions:
1. Cell phone use by experienced drivers;
2. Cell phone use by inexperienced drivers;
3. No cell phone use by experienced drivers;
4. No cell phone use by inexperienced drivers.
The Person x Situation Factorial Design: 2 x 2 x 2 Design Example
Of course, many subject variables can have more than two levels. In our cell phone experiment, we could measure the driving experience of a sample of people and then select participants for placement into one of three conditions: inexperienced, moderately experienced, and highly experienced. We can also examine more than one subject variable in a factorial design. For example, we could manipulate cell phone use (yes, no) and include driver experience (experienced, inexperienced) and gender (male, female drivers) as subject variables. This would yield a 2 x 2 x 2 factorial design
Designing a Factorial Experiment:
Thinking Critically About Subject Variables
In describing the design of a person x situation experiment,
each subject variable is considered an independent variable
. Thus we could say that our 2 x 2 (cell phone- driver experience) design has two independent variables. But when interpreting the results, we must keep in mind that there is an important distinction between independent variables that are being manipulated and those that are created by measuring subjects' natural characteristics. Many researchers refer to subject variables as
selected independent variables
or
quasi-independent variables
, to distinguish them from manipulated independent variables.
Designing a Factorial Experiment: Thinking Critically About Subject Variables:
Cell-Phone Example
In our experiment, we are manipulating the situational factor of cell phone use. Either through counterbalancing (in a within-subjects design) or random assignment (in a between-subjects design), we are the ones who control whether the participant will use a cell phone during any particular time segment of the simulated driving route. In contrast, we have not manipulated peoples driving experience. Rather we have created an independent variable by forming two conditions (experienced/inexperienced) based on participants preexisting level of driving experiencing. Through a questionnaire or interview item, for example, we have measured how much driving experience people have naturally accumulated and then used this information to place participants in the experienced and inexperienced conditions.
Designing a Factorial Experiment: Thinking Critically About Subject Variables: Main Effect and Interactions
When subject variables produce a main effect of interaction in a factorial experiment, we need to be especially cautious about drawing conclusions concerning their causal influence. The reason for this is that any particular subject variable is likely to be correlated with other subject characteristics. In sum, always keep the distinction manipulated and non-manipulated independent variables in mind.
Designing a Factorial Experiment: Thinking Critically About Subject Variables: Main Effect and Interactions: Example
For example, if cell phone use impairs the performance of inexperienced but not experienced drivers, is it really the level of driving experience that has caused this interaction? Experienced drivers are likely to be older, and perhaps some personal characteristic associated with age (e.g., greater cautiousness or vigilance) was the real factor that enables experienced drivers to cope with the potentially distracting effects of talking on a cell phone. Anticipating this, the experimenter could try to match the inexperienced and experienced drivers on the variable of age, or at least measure their age and see whether an age correlates with driver simulator performance. If it does, there are statistical procedures that can be used to filter out the correlation with age from the analysis of main effects and the interaction. Still, it is unlikely that the experimenter could match participants on, or even measure, every relevant personal characteristic associated with driver experience.
Designing a Factorial Experiment: Examining Changes in a Dependant Variable Over Time
Psychologists often want to know how exposing people or nonhuman animals to an independent variable changes subjects behaviour or other characteristics over time. To examine this, the researcher measures the dependent variable on at least two occasions. When researchers measure a dependant variable multiple times, one option for analyzing the change in the dependent variable over time is to treat the multiple measurement periods as levels of an additional independent variable. In such cases, the factor is often labelled as time, trial, or testing session. In the simples case, the dependant variable is measured only twice: once before and once after exposure to the independent variable. When you read about an experiment or other study that examines how participants change over time, time of measurement may be labelled as an independent variable in the research design or statistically analyses.
Understanding Main Effects and Interactions: Possible Outcomes in a 2 x 2 Factorial Design
In a factorial design that has two independent variables, there are eight possible combinations of outcomes that can occur. For variable A, there either will or will not be a main effect. Likewise, for variable B, there either will or will not be a main effect. Finally, there either will or will not be an A x B interaction. Multiplying these possibilities yields eight potential combinations. For each example, we will assume that there are 45 participants in each of the four conditions.
Cell Mean
In the tables, the statistic in each cell (called a
cell mean
) represents participants mean rating of the probability (expressed as a percent) that the suspect committed the crime.
Marginal Mean
In the margins, the statistic to the right of each row and below each column (called the
marginal mean
) is the average of the cell means for that participant row or column
Possible Outcomes in a 2 x 2 Factorial Design:
Example #1: a.) Is there a main effect of number of witnesses?
For this example only, the four cells (i.e., conditions) are numbered. If we compare the average rating of the 90 participants exposed to the three witnesses to the average rating of the 90 participants exposed to one witness, did an overall difference occur? To answer this, we examine the
marginal means
at the bottom of each column. For the moment, we combine to results form Condition 1 and 3: together, these are the "one witness" results. The
marginal mean
beneath the left column indicates that, on average, participants exposed to one alibi witness believed there was a 60% chance that the suspect committed the crime. Similarly, when combining the results from Conditions 2 and 4 (the "three witnesses" results), the marginal mean beneath the right column is the same 60% as that beneath the left column. Therefore, ignoring for the moment that there is another independent variable (witness-suspect relationship) and just focusing on the one versus three witness conditions, no overall difference in their ratings occurs.
There is no main effect of number of witnesses
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #1: b.) Is there a main effect of witness-suspect relationship?
To answer this question, we compare the average rating of the 90 participants exposed to daughter witnesses to the average rating of the 90 participants exposed to neighbour witnesses. Thus, we compare the marginal means at the ends of the rows. When the alibi witness are daughters (Conditions 1 and 2), the average probability rating is 70%. But, when the alibi witness are neighbours (Conditions 3 and 4), the average rating is 50%. Thus,
the witness-suspect relationship produces a significant main effect: Overall, participants' ratings were influenced by whether the witness was a daughter or neighbour
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #1: c.) Is there a Number of Witness x Witness-Suspect Relationship Interaction?
Does the effect of the witness-suspect relationship (daughter vs. neighbour) on participants judgements differ, depending on the number of witnesses? To answer this question, we ignore the marginal means and examine
the patterns of means across the four individual cells
(each of which has 45 participants). Alibis provided by daughters rather than neighbours led to a 20% higher average guilt rating (70%-50%) when there was one witness, and a 20% higher average guilt rating (70%-50%) when there were three witnesses. Thus, the influence of having daughter versus neighbour witnesses was the same, regardless of the number of witnesses.
No interaction exists between the two independent variables
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #1: Graphical Analyses: Line Graph
Finding were then displayed on a line graph and a bar graph. In the
line graph
, if you determine the vertical midpoint between the yellow line and the orange line when there is one witness, and do the same when there are three witnesses, those midpoints are the same. Thus,
there is no overall main effect of number of witnesses
. In contrast, the midpoint of the yellow line (daughters) clearly is above the midpoint of the orange line (neighbours). This represents the main effect of the witness-suspect relationship. The fact that the two lines are parallel says there is no interaction:
The flat slope of both lines indicates that the number of witnesses has no effect on the mean ratings, regardless of the witness-suspect relationship
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #1: Graphical Analyses: Bar Graph
For the bar graph, the average height of the two bars in the one-witness condition is the same as the average height of the two bars in the three-witness condition. This indicates
the absence of a main effect for number of witnesses
. In contrast, the average height of the two yellow bars differs from the average height of the two orange bars, indicating
a main effect for the daughter-neighbour manipulation
. Finally, the difference in height between the yellow and orange bars in the one witness condition is the same as the height difference in the three witness condition,
indicating the lack of an interaction
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #2: Comparing the Marginal Means BENEATH the Cell Columns
In this data set, we see two main effects and no interaction . First we compare the marginal means beneath the two columns to determine if number of witnesses produced a main effect. On average, participants gave higher guilt probability ratings when there was one witness (65%) than when there was three witnesses (45%). This difference indicates that indeed
there is a main effect for number of witnesses
. Note that if their difference had occurred in the other direction (higher average rating for the three witnesses condition) this would also reflect a main effect. The key is that there is an overall difference, regardless of direction, between the one versus three conditions.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #2: Comparing the Marginal Means at the END OF CELL ROWS
The marginal means at the end of the rows indicate a witness-suspect relationship also produced a main effect. The average guilt probability rating was higher among the participants exposed to a daughter witness (70%) rather than a neighbour witness (40%). Overall, participants were more skeptical about alibis provided by daughters rather than neighbours. In addition, there is
NO Number of Witnesses x Witness-Suspect Relationship interaction
. The difference in ratings between the daughter and neighbour conditions was the same regardless of wether was one witness (80%-50%) or three witnesses (60%-30%).
Possible Outcomes in a 2 x 2 Factorial Design:
Example #2: Line Graph Analysis
In the line graph, the midpoint between the yellow and orange lines in the one-witness condition is higher than the midpoints between the lines in the three-witness condition.
This is the main effect of number of witnesses
. The main effect of the witness-suspect relationship is the fact, as seen in the graph, that the orange line is always beneath the yellow line (and thus the midpoints of these lines differ). Finally, these lines are parallel, indicating that
no interaction occurs
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #2: Bar Graph Analysis
In the bar graph, the average height of the two bars in the one-witness condition differs from the height of the two bars in the three-witness condition. This represents the
main effect of number of witnesses
. Similarly the average height of the two yellow bars and two orange bars differs, illustrating the main effect of the witness-suspect relationship. Finally, the difference in height between the yellow and orange bars in the one-witness condition is the same as the difference in the three-witness condition, indicating that
no interaction occurs
.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #3: Marginal Means Analysis
The marginal means beneath the columns differ, indicating that there is a main effect for number of witnesses. Overall, participants believed it was more likely that the suspect was guilty when there was only one alibi witness rather than three. Both row marginal means are 50% indicating that there is no main effect of witness-suspect relationship. Examination of the four cell means reveals an interaction: The number of witnesses had no influence on the believability of daughters, but participants were more likely to think the suspect was guilty when there was one neighbour witness rather than three neighbour witnesses.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #3: Main Effect
In determining whether a main effect exists, we only examine marginal means
. In this case, the column marginal means tells us that, statistically, there is main effect of number of witnesses. However, its clear that that this main effect has resulted entirely from the results that occurred in the one-neighbour versus three-neighbours conditions. The key issue is that the interaction overrides or "qualifies" the main effect of number of witnesses. In our experiment, more witnesses = lower ratings of suspect guilt, ONLY WHEN the witnesses are neighbours, not when they are the suspects daughters. This is the accurate description of the findings. The essential point to remember is that
whenever main effects occur along with an interaction, the interaction requires us to carefully inspect the findings in each of the four conditions, along with the main effects
. Depending on the nature of the interaction, a conclusion based on main effect may be an inaccurate way to portray the findings.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #3: Graphical Analysis
The
line graph
reveals that the two lines are not parallel, which indicates that an interaction does occur. Keep in mind that with real data,
nonparallel lines suggest the possibility of an interaction, and we would perform statistical tests to determine whether the interaction is statistically significant
. This type of interaction is called a
disordinal (or crossover) interaction
. At one end of the lines, the mean score for the daughter condition is above the mean for the neighbour condition, whereas as the other end, the opposite "above/below" order occurs. Both graphs also make it clear how the interaction qualifies the main effect of number of witnesses. In the line graph, the slope of the yellow line is flat, indicating that when the witnesses were daughters, the number of witnesses had no effect on participants ratings. The slope of the orange line illustrates that the number of witnesses did affect participants ratings when the witnesses were neighbours. Analogously, in the
bar graph
, the height of the bars is the same in the two daughters conditions, but differs considerably in the two neighbours conditions. This indicates that the number of witnesses affected the ratings only when the witnesses were neighbours.
Possible Outcomes in a 2 x 2 Factorial Design:
Example #4: Marginal Means Analysis
The marginal means indicate that we have
two main effects
. Overall participants were less likely to judge the suspect guilty when there were three witnesses rather than one, and when the witnesses were neighbours rather than daughters. Looking at the four individual cells
reveals an interaction
. When the witnesses were daughters, having more witnesses decreased the probability ratings from 70% (one witness) to 50% (three witnesses). When the ratings were neighbours, having more witnesses decreased the probability ratings from 60% (one witness) to 10% (three witnesses). Thus, the effect of number of witnesses on participants ratings was in the same direction regardless of whether the witnesses were neighbours or daughters, but the effect was much stronger when the witnesses were neighbours.
Possible Outcomes in a 2 x 2 Factorial Design: *Example #4: Graphical Analysis
The line graph and the bar graph visually portray that in this data set, we have an
ordinal interaction
: The ratings in the daughter conditions remain significantly higher than those in the neighbour conditions, regardless of the number of witnesses, but the size of the difference is much greater when there are three witnesses.
Possible Outcomes in a 2 x 2 Factorial Design:
Key Points
- One common type of interaction occurs when an independent variable influences behaviour in the same direction at all levels of another independent variable, but the strength of the affect differs, depending on the level of that other independent variable.
- It is most easily seen in the line graph of example 4 that the interaction in this example does not negate the truthfulness of the conclusion based on the main effects. It is accurate to say that, overall, the presence of the three alibi witnesses led to lower ratings of the suspects guilt. Likewise, it is accurate to say that overall, participants were less likely to judge the suspect to be guilty when the alibi was supported by neighbours rather than daughters. However, the presence of an interaction requires us to add this: "but the degree ti which neighbours were more believable than daughters as alibi witnesses was much greater when there were three witnesses, rather than just one".
Interactions and External Validity
Suppose that we conduct our 2 x 2 experiment with the results portrayed as in Example 2:
two main effects and no interaction
. These findings indicate that, at least in our experiment, increasing the number of witnesses decreased participants judgements about the suspects guilt to the same degree for both types of alibi witnesses. Compared to a single-factor experiment that only manipulates the number of witnesses, we have provided a stronger basis for the external validity of our finding. In sum, to repeat an earlier point,
interactions always must be examined carefully for the way in which they might qualify any general conclusions that otherwise would be implied by statistically significant main effects
.
Analyzing the Results: General Concepts
There are different ways to statistically analyze the data from factorial experiments. In one common approach, the first step is to determine whether any main effects or interactions are statistically significant. Depending on the results, the researcher may then conduct follow-up tests to examine the findings more closely.
Analyzing the Results: ANOVA
When dependent variables have been measured on an
interval or ratio scale
, analysis of variance (
ANOVA
) is a widely used statistical procedure for determining whether the main effects and interactions in factorial designs are statistically significant. The basic logic of the ANOVA procedure in an experiment with one independent variable the ANOVA yields a straight test of statistical significance. With a factorial design with two independent variables, the ANOVA involves three tests of statistical significance. There is one test for the main effect of independent variable A, one for the main effect of independent variable B, and one for the A x B interaction.
Analyzing the Results: Example: 3 x 2 Factorial Design
Lets consider one example, for which the findings from the 3 x 2 factorial design will best serve our purpose. Suppose than an ANVOA reveals no significant main effect of room temperature, a significant main effect of gender, and a significant Temperature x Gender interaction. The graph shows the interaction: The number of task errors made by men increases as room temperature increases, and surprisingly, women's errors seem to show a weak opposite trend. We can also see the main effect of gender: Overall, men made more errors than women. However, the interaction qualifies the gender main effect: At the lowest temperature level, women averaged slightly more errors than men. Our goal now is to dig deeper into the nature of the interaction by performing follow-up analyses. One approach we can take is to break down the interaction into what are called
simple main effects
.
Simple Main Effects
A simple main effect represents the effect of one independent variable at a particular level of another independent variable.
- So, for the variable of gender, there would be three simple main effects we could examine, one at each of the three levels of room temperature. If we are interested in determining whether statistically significant gender differences occur at each temperature level, then we would perform a separate statistical test of each of these simple main effects. We may find that all, or none, or just one or two of these simple main effects are statistically significant. For example, we might determine that the difference in men's versus women's performance is statistically significant only at the highest temperature level.
Simple Contrasts
The comparisons of specific pairs of means. They represent
post-hoc comparisons (also called post-hoc tests)
, because we decided to conduct them after the fact: after we saw the pattern of our findings and obtained the ANOVA results.
Planned Comparisons
In contrast to the multistep analysis approach, researchers sometimes plan, prior to gathering their data, to bypass an ANOVA and instead perform a small number of tests that will determine whether the differences between certain pairs or combinations of means will be statistically significant. For example, a study may test a theory or hypothesis that predicts the mean scores in two particular conditions will differ significantly. Such tests are called
planned comparisons
.
Planned Versus Post-Hoc Comparisons
The reward for using planned rather than post-hoc comparisons is that, statistically (other things being equal), differences between means do not have to be as large in order to be found statistically significant. The drawback is that planned comparisons should be limited in number, and therefore, in a factorial design that involves many means, unexpected or other potentially important findings that are not within the scope of the planned comparisons go untested.
Experiments with Three Independent Variables
By examining more than two independent variables, factorial experiments can better capture the complexity of real life. But as noted earlier, as the number of independent variables increases, the number of conditions may rapidly increase beyond what is practical for researchers to study.
Experiments with Three Independent Variables: Example
Suppose we want to examine whether gender differences exist in how cognitive performance is affected by room temperature and humidity. To simplify the discussion, we'll label each of the eight conditions with a number and assume that we have a completely
between-subjects design
, with 20 participants in each condition.
Experiments with Three Independent Variables: Example:
Main Effect of Temperature
Just as with the 2 x 2 design, in a 2 x 2 x 2 design the main effect of temperature is determined by comparing the average performance of all participants exposed to the other temperature. To make this concrete, we are simply asking whether, overall, the 80 participants who performed the task at 20 degrees Celsius (Conditions 1, 3, 5, 7) did better or worse on average than the 80 participants who performed the task at 35 degrees Celsius (Conditions 2, 4, 6, 8).
Experiments with Three Independent Variables: Example:
Main Effect of Humidity and Gender
Similarly, to examine the main effect of humidity, we would determine whether, on average, the 80 participants exposed to high humidity (Conditions 3, 4, 7, 8) performed better or worse than those exposed to low humidity (Conditions 1, 2, 5, 6). And, to examine the main effect of gender, we would compare the average performance of the 80 women (Conditions 1, 2, 3, 4) to the average performance of the 80 men (Conditions 5, 6, 7, 8).
Experiments with Three Independent Variables: Example:
Interaction Effects
Turning to interaction effects, we've seen that in a 2 (variable A) x 2 (variable B) factorial design, there is one possible interaction: A x B. In a factorial design with three independent variables, A, B and C, the number of possible interactions jumps to four. Consider first that A might interact with B, A might also interact with C, and C and B might interact. These are called
Two-Way Interactions
.
Two-Way Interactions
Among two independent variables, the way that one independent variable influences a dependent variable depends on the level of the second independent variable. This is the type of interaction that we have discussed throughout the chapter.
Three-Way Interaction
In addition to the three possible two-way interactions, the fourth possible interaction is A x B x C. This is called a
three-way interaction
: The interaction of two independent variables depends on the level of the third variable. You can see that when a third variable is added to our factorial design, the findings may become much more complex. But, potentially, we stand to learn a great deal more about the variables of interest.
;