Upgrade to remove ads
Only $35.99/year

Research Methods Chapter #13

Terms in this set (14)

External Validity
The extent to which the results of a research design can be generalized beyond the specific way the original experiment was conducted.
Generalization
The extent to which relationships among conceptual variables can be demonstrated in a wide variety of people and a wide variety of manipulated measures or measured variables.

The major issue underlying external validity.

The goal is to get to get external validity is generalization. Across participants (will every person show this same response?) and across settings (will every situation produce this same response?).
Generalization Across Participants
-Any sample of research participants, no matter who they are, will be limited in some sense.

-We want to make assumptions on the general populations, not the specific sample

-Because of the impossibility of the scientist drawing a representative sample of all human beings, true generalization across people is not possible. For this reason, we frequently make the simplifying assumption that unless there is a specific reason to believe otherwise, relationships between conceptual variables that are observed in one group of people will also generally be observed in another group of people (therefore scientists are often content of having college students as their samples, but obviously they may actually have big differences)

-Unless a researcher has a specific reason to believe that generalization will not hold, it is appropriate to assume that a results found in one populations will generalize the other populations. The burden of proof rests on those who claim that a result will not generalize to demonstrate that this is indeed the case.
Generalization Across Setting
-The uniqueness of any one experiment makes it possible that the findins are limited in some way to the specific settings, experimenters, manipulations, or measured variables used in the research.
- For example, it is often that different experimenters (warm versus cold) get different results from their participants)
Ecological Validity, Field Experiment
Ecological Validity
It is possible to increase the potential generalization of a single experiment by increasing its ecological validity.

The extend to which the research is conducted in situation that are similar to the everyday life experiences of the participants (ex. a research design that deals with how children learn to read will have higher ecological validity if the children read a paragraph taken from one of their textbooks than it would if they read a list of sentences taken from an adult magazine)

Can be done using field experiments.
Field Experiments
A way to get ecological validity.

Are experimental research designs that are conducted in a natural environment such as a library, a factory, or a school, rather than a research laboratory.

Because are conducted in the natural environment of the participants, they will generally have a high ecological validity than lab experiments.

PRO: - good parallel environments...
- participants acts more naturally

CON: - permission problematic
- difficult to use random assignment (ex. children often cannot be randomly assigned to different teaching methods)
- greater potential for error of both types (bc don't know what unexpected events may occur that would have been controlled for in a lab)

In general, we have more confidence that a finding will generalize if it tested in an experiment that has high ecological validity, however, field experiments are not necessarily more externally valid than are laboratory experiments (ex. experiment held in one factory may not have the same results as in another factory). Limited because only provide a sample of people at one place, at one particular time. There will always be threats to a study's external validity.
Replications
The process of repeating previous research, which form the basis of all scientific inquiry. Allows scientists to test both the generalization and limitations of research findings.

4 Basic types.
-Exact replications
-Conceptual replications
-Constructive replications
-Participant replications
Exact Replications
Goal: to repeat a previous research design as exactly as possible, and see if the effect happens again.

Usually to see if the same effect can be found with different lab, different experimenters, etc.

Although exact replications may be used to test whether a finding can be discovered again, they are actually not that common in behavioral science because even if the exact replication does not reproduce the findings, that does not mean that the original findings are invalid as it is always possible that the experimenter did not replicate it properly
-need to have very detailed methods section

Other types of replication are usually more useful, because they also provide additional information.
Conceptual Replications
The scientist investigates the relationship between the same conceptual variables that were studied in previous research, but she/her tests the hypothesis using different operational definitions of the independent variable and/or the measured dependent variable.

If the same relationship can be demonstrated again, the confidence that the observed relationship is not specific to the original measures is increased. If not, it may nevertheless provide information about the situation in and measures for which the effect does or does not occur.

Although they are useful, it is difficult to draw conclusions about exactly what changes between the original experiment and the replication experiment might have produced differences in the observed relationships.
Constructive Replications
The most popular form of replication.

The researcher tests the same hypothesis as the original experiment (in the form of either exact ot a conceptual replication), but also adds new conditions to the original experiment to assess the specific variables that might change the previously observed relationship. Add another iv, control condition, moderator variable etc.

Goal: to rule out alternative explanations or to add new information about the variables of interest.

Ex. Lateness and helping behavior are positively associated...but more so in cities?

Height and salary are positively associated...but more so for men? Developmental status?
Participant Replication
Unlike other replications that deal with generalization across settings, participant replication is used in cases where there is reason to believe that an observed relationship found with on set of participants will not generalize to or will be different in another population of people.

Should not be done by simply repeating the original experiment with a new population, because if a different relationship between the independent and dependent variable is found, the experimenter cannot know if that difference is due ot the use of different participants or other potentially unknown changes in the experimental setting.

The experiment should be designed as a constructive replication in which both the original population and the new one are used.
Research Programs
Because scientists are aware of the limitation any one experiment, they frequently conduct collection of experiments, known as research programs, in which they systematically study a topic of interest through conceptual and constructive replications over a period of time.

PRO
- scientists are able to increase their confidence in the validity and the strength of a relationship as well as the conditions under which it occurs or does not occur
Review Papers
A document the discusses the research in a given area with the goals of summarizing the existing findings, drawing conclusion about the condition under which relationships may or may not occur, linking the research findings to other area of research, and making suggestion for further research. (summarize the results of research programs)

CON
-potential for bias due to only a subset of relevant research being included
Meta-Analysis
A statistical technique that uses the results of existing studies to integrate and draw conclusions about those studies.

Provides relatively objective method of reviewing research findings because it
(1) specifies inclusion criteria the indicate exactly which studies will or will not be included in the analysis
(2) systematically searches for all studies that meet the inclusion criteria
(3) uses the effect size statistic to provide an objective measure of the strength of observed relationships

How to
(1) define inclusion criteria
(2) code the studies
(3) analyze the effect size

PROS
-the use of explicit inclusion criteria and an in-depth search for ALL studies the meet these ensures objectivity in what is and what is not included in the analysis
-the use of the effect size statistic provides an objective measure of the strength of observed relationships
-therefore, more accurate than narrative research reviews, which tend to underestimate the magnitude of the true relationships between variables in comparison to meta analysis -- meta analysis says which direction is stronger even if there is some contradiction between studies

CONS
-conclusions are only drawn on data that has been published (studies that are non-significant are less likely to be published) and therefore the published articles are more likely to overestimate the size of a relationship between the variables
Recommended textbook explanations

Myers' Psychology for AP

2nd EditionDavid G Myers
900 explanations

Myers' Psychology for the AP Course

3rd EditionDavid G Myers
955 explanations

Psychology: Principles in Practice

1st EditionSpencer A. Rathus
1,024 explanations

Psychology

1st EditionHOUGHTON MIFFLIN HARCOURT
1,007 explanations