Upgrade to remove ads
Chapter 10 - Experimentation and Validity
Terms in this set (122)
Categories of Inference and Validity Type
Construct Validity - Inferences about constructs.
Statistical Inferences - Statistical.
Causal Inferences - Internal.
Inferences about generalizability - External.
The degree to which the items on a measure adequately represent the entire range or set of items that could have been appropriately included.
Questions to ask:
-Are you measuring what you are intending to measure?
-Are your operational definitions true to the underlying construct? ex: What is the "correct" operational definition of aggression?
-How do you now whether your measures are valid? (Measure should predict theoretically relevant behaviour (convergent) and be unassociated with unrelated measures (discriminant).
Construct validity concerns the issue of whether the constructs (conceptual variables) that researchers claim to be studying are, in fact, the constructs that they are truly are manipulating and measuring. It is affected by how faithfully the operational definitions of the independent and dependent variables represent the constructs that the researchers intend to study.
It concerns the validity of the leap from the operational to the conceptual level.
The principle that scores on a measure should not correlate too strongly with scores on measures of other constructs.
Statistical Conclusion Validity
The degree to which the statistical treatment of data is proper and the researchers' statistical conclusions are sound.
Questions to ask:
-Does data meet assumptions surrounding particular tests?
-Is data normally distributed or skewed?
-Is the proper scale of measurement used?
-Were correct statistical tests/procedures employed?
Concerns the proper statistical treatment of data and the soundness of the researchers' statistical conclusions.
The key question is this: When researchers conclude that there is or is not a statistically significant relation (no chance) between the IVs and DVs, is their conclusion based on the appropriate statistical analyses?
The generalizability of the findings beyond the circumstances of the present study. Refers to inferences about the generalizability of findings beyond the circumstances of the present study.
Questions to ask:
-Can findings be generalized beyond the study to other populations, settings, species, etc?
Concerns the generalizability of findings beyond the present study - across populations, settings and species.
External Validity - Replication
Studies are replicated to determine whether original findings are upheld.
Different types of replication:
-Direct: Same measures, procedures, data analyses as original.
-Conceptual: Same original concept, but operationalized differently.
-Replication with extension: Adds some new element to original study.
Internal Validity Threat - History
Events that occur during a study that are unrelated to experimental manipulation. Such as changes in health, relationship status, financial status, political or environmental changes, etc.
Addresses the generalizability to "real life" settings.
The degree to which responses obtained in a research context generalize to behaviour in natural settings; also refers to how well a research setting (tasks, procedures) corresponds to what people encounter in daily life.
Question to ask:
-Will the findings hold up in the workplace, family gatherings, athletic events, political rallies, classrooms, friendships, etc.
Concerns the degree to which responses obtained in a research context generalize to behaviour in natural settings. Some view it as a subset of external validity which asks whether research conclusions are generalizable to other settings, and this includes both other laboratory and real-life settings.
It is also discussed in reference to how well the research setting and methods - such as tasks, stimuli, and procedures used in a laboratory experiment - correspond to what people encounter in daily life.
Why do some experimenters believe it is important to use tasks and procedures that are as ecologically valid as possible?
Because it will increase the likelihood that research findings will generalize to settings in the real world. Others say it won't guarantee that research findings will generalize to real-life situations .
Internal Validity Threat - Maturation
People naturally change over time irrespective of what happens to them during a study.
Ex: Children acquiring object permanence, or an Alzheimer's patient loses vocabulary.
Internal Validity Threat - Testing
Measuring participants responses affects how they respond on subsequent measures.
Similar to maturation but change is caused by the testing itself.
Internal Validity Threat - Instrumentation
Changes occur in measuring instruments during data collection.
Ex: Testing apparatus is recalibrated over the course of the study.
Internal Validity Threat - Regression to the Mean
Statistical phenomenon wherein participants who receive extreme scores tend to have less extreme scores when retested even in the absence of any treatment effects.
Attributable to measurement error.
Internal Validity Threat - Attrition
Participants drop out of a study.
Particularly threatening when participants who dropped out differ meaningfully from those who stayed (i.e., differential attrition).
Internal Validity Threat - Selection
At the start of the study, groups are non-equivalent, and this difference affects the results.
Group 1 Mean IQ = 100
Group 2 Mean IQ = 120
What is a solution to the internal validity threat of "history"?
Internal validity threat "history" is when events that occur during a study that are unrelated to experimental manipulation. Such as changes in health, relationship status, financial status, political or environmental changes, etc.
Block randomization is an experimental procedure in which researchers conduct a round of all the conditions, then another round, then another, for as many rounds as needed to complete the experiment. Within each round, the order of conditions are randomly determined.
Block randomization assists as a solution because by assigning participants randomly to each of the various conditions the potential influence of such history effects could be distributed equally across the conditions.
An experimental procedure in which researchers conduct a round of all the conditions, then another round, then another, for as many rounds as needed to complete the experiment. Within each round, the order of conditions are randomly determined.
What is the solution to the internal validity threat of "maturation"?
Maturation is how people naturally change over time irrespective of what happens to them during a study.
Ex: Children acquiring object permanence, or an Alzheimer's patient loses vocabulary.
Random assignment is a procedure in which each participant has an equal probability of being assigned to any one of the conditions in an experiment.
Using random assignment to address maturation works by assuming that any maturation effects would be equivalent across the various conditions. This helps to ensure that maturation isn't a plausible confounding variable.
What is a solution to the internal validity threat of "testing"?
Avoid pretesting or ensure that all participants complete a pretest.
Testing is a threat when the measurement of participants responses affects how they respond on subsequent measures. Similar to maturation but change is caused by the testing itself.
To avoid pretesting or ensuring ALL complete a pretest works because it eliminates the possibility of a testing confound. If an experiment involves a pretest, because all the participants take it, the effects should be equivalent in all the conditions and therefore won't be a confounding variable.
What is a solution to the internal validity threat of "instrumentation"?
Random assignment and counterbalancing.
Instrumentation is a threat when changes occur in measuring instruments during data collection. Ex: Testing apparatus is recalibrated over the course of the study.
As long as the random assignment (combined with block randomization and proper counterbalancing procedures are sued, then any instrumentation effects that might occur over the course of an experiment should, overall, affect participants in all conditions to an equivalent degree. Instrumentation is unlikely to be a confounding variable.
What is a solution to the internal validity threat of "regression to the mean"?
Random assignment and exclude participants with extreme scores.
Regression to the mean is the statistical phenomenon wherein participants who receive extreme scores tend to have less extreme scores when retested even in the absence of any treatment effects. It is attributable to measurement error.
Random assignment and the exclusion of participants with extreme scores works because it will eliminate regression as a plausible confounding variable. Even if you do not exclude participants with extreme scores, random assignment ensures that the mean will be equivalent across conditions as long as participants are randomly assigned.
What is a solution to the internal validity threat of "attrition"?
To establish why participants drop out and whether participants who remain are different from the ones that left.
Attrition is when participants drop out of a study. It is particularly threatening when participants who dropped out differ meaningfully from those who stayed (i.e., differential attrition).
The reason is to determine whether continuing vs. discontinuing participants differ in ways that could plausibly account for findings.
What is a solution to the internal validity threat of "selection"?
This works because experiments involve multiple conditions, and when between-subjects designs are used, the key to preventing a selection confound is to create equivalent groups at the start. This is achieved through random assignment.
Participation in an experiment involves an implicit social contract.
They shape participants beliefs about the hypothesis and how they are expected to behave. Involve cues about the setting, instructions, apparatus, or experimenter characteristics.
Are the participants "good" or "defiant" subjects?
Refers to the cues that influence participants' beliefs about the hypothesis being tests and the behaviours expected of them. Can be an experimenters behaviour, the lab's layout, and the nature of tasks.
What are some solutions to demand characteristics?
Suspicion probes can be incorporated into debriefing to explore participants' beliefs about the hypothesis. Ex: What was the purpose of this study?
They increase psychological realism, pilot the experiment and use unobtrusive dependent measures.
It avoids within-subjects designs. Separates out participants who claim to know the hypothesis. It manipulates participants knowledge of the hypothesis.
You can apply the "red herring" technique.
Participants expectations surround treatment effects influence their responses to treatment.
To combat, include a placebo control group wherein participants don't receive the experimental treatment BUT are led to believe they did.
Typically discussed in the context of drug trials, but with a much broader application.
Experimenter Expectancy Effects
Researchers may unintentionally influence their participants to respond in line with the hypothesis (self-fulfilling prophecy).
Unintentional ways in which researchers influence their participants to respond in a manner consistent with the researchers' hypothesis.
Dealing with Experimenter Expectancy
Combat this by training experimenters to follow a research protocol. i.e., follow a script.
Automate as much as possible. Keep experimenters unaware of the hypothesis, and/or experimental condition being run through "masking" (blinding).
Double Blind Procedures
Using masking along with placebo control.
Neither participants nor the experimenters know who is receiving what.
Yoked Control Groups
Control group members linked (yoked) to experimental group members.
Experimental participants behaviour dictates how control participant is treated.
Yoking occurs through random assignment or matching.
Ceiling and Floor Effects
In order to make a causal inference, there must be variability in the DV (i.e., no range restriction).
Ceiling effects occur when scores bunch up at the maximum DV level. E.g., All employees receive "outstanding" performance ratings.
Floor effects occur when scores bunch up at the minimum DV level. E.g., All employees receive "unsatisfactory" performance ratings.
Research Design Tips
Combat ceiling and floor effects by using highly sensitive measures and strong manipulations.
Pilot studies can identify problems here and elsewhere.
Incorporate manipulation checks to assess validity of IV manipulation.
Debrief participants throughly to gain insight into their experience.
Compute > Recode > Analyze > Correlate > T-Test > One-way ANOVA > Two-way factorial ANOVA > Graphics output
Or called blinding, is a procedure in which the parties involved in an experiment are kept unaware of the hypothesis being tested and/or the condition to which each participant has been assigned.
The effect that occurs when scores on a dependent variable bunch up at the maximum score level.
Occurs when scores on a dependent variable bunch up at the maximum score level (all participants attain a maximum score for lifting a weight).
An effect that occurs when scores on a dependent variable bunch up at the minimum score level.
Occurs when scores on a DV bunch up at the maximum score level (all scores for lifting a 500 lb weight are zero).
An approach that seeks to improve the validity of an experiment by determining whether the procedures used to manipulate an independent variable successfully captured the intended construct.
What are some inferences regarding validity that scientists make?
-Inferences about constructs.
-Inferences about generalizablity.
The statement "the difference in blood pressures measurements between the two conditions is beyond what we would expect simply by chance" is a(n) ____________________ inference.
Consists in the use of statistics to draw conclusions about some unknown aspect of a population based on a random sample from that population.
Is the procedure through which inferences about a population are made based on certain characteristics calculated from a sample of data drawn from that population. Wishing to make statements not merely about the particular subjects observed in a study but also, more importantly, about the larger population of subjects from which the study participants were drawn.
Formal statistical inference uses calculations based on probability theory to substantiate those conclusions. Statistical inference can be divided into two areas: estimation and hypothesis testing.
It mainly concerned with providing some conclusions about the parameters which describe the distribution of a variable of interest in a certain population on the basis of a random sample.
Often based on a test of significance, "a procedure by which one determines the degree to which collected data are consistent with a specific hypothesis.
Based on blood pressure measurements, the statement that participants had higher anxiety after viewing the crime video rather than the comedy video is an inference about ____________________.
Referring to construct validity; refers to the degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based.
The statement "watching the videos influenced participants blood pressure measurements" is a(n) ____________________ inference.
One reasons to the conclusion that something is, or is likely to be, the cause of something else.
The statement "in this experiment, participants watched a high-threat (crime video) or low-threat (comedy video) stimulus" is an inference about ____________________.
What are the 4 types of validity?
Construct, statistical conclusion, internal, and external.
Allow researchers to draw conclusions about a population based on data from a sample. Often involves using statistical tests to help determine whether the findings are statistically significant (unlikely due to chance).
Represents the degree to which we can confidently infer that the study demonstrated that one variable had a causal effect on another variable.
Concerns the degree to which we can be confident that a study demonstrated that one variable had a causal effect on another variable.
It is how you find out in an experiment if you can conclude that it truly was exposure to the different conditions of the IV - rather than some other factor - that caused the differences in the DV.
Inferences about causality have internal validity when the research design and experimental procedures are sound and thus enable out to rule out plausible alternative explanation for the findings.
Poor internal validity results from the presence of confounding variables that provide a reasonable alternative explanation for why participants' responses different, overall, across the various conditions of the experiment.
Simply whether the test appears (at face value) to measure what it claims to.
It does not mean that a test really measures what the researcher intends to measure, but only in the judgment of raters that it appears to do so.
This is the degree to which a test corresponds to an external criterion that is known concurrently (i.e. occurring at the same time). If the new test is validated by a comparison with a currently existing criterion, we have concurrent validity. Very often, a new IQ or personality test might be compared with an older but similar test known to have good validity already.
This is the degree to which a test accurately predicts a criterion that will occur in the future. For example, a prediction may be made on the basis of a new intelligence test, that high scorers at age 12 will be more likely to obtain university degrees several years later. If the prediction is born out then the test has predictive validity.
What are the types of validity?
Content related: face and construct.
Criterion related: concurrent and predictive.
What is the difference between internal and external validity?
Internal validity refers to whether the effects observed in a study are due to the manipulation of the independent variable and not some other factor. In-other-words there is a causal relationship between the independent and dependent variable. It can be improved by controlling extraneous variables, using standardized instructions, counter balancing, and eliminating demand characteristics and investigator effects.
External validity refers to the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity) and over time (historical validity). It can be improved by setting experiments in a more natural setting and using random sampling to select participants.
A subfield that specializes in issues concerning research design, the measurement of variables, statistical analysis, and mathematical models of behaviour.
What do inferential statistical tests typically require for determining statistical significance?
That certain assumptions be met in order for a particular test to be used in a valid manner. The proper use of some statistical tests assumes that there is a certain minimum number of observations in each cell of a research design.
Another assumption involves the scale of measurement. Some tests are designed for use with variables that have been measured on an interval or ratio scale, while others can be used for variables measured on an ordinal scale or nominal scale.
A third assumption is for some statistical tests involves the patter - the distribution - of scores in a data set. Ex: The Bell Curve. Some inferential stats tests such as analysis of variance assume the data being analyzed taketh form of (least appropriate) a normal distribution.
The Bell Curve
The mathematical concept of normal distribution, which portrays hypothetical distribution for a population of scores. Mean, mode and media and identical and 50% of cases fall on either side of the mean. As scale values move further away from the mean, the frequency of cases becomes progressively smaller. This produces a symmetric curve that peaks in the middle.
Analysis of variance, helps researchers determine whether the overall pattern of differences among the mean scores of the conditions is statistically significant.
Helps researchers determine whether the difference between the mean scores of two conditions is statistically significant.
What does "robust" mean within statistical tests?
That a statistical test can yield accurate results (or that the amount of error will only be slight) even if a data set violates the test's statistical assumptions.
What does statistical conclusion validity allow researcher to claim?
That there is an association between the IVs and DVs, and that it's unlikely this relation is due merely to chance (random variations in behaviour).
What were some of the extensive environmental controls Tyron used to increase internal validity in his rat-maze experiments?
-Similar cages, same food, automated equivalent conditions.
-Minimum human handling; a mechanical system for delivering rats to the maze.
Serial Position Effect
Enhanced recall for the 1st and last few items on a list.
What is the difference between ecological validity and external validity?
Ecological validity tells us whether or not the findings can be generalized to "real-world" settings, whereas external validity refers to whether the study can be generalized to the population you are studying across settings and time.
The surface similarity between the experimental environment and real-world settings. It is less important than psychological realism.
Represents the degree to which the experiment setting is made psychologically involving for participants, thereby increasing the likelihood that they will behave naturally rather than self-monitor and possibly distort their responses.
Refers to the process of repeating a study in order to determine whether the original findings will be upheld. It is central to science.
What kind of validity: Was it exposure to the IV that truly causes the changes to the DV?
What kind of validity: Did the experimental manipulation truly create high and low social support, and not something else? Did the measurement techniques truly assess stress?
What kind of validity: Do the findings generalize to other populations and settings?
What kind of validity: Was the data analysis appropriate and properly conducted?
Statistical Conclusion Validity
What are the 7 different threats to validity?
Maturation, Attrition, Selection, History, Instrumentation, Regression To The Mean, Testing
(MASHIRT) That's "Ma Shirt!"
When a patient experiences full or partial recovery without treatment - occurs with many disorders and illnesses.
Refers to ways that people naturally change over time, independent of their participation in a study.
Includes cognitive changes and changes in physical capabilities that occur with aging, fluctuations in alertness and fatigue that accompany biological rhythms, and normal recovery from physical illness or psychological disorders. It also includes the general accrual of knowledge and skills as we gain more experience over time.
Ways in which people naturally change over time, independent of their participation in a study.
Refers to events that occur while a study is being conducted, and that are not a part of the experimental treatment or manipulation.
Examples are the national or local economy, time of year (spring, summer, fall, winter), wars, elections, etc.
Events to which people are exposed while participating in a study, but that are not part of the experimental manipulation (treatment or intervention) being examined.
Concerns whether the act of measuring participants' responses affects how they respond on subsequent measures.
Examples are when participants take the same inventory twice - does the previous pretest experience affect their post test scores. The second time around people are more familiar, less anxious, etc., creating a testing confound.
Changes in motivation, practice habits, that occur in response to taking a performance pretests also illustrate testing effects.
The act of measuring individuals responses may affect their responses on subsequent measures.
Refers to changes that occur in a measuring instrument during the course of data collection.
Examples are using a poorly constructed weigh scale that over time wears down and under or overestimates weights.
Another is the clinical psychologist as a measurement instrument. When observing each participant, there are systematic changes in their performance thus creating an instrument effect - illness, fatigue, gaining experience with a new system, and non consciously adopting different criteria when deciding how to rate or categorize a response.
Changes that occur in a measuring instrument during the course of data collection.
Also called subject loss.
Occurs when participants fail to complete a study. It can occur for many reasons such as a piece of equipment may malfunction or a participant may feel uncomfortable and not wish to continue.
In longitudinal research, some participants may move away, lose interest, become too ill, or die before the study is completed.
It poses the greatest threat to internal validity when participants who discontinue differ from those who compete the study in some attribute that could account for the changes obtained in the DV.
Regression To The Mean
The statistical concept that when two variables are not perfectly correlated, more extreme scores on one variable will be associated overall with less extreme scores on the other variable.
When people selected for a treatment have an extreme pretest mean score and a less extreme post-test mean score (ex: high depression scores decrease; low performance scores increase), this may partly reflect statistical regression.
Refers to situations in which, at the start of a study, participants in the various conditions already differ on a characteristic that can partly or fully account for the eventual results.
How do experiments address the validity threat of history?
Outside national or local events can have an effect, so by using block randomization to assign participants to the various conditions, the potential influence of history effects should be distributed equivalently across the conditions.
Block randomization renders poorly executed experiments (measuring students at the beginning of a semester then again right before midterms) will render this issue and other history effects (time of day, week, events) implausible as confounding variables.
How do experiments address the validity threat of maturation?
Experiments don't prevent maturation, but random assignment can help to assume that maturation effects would be equivalent across the various conditions.
How do experiments address the validity threat of testing?
Most tests don't include a pretest due to random assignment, therefore conditions are assumed to be equivalent from the beginning which eliminates the possibility of testing as a confound. But, if you do use a pretest, if all participants take it then the testing effects should be equivalent in all conditions so that testing isn't a confounding variable.
How do experiments address the validity threat of instrumentation?
Utilizing random assignment and block randomization and/or proper counterbalancing procedures will cut down on instrumentation effects because participants are equivalent across conditions.
How do experiments address the validity threat of regression to the mean?
Do not select participants based on extreme scores, but if you do, then random assignment assists in eliminating regression to the mean as plausible confounding variables.
Occurs when significant different attrition rates or reasons for discontinuing exist, overall, across the various conditions. Suggests that something intrinsic to certain conditions caused attrition and may bias the results.
Even if you assume that random assignment created equivalent groups at the beginning of the experiment, differential attrition can result in nonequivalent groups by the end of the experiment.
How do experiments address the validity threat of attrition?
Experimenters should determine why participants discontinue and examine pretest scores to determine whether continuing vs. discontinuing participants differ, overall, in ways that could plausibly account for the findings. In many lab experiments, attrition may be minimal or nonexistent.
How do experiments address the validity threat of selection?
Experimenters use multiple conditions, and when between-subjects designs are used, the key to preventing a selection confound is to create equivalent groups by random assignment.
Randomized Controlled Trial
Also called randomized clinical trial; an experiment in which participants are randomly assigned to different conditions for the purpose of examining the effectiveness of an intervention. They are conducted in fields such as clinical, counselling, health, educational psychology, psychopharmacology, medicine and nursing.
Wait-List Control Group
A group of randomly selected participants who do not receive a treatment, but expect to and receive it after treatment of the experimental group(s) end.
True or False? Simply by taking the same test twice, participants mean score improves. This is called an instrumentation effect.
True or False? Outside events cause participants responses (DV) to change during a study. This is called a history effect.
Ture or False? Highly aggressive children are randomly assigned to a behaviour therapy or control condition. Mean pretest aggression scores in the two conditions are equivalent. the therapy group shows a larger pre- to post-test aggression score decrease than the control group. Statistical regression is a major confounding variable.
True or false? Even in a randomized experiment, the potential for attrition remains a concern.
Unwritten rules about how research participants ought to behave.
Good Subject Rule
An implicit norm that calls for participants to provide responses that help support the perceived hypothesis of the study - party arises from people's hope that their responses will contribute to the science and the study's success.
Conversational strategies conducted during debriefing in which experimenters explore participants' beliefs about the study and hypothesis - are probably the most common approach to addressing demand characteristics are influencing a participants behaviour.
For them to be effective, the experimenter first needs to establish rapport with participants during debriefing, introduce probes gradually, being probing prior to revealing the true hypothesis, and progressively pursue the participants beliefs in greater depth if they initially claim they weren't aware of the hypothesis - before rapport is established.
What are some ways to address demand characteristics?
-Suspicion probes - conversational strategies during debriefing.
-Increasing the psychological realism of the experiment so that participants will be more involved in the situation and more likely be behave spontaneously.
-Pilot testing the experiment to assess characteristics ahead of time.
-Using dependent measures that are unobtrusive or difficult for subjects to distort (measures of nonverbal behaviour or physiological responses).
-Avoid within-subjects designs.
-Identifying subjects who claim they are aware of the hypothesis and analyzing their results separately to gain insight into whether their knowledge affected their responses.
-Manipulating participants knowledge of the hypothesis (tell some the hypothesis and others an opposite hypothesis, and not others at all) to determine whether this affects responses on the DVs.
-Use red herring technique.
Red Herring Technique
Refers to diverting people's attention fro ma real issue by raising an irrelevant issue. Telling participants fictitious stories about the experiments purpose and create misleading demand characteristics to divert attention of any participants who may still be trying to figure out the hypothesis.
*Effective but adds ethical drawback of deception.
How do you address experimenter expectancy effects?
- Rigorous training for experimenters and a well scripted protocol. Standardize behaviour.
- Automating participants instructions and the procedures for presenting tasks and collecting data.
- Masking (blinding) in which the parties involved in an experiment are kept unaware of the hypothesis being tested and the condition to which each participant has been assigned. Keeping an experimenter blind is accomplished by assigning the role to someone other than the researcher who designed the study.
People's expectations about how a treatment will affect them influence their responses on the DV to that treatment.
Sometimes a patients mere expectation that a drug will help them partly or fully can produce pain reduction, immune and hormonal functioning, sexual arousal, social behaviours, and even symptoms of Parkinson's.
Yoked Control Group
In which each control group member is procedurally linked (yoked) to a particular experimental group member, whose behaviour will determine how both of them are treated.
Typically done via random assignment or matching.
By yoking one control group member to each experimental group member, upon completion of an experiment the two groups should overall been studied/evaluated/tested, an equivalent number of times and any resulting differences between the two groups can't be attributed to plausibility to confounds created by differences.
They are designed used when procedures applied to each experimental group member (number of forced awakenings, amount of reinforcement given for correct responses) vary because they depend on the participants own responses (number of episodes, correct performances).
They make the overall treatment of the experimental and control groups as similar as possible on factors other than the intended manipulation of the IV.
Why are floor and ceiling effects problematic?
Because they can lead to the false conclusion that an IV doesn't influence a behaviour when in fact it does.
They are not limited to performance tasks.
What are floor and ceiling effects examples of?
Reduced variability in the scores on a dependent measure. A problem that can affect correlational studies as well as experiments.
Refers to the ability to detect an effect that is actually present.
A sensitive measure is one that can detect differences or changes that actually occur in a response, even when they are small.
It is also a term applied to the overall design of an experiment which is one that is likely to detect the influence of an IV on a DV if indeed, in the true state of the natural world, such an effect exists.
A trial run, usually conducted with smaller numbers of subjects prior to initiating the actual experiment. It allows experimenters to practice the procedures, verify if equipment is functioning, and identify potential problems in the way IVs are manipulated and DVs are assessed.
Measures to asses whether the procedures used to manipulate an IV successfully captured the construct that was intended.
A conversation with the participant that conveys additional information about the study.
A cue that influences participants beliefs about the hypothesis being tested and about how they should behave is called ____________________ ?
The purpose of the double-blind procedure is to control for ____________________ effects and ____________________ effects.
Experimenter expectancy; placebo.
In a ____________________ control group design, the responses of each experimental group member determine how she or he and a corresponding group member will be treated.
Occurs when researchers follow up their initial study with one or more replications and present this series of studies in a single research report.
A replication conducted by researchers who weren't part of the original research.
Also called full replication, includes all the conditions of the original study.
It only includes some of the original conditions from a study.
Or exact replication, the researchers follow the procedures used in the original study as closely as possible.
Examines the same question investigated in an original study, but operationalizes the constructs differently. Typically, it operationalizes the IV, the DV, or both in new ways.
Replication and Extension
Also called replication with extensions, is a replication that adds a new design element to the original study. It can be full or partial, direct or conceptual. It often involves one or more of the following:
-Adding new conditions to an original IV
-Adding a new IV.
-Adding a new DV.
Researchers can also change the time periods, a long term follow up assessment, etc.
Extending Independent Variables
Adding a new IV or new conditions to an existing IV is an excellent strategy for assessing external validity.
Researchers add new conditions to quantitative IVs for many reasons and adding new IVs allow for the examination of external validity across populations and settings.
Adding a selected IV such as age, gender or severity assist in assessing generalizability.
What are some advantages of replication and extension?
To examine interaction effects you need to have more than one IV. By created a 2x2 factorial design, you can now combine whether there is a main effect such as therapy, age group, or whether an interaction occurs. Also, by adding different age groups, you can replicate findings to other areas of the population.
Why are factorial designs ideal for assessing the external validity of previous research findings?
Different populations, tasks, stimuli, or other aspects of the setting can be added as a new IV and factorially combined with the original IVs. The conclusions drawn about external validity will depend on the nature of main effects and interactions obtained.
THIS SET IS OFTEN IN FOLDERS WITH...
Chapter 4 - Defining and Measuring Varia…
Chapter 6 - Case Studies and Observational Research
Chapter 7 - Survey Research (Under Construction)
Reliable & Valid Measurement in Psychological Scie…
YOU MIGHT ALSO LIKE...
Ch. 8 - Introduction to Quantitative Research
Research Methods- Chapter 6
Research Methods Chapter 4 questions
OTHER SETS BY THIS CREATOR
Chapter 6 - Chronology Building: How To Get A Date
Chapter 5 - Geoarchaeology and Site Form…
Chapter 4 - Doing Fieldwork
Chapter 3 - Doing Fieldwork: Surveying For Archaeo…
OTHER QUIZLET SETS
Religion Final Study Guide 2
AP Chapter 11 Diagnostic Criteria
Orthopedics Exam I - HS