Research Methods I
Terms in this set (139)
Measurable quantities that change under different circumstances rather than remaining constant.
Empirical research is concerned with the relationship between variables. True or False?
A variable in an experiment that is controlled and manipulated by the researcher to see what effect it has on subsequent behavior.
The IV is represented on the X or Y axis?
The X axis.
A variable whose value is affected by the value of an independent variable; the observed effect.
The DV is represented by the X or Y axis?
The Y axis.
What is the equation to measure the relationship between the independent and dependent variables?
Dependent = Y Intercept + Slope(Independent)
What is the relationship between the variables in a positive linear function?
A positive linear function yields a positive relationship between variables: when X increases, so does Y.
What is the relationship between the variables in a negative linear function?
A negative linear function yields a negative relationship between variables: when X increases, Y decreases.
This type of research allows us to draw quantitatively based conclusions about cause and effect;
Looking for predictable relations among variables
Or "Independent Variable," is a variable that can be manipulated by the experimenter.
Variables that cannot be manipulated, because they are attributes of the individual that are preexisting.
Give examples of attribute variables.
Age, height, gender, race, intelligence, a disorder like SLI.
Variables that may be measured along some type of dimension.
Measured on a scale
Values change rather smoothly
Give examples of continuous variables.
Age, vocal intensity, temperature, GPA
Variables that can only be categorized, or named; cannot be measured along a continuum
Give examples of categorical variables.
Type of dysphagia, eye color
Name the 3 types of Experimental Research.
Bivalent Experimental Research
Experimenter studies the effects of two values; of 1 IV and 1 DV
e.g., yes/no, presence/absence/ noise/no noise
Multivalent Experimental Research (+give example)
Experimenter studies the effect of several values of that IV; at least 3 values must be present.
e.g., chocolate consumption (low, med, high) and weight gain
Parametric Experimental Research (+give example)
Experimenter is examining the simultaneous effect of more than one IV on a DV; two or more IVs running concurrently
The interaction of the first IV and parameter (second IV) on the DV in a parametric design.
Describe the (two) different types of effects the IVs can have on the DV in a parametric experimental research design.
During the interaction effect, one IV will have a more pronounced effect on the DV; this is the main effect. The other, less pronounced effect is the interaction.
Interaction effect can only be observed when two or more IV are studied simultaneously. True or False?
Name some advantages of Experimental Research
-Can observe cause and effect relationships quantitatively
-Can repeat observations under same conditions
-Can control events to make observations on our own time.
Name some disadvantages of Experimental Research.
-A single study does not prove a cause and effect relationship.
-Need multiple repetitions to prove that there is a relationship.
-Always some level of artificiality
-Unethical to conduct many experiments
-No 2 participants are the same
How many studies does it take to disprove a theory?
Only one study.
Conducted to observe group differences, developmental trends, or relationships among observable variables that can be measured by the researcher; observation of what exists without manipulation of the IV.
How are participants assigned to groups in descriptive research?
Participants are self-assigned based on inherent characteristics
We can also establish cause and effect relationships with descriptive research. True or False?
Name the two variables present in descriptive research studies.
Attribute (or classification, like IV) and Criterion (or predicted, like DV)
Name the six types of descriptive research.
Comparative Descriptive Research
Case Study "
Comparative Descriptive Research
Measures the behavior of 2 or more types of subjects at one point in time, to draw conclusions about their similarities or differences.
Name 3 types of Comparative Descriptive Research.
Bivalent Comparative Descriptive Research
Selection of participants from different classifications
E.g., children vs. adults
Multivalent Comparative Descriptive Research
Comparison of 3 or more groups that are classified among some type of continuum.
e.g., groups with mild/moderate/severe phonological delays
Parametric Comparative Descriptive Research
Groups that differ with respect to 2 or more classification variables.
e.g., comparing gestures and word development in children with downs syndrome vs. normally developing children
Developmental Descriptive Research
Measuring changes of behavior or characteristics of people over time to examine the influence of aging.
Name the 3 subdivisions of Developmental Research.
Cross-Sectional Developmental Descriptive Research
Have a selection of subjects from different age groups for observation for observation of differences among average behaviors or characteristics of the groups.
e.g., look at effect of noise exposure over 3, 6, 8, 15 yrs
Name some advantages and disadvantages to the cross-sectional design.
Adv: less expensive and time consuming than longitudinal
Disadv: may not be able to attribute changes to age alone, as people have varying life experiences, and we are (over)generalizing results.
Longitudinal Developmental Descriptive Research
Follow and measure the same subjects over a period of time; looking for age related and behavioral changes.
Name some advantages and disadvantages of longitudinal studies.
Adv: can naturally observe changes over time
Disadv: expensive, time consuming, attrition
Semi-Longitudinal Developmental Descriptive Research
(or Cohort Sequential) divides the total age-span of the study into several overlapping age groups
e.g., 0-6yrs, 3-9yrs
Name some advantages of semi-longitudinal design
Adv: observe naturally occurring changes in same subjects, compare changes to other groups, less expensive than longitudinal
Disadv: making an assumption/overgeneralization based on results
Correlational Descriptive Research
The study of the relationship among 2 or more variables by examining the degree to which changes in one variable corresponds with or predicts changes in another.
Name some advantages and disadvantages of correlational descriptive research.
Adv: can be used to estimate the amt. of variation in the DV that may result from an attribute variable.
Disadv: correlation does not imply causation; extraneous variables to contribute to results
Retrospective Descriptive Research
To examine data that we already have on file before the formulation of a research experiment.
Name some advantages and disadvantages of retrospective research.
Adv: saves time, cheap, try to establish source of problem without causing the problem, data has already been collected
Disadv: validity issues, no control over participant selection or data collection procedures or methods of measurement
Survey Descriptive Research
Administration of a verbal, written, over-the-phone, or face-to-face survey.
Interactive (rather than observation)
Used to provide a detailed inspection of the prevalence of conditions, practices, or attitudes in a given env.
Name some advantages and disadvantages of survey research.
Adv: may be only means of getting certain responses (socially sensitive issues)
Disadv: poor content validity, accuracy (bias, lies), poor response volume
Case Study Descriptive Research
Look at individual(s) in a certain type of situation; want to reveal certain principes that might be overlooked
Name some advantages and disadvantages of case studies.
Adv: reveal important, in depth info that may have been overlooked in group-design studies.
Disadv: no generalize-ability, usually outliers
From a statistical standpoint, what is the weakest type of study?
Case Study (Descriptive Research)
What are some advantages to conducting descriptive research designs?
Allows researchers to study variables that can not be manipulated experimentally.
What are some disadvantages to conducting descriptive research designs?
Can not develop conclusions about cause and effect relationships
Can not overgeneralize or assume that this relationship exists outside of the test population
Investigate the effect of manipulation of one or more IVs on the performance of participants who have been selected based on attribute variables, such as age, gender or pathology.
Single Subject Research Design
To provide data concerning the typical behavior of individual subjects under an experimental condition.
Not necessary to assume that each subject will respond similarly to experimental conditions.
Necessary for each subject to be run more than once under each experimental condition.
Group Research Design
To provide data concerning the behavior of the typical member of a group under an experimental condition.
Necessary to assume that the subjects are going to respond similarly in an experimental condition
Not necessary for subjects to be run more than once
Between Subjects Design
The performance of separate groups are measured and the comparisons are made between the groups.
Name the 3 types of experimental between subject design.
Bivalent Experimental Between Subject Design
One experimental group is compared to one control group to study the effect of the experimental treatment.
Parametric Experimental Between Subject Design
Several groups can receive different values of different IVs in different combinations
Name the 3 types of descriptive between subject design
Bivalent Descriptive Btw Subj Design
Bivalent Descriptive Between Subject Design
To study the presence or absence of a treatment; only 2 groups.
e.g., normal hearing vs. deaf
Multivalent Descriptive Between Subjects Design
e.g., mild moderate, severe profound
Parametric Descriptive Between Subject Design
Different groups that receive different values
e.g., gender differences vs. age differences + disorder
What are two ways to equivocate the experimental and control groups?
Assignment of the subjects to an experimental group and to a control group on a random bias.
Match the members of the two groups on all extraneous variables that were considered relevant to the experiment.
e.g., matching on inherent characteristics
Frequency Distribution Technique
Groups are matched in their overall frequency distribution rather than comparing participants case-by-case.
Precision Control Technique
Match pairs of participants for assignment to our experimental and control group on a case-by-case basis.
Within Subject Group Design
The performance of the same subject is compared in different conditions where we would have multiple tests on the same group.
Occurs when participants participate in a number of treatment conditions and their participation in an earlier condition may affect their performance in subsequent conditions.
How do we control for the sequencing effect?
Randomization: present conditions in a random sequence
Counterbalancing: arrange all possible sequences of treatment, then randomly assign the subjects to each sequence.
How are single subject designs and within subjects designs similar?
We are testing participants into all conditions of the experiment that represent all levels of the IV
How do single subject designs differ from within subjects designs?
Single subj design focuses on the analysis of the performance of the individual participant in each condition rather than the group on average.
Time Series Design
(AB) Have one baseline and one treatment
Reversal Treatment Design
(ABA) Have baseline, treatment, stop treatment, then see what happens at new baseline after treatment.
"ABAB" Treatment Design
Baseline, treatment, stop treatment, treatment again
Purpose is to see if the effects after the first treatment will change after the second.
The extent to which the test measures what it set out to measure.
Want to see if the researcher has controlled or accounted for all the factors that could have a significant effect on the data collected.
If the study has good internal validity, then the IV has caused the change in the DV
Name some threats to internal validity.
Differential Selection of Subjects
Interaction of Factors
The effect that merely taking a test may have on scores achieved on subsequent administrations of the test (may be better at taking subsequent test after taking the pretest)
Changes in the calibration on a measured instrument
Phenomenon in which the participants who are selected based on typically low or high scores may change on a subsequent test to perform better or poorer than thought originally.
Differential Selection of Subjects
Selection of subjects to form experimental and control groups
*Prevent this by using randomization and matching.
Refers to the loss of participants
Events occurring between the first and second or more measurements may alter our outcomes; can not control these events (e.g., home environment)
Refers to change in participants themselves over time (e.g., aging, spontaneous recovery)
Interaction of Factors
Interaction of two or more threats to internal validity.
Extent to which we can generalize the results of our study to real-world populations
Name some threats to external validity
Participants who were given a pretest may react differently than those who were not given the pretest
Name 3 types of validity
Criterion Referenced Validity (2 fold)
A measure of the strength of a phenomenon.
Name 6 measures of effect size.
Relative Risk (or Risk Ratio)
The means of two or more groups are compared. A low cohen's d would indicate necessity for a larger sample (and visa versa).
Proposed estimator of effect size that only uses standard deviation of the second group. Uses second group like a control group
Based on standardized difference.
Follows a non-central distribution and degrees of freedom.
Uses an F test for an ANOVA
Estimates for the sample rather than the population
Based on comparing the probabilities of dichotomous events or outcomes in different groups.
Appropriate when both variables are binary in nature.
e.g., comparing the odds of an event in one group with the odds of the same event in another group.
(or Risk Ratio) Looks at the probability of an event relative to some other IV.
Comparing probability rather than odds.
Measures the amount of actual positives that are correctly identified as such.
Measures the amount of negatives that are correctly identified as such.
What is a type 1 error?
Rejecting a true null hypothesis
What is a type II error?
β: Accepting a false hypothesis
What is power?
Rejecting a false hypothesis.
How can we increase power?
Increase effect size
Use a less strict alpha level
Use a more accurate measuring device
Level of Significance
Indicates degree of confidence that the researcher has, that the differences seen in the data would not have occurred due to chance alone.
The probability of making a type I error.
Describe the two most common alpha (α) levels.
p=.05 --> 95% confidence that there is no error and only a 5% chance that the results might have occurred due to chance alone.
p=.01 --> 99% confidence that there is no error and only a 1% chance that the results might have occurred due to chance alone.
Confidence Interval (CI)
A particular kind of interval estimate of the population parameter; used to indicate the reliability of an estimate.
Standard Error of Measurement (SEM)
Average mean value of a distribution that represents the best estimate of the true value of what we are measuring.
The assignment of numerals to objects or event according to rules. The type of measurement is based on the kind of data collected.
Assignment of numbers or symbols to designate subclasses that represent unique characteristics; mutually exclusive categories.
e.g., pass or fail (because you don't "kind of" fail), male or female
Type of measurement for objects or events put into relative ranking by determination of a greater or lesser value.
Mutually exclusive categories + rank order
e.g., Freshman, Sophomore, Junior, Senior
Type of measurement that contains the assignment of numbers to identify ordered relations of some type of characteristic.
Arbitrarily assigned, no absolute zero
e.g., Farenheit/Celcius; standardized scales on CELF 4
Type of measurement: mutually exclusive categories, rank ordering, equivalence of units, constant distance between adjacent intervals, equivalence of ratios among the scales, true zero present
e.g., Decibals; stuttering frequency
Name four factors that affect quality of measurement.
Instructions to participants
The degree to which you can depend on your measurements
A measure will remain stable if we repeat it
The trustworthy-ness of data
The consistency of measurement; would you obtain the same results if you repeated the experiment?
Name 3 categories of reliability
Test/Re-test method. Consistency or repeatability of measurements. Involves performing a complete repetition of exact measurements and correlating the results of the two measurements.
Accomplished by correlating the scores of two different forms of a measure of the same attribute.
(or Split-Half method). Two halves of a measure may be seen as constituting two alternate forms; each half is then correlated with the other to receive the reliability coefficient.
e.g., Administer test to large group of students, then correlate test results of odd vs. even numbered students.
What are two types of errors that may influence reliability? Explain.
Systematic Error: reoccurs consistently with every repeated measure (e.g., audiometer is out of calibration and consistently produces an output of 20dB each time)
Unsystematic Error: occurs in an unpredictable way during repeated measures (e.g., accidentally set audiometer at 2000Hz)
What are the different notations for research designs created by Campbell and Stanley?
X = administration of experimental treatmt.
O= observation and measurement of DV
R=random assignment to the group
MR=participants are matched and randomly assigned
"XO" One-Shot Case Study
Single group observed once after treatment
"01X02" One-Group Pretest/Posttest Design
One group is assembled, pretested, given treatment, then post-tested.
"X01/02" Static Group Design
A group has been exposed to treatment and is being compared to another group that has not been exposed to treatment.
"01X02/0304" Non-equivalent Control Group Design
First group is pretested, exposed to treatment, then post-tested;
Second group is pretested and post-tested with no treatment.
"R01x02/R0304" Randomized Pretest/Post-test Control Group Design
Both groups pre and post-tested
Control group will not receive treatment
High internal validity
Multivalent Mixed Design
Different levels of the IV, or of the treatment
Four groups, all randomized, pre and post-tested.
Three groups given varying levels of treatment, last group not given treatment.
Solomon Randomized Four Group Design
Four groups, all randomized, some pretested, some given treatment, some post-tested.
*review in notes
AB Randomized Time Series Design
Mixed Design (between subjects and within subjects)
Both groups randomized, pretested several times, experimental group gets treatment, both groups post-tested several times.
Group is randomized, pretested several times, given treatment, post-tested several times, remove treatment, observe effects.
The degree to which the score are going to spread out from the center of the distribution