NCE Research

Not the same as causality - simply an association. The _______ between people who have an umbrella open and rain is very high, but opening your umbrella does not cause it to rain.
Correlations go from negative 1 to 0 to positive 1. Zero means no correlation while positive 1 and negative 1 are perfect correlations. A negative .5 is not higher than a correlation of -.5. In fact, correlation of -.8 is stronger than a correlation of .5
Positive correlation
When X goes up, Y goes up. For example, when you study more, your GPA goes up.
Negative correlation
When X goes up Y goes down. For example, the more you brush your teeth, the less you will be plagued by cavities.
is quantitative when one quantifies or measures things
Quantitative research
yields numbers
Qualitative research
when research does not use numerical data
Flaws in research
True experiment
two or more groups are used
Random sampling, random assignment
In a true experiment, the people are picked via _______ and placed in groups using _________.
Systematic sampling
Where every nth person is chosen can also be used, however, researchers still prefer random sampling and random assignment.
Quasi-experimental research
When the groups are not picked at random or the researcher cannot control the IV then it is quasi rather than a true experiment. This type of research does not ensure causality.
Independent variable (IV)
The experimental groups get the __________ also known as the experimental variable.
The control group
Does not receive the IV
DV (Dependent variable)
The outcome data in the study. If we want to see if eating carrots raises one's IQ then eating carrots is the IV while the IQ scores at the end of the study would be the DV.
Null hypothesis
Each experiment has a __________: there is no significant difference in people's IQs who eat carrots and those who don't eat carrots (from example)
The experimental or alternative hypothesis
there is a significant difference between people's IQ's who do eat carrots vs. those who do not.
Type I alpha error
when a research rejects a null hypothesis to be true (when it's correct). In short, you make an error.
Type II beta error
when a research accepts null when it should have been rejected
.05 or less (.01 or .001)
The significance level for the social sciences is usually set at _________. The significance level gives you the probability of a type I error.
N = 1
is known as a single subject design or case study and thus does not rely on IV, DV, control group, etc. Case studies are becoming more popular.
Demand characteristics
are evident when subjects in a study have cues regarding what the researcher desires or does not desire that influence their behavior. This can confound an experiment rendering the research inaccurate.
Obtrusive or a reactive measure
If subjects know they are being observed we refer to the process as ___________. Observers' presence can influence subject's behavior rather than merely the experimental variable or treatment modality.
Unobtrusive measure
when subjects are not aware that they are being measured we say it is __________
Internal validity
is high when an experimental has few flaws and thus the findings are accurate. In other words, the IV caused the changes in the DV, not some other factor (known as confounding extraneous variables or artifacts). When the ________ is low the researcher didn't measure what he thought he measured.
External validity
is high when the results in a study can be generalized to other settings.
t test
a popular parametric test for comparing two means; used to determine whether the mean scores of two groups are significantly different from each other.
ANOVA or analysis of variance (also called a one-way ANOVA)
is used when you have two or more means to compare. The t test and the _________ are parametric measures for normally distributed populations. The _____ provides F values. There are 3 basic kinds of ANOVA: one-way, factorial, and multivariate. Since you now have 3 groups the t-test is no longer appropriate. You would use the one-way analysis of variance when you have only one variable at three or more levels.
F test
will tell you if significant differences are present
Use the MANOVA or multivariate analysis of variance when you are investigating more than one DV. Use the factorial analysis of variance when you are investigating more than one IV/experimental variable (i.e., if you have two IVs it would be called a two-way ANOVA, three IVs, a three-way ANOVA, etc.)
Chi Square, Kruskal-Wallace
If the population is not necessarily normal then a nonparametric test such as a _________ (the most common nonparametric test) or _________ (similar to the ANOVA) can be used.
ex post facto, causal-comparative design
If the researcher did not manipulate the variable and you are looking at after-the-fact data, then the research is not a true experiment but rather an ______ or so-called ________.
Descriptive statistics
are statistics that describe central tendency like the mean, median, the mode, the range, quartiles, the variance, and the standard deviation
Statistical analyses
include correlation coefficients, t tests, ANOVAs, Analysis of Covariance, Chi square, Kruskal Wallis etc.
Cohort studies
examine a group of people who have something in common (e.g., all soldiers who fought in Vietnam or all counselors who received their license in 2007).
Longitudinal research
takes place when the same individuals are evaluated over a period of time. It is usually contrasted with Cross-sectional research that relies on observation or data from a given point in time.
Formative evaluation
takes place during treatment or while a program is going on while summative or outcome evaluation occurs at the end of a program or treatment (e.g., after the final session of counseling).
Between groups design
uses different subjects in the different groups (e.g., one group of subjects for the control group and another group of subjects for the experimental group).
Research is the systematic process of:
collecting and analyzing data for some purpose such as investigating a problem or answering a question
Evidence-based inquiry is:
the search for knowledge using empirical data which as been gathered systematically
Quantitative research
assumes social facts have a single objective reality, tends to study samples or populations, researchers try not to influence collection of data (instruments), statistical methods comparing and contrasting groups occurs, researchers examine for causes and relationships
Qualitative research
assumes multiple realities socially constructed by individuals and groups, tends to study individual units--person, family, community--in natural setting, researchers may be primary instrument for collecting data (through observation), researchers' impressions, judgments and feelings may be used, goal is to describe the nature of things
(type of qualitative research) a description and interpretation of a cultural or social group or system. Data is typically collected through observation and interviewing. The issue of observer bias is important.
(type of qualitative research) conducted primarily through document analysis. Examples might be historical analysis (collecting and analyzing documents describing former events) and legal analysis which focuses on law and court decisions
Threats to internal validity?
1-Selection of subjects (composition of two groups are different to begin with...probably not randomly selected) 2-Instrumentation 3- Maturation (especially important if research data is gathered over a long period of time. 4-Mortality or attrition (losing subjects during the study 5-Experimenter bias (responses of subjects may be influenced by the researcher) 6-History
Threats to external validity?
1-selection of subjects 2-ecological validity (sometimes conditions or physical surroundings of the research are so unique that the results cannot be generalized beyond that study) 3-Subject reactions (hawthorne effect, demand characteristics, experimenter bias or rosenthal effect) 4-placebo 5-novelty and disruption effect
Random sampling
all the individuals in the population have an equal and independent chance of being selected
Stratified sampling
this refers to selecting in such a way that major subgroups in the population will be sampled. these subgroups may be based on ethnicity, gender, age, etc.
Examples of parametric statistics
t-test and analysis of variance
used when a sample is randomly drawn from a population that is normally distributed. You have para (two-sided) data that yields a bell-shaped curve. you assume that the variance of the sample you are studying is homogeneous (similar) to the variance from the population from which your sample is drawn. Rely strictly on interval and ratio data
used when you cannot make any assumption about the shape of the curve or variance of the population scores (they may not be normally distributed or homogeneous). Rely strictly on norminal or ordinal information.
Examples of nonparametric statistics
Chi-square, mann-whitney u test, wilcoxon signed-ranks test for matched pairs, Soloman and the Kruskal-Wallis H-test
Dependent variable
the variable you are measuring or trying to change. the value of this variable depends upon the value of the independent variable you selected. For example: the effect of three kinds of therapeutic techniques (independent variables) on anxiety (dependent variable).
The independent variable is sometimes called:
stimulus variable, predictor variable or experimental variable
The dependent variable is sometimes called:
response variable, outcome variable, or criterion variable.
The null hypothesis
states there is no difference between the variables or groups measured
Conventional significance levels used in research are
.05, .01, and .001
At .05 you are:
willing to accept the possibility of rejecting the null hypothesis in error five times out of one hundred times (if you performed the experiment 100 times).
What happens when the significance level goes down (e.g., .05 to .01)?
Type I error decreases but type II error increases, i.e., the failure to reject the null when you should. As one type of error goes up, the other goes down and vice versa.
Multiple regression
this is the use of the correlation of coefficient to determine the strength of the relationship of predictor (independent) variables on a criterion (dependent) variable. For example: predictor variables such as high school GPA, class rank, and ACT scores may be used to predict the criterion (outcome) variable, which could be end-of-college freshman year GPA
Correlation coefficient
a statistic that indicates the degree or magnitude of relationship between two variables; often abbreviated using lower-case r. It makes a statement regarding the association of two variables and how a change in one is related to the change in another.
A positive correlation is not a stronger relationship than a negative one of the same numerical value. A correlation of -.70 is still indicative of a stronger relationship than a positive correlation of .60. The minus sign merely describes the fact that as one variable goes up the other goes down.
Factor analysis
This is a statistical method using the correlation coefficient to determine whether a set of variables can be reduced to a smaller number of factors.
Scatterplot (scattergram)
This is a graphic representation of the relationship between two variables for a group of individuals.
Likert scale
This is a widely used technique for measuring attitudes or opinions. (memory device: "like")
Biserial correlation
an appropriate correlation coefficient to use when one variable yields continuous data and the other yields data that is dichotomous (memory device: Bi)
studying or measuring characteristics of several groups at the same time, vs. Longitudinal--studying or measuring characteristics of a group over a period of time
Degrees of freedom
the number of observations that are free to vary
Double-blind technique
this occurs when neither the researcher nor the subject knows who is getting the active substance or the placebo
Halo effect
this is the tendency for the observer (researcher or data collector) to form an early impression of the person being observed and then letting this impression influence observations or ratings of that individual. May be positive or negative
one end of a distribution of scores has more variability than the other end resulting in a fan-like appearance when plotted on a graph
there is an equal distribution of scores throughout the range of scores, i.e., around the line of best fit
Inter-rater reliability
in qualitative research, the reliability calculated by correlating the responses of several raters
Observer bias
this is the tendency of researchers to see, hear and remember what they want to
Pilot study
this is a preliminary trial or test of the research techniques and measures
means a control treatment that gives subjects the same amount and kind of attention as experimental group subjects get (this reduces Hawthorne and Rosenthal effect - both are caused by subject reactions).
Rank-order correlation (Spearman rho)
used when the values of the variables are reported in rank from rather than continuous
Semantic differential
a kind of scale in which a person makes a response between adjective pairs (e.g. good-bad, hot-cold) that describes a concept.
Formative evaluation
this is ongoing, process evaluation to measure the effectiveness of a technique or part of a program; tries to determine how well a new technique, process or treatment works.
Summative evaluation
this is a summary or product evaluation designed to measure the effectiveness of a program, usually conducted at the end of a cycle such as a school year, fiscal year, etc. Is conduced to see how well agency or program goals have been met. Usually a product (document) is generated so this is "product" evaluation versus "process" evaluation (formative).
Donald Campbell and Julian Stanley
These individuals wrote a book, which has become a classic in the field, titled, "Experimental and Quasi-Experimental Designs for Research."
Nominal scale
strictly a qualitative scale, and is the simplest type of scale. It is used to distinguish logically separated groups.
the Hawthorne effect
if subjects know they are part of an experiment--or if they are given more attention because of the experiment--their performance sometimes improves. Also called reactive effect of observation/experimentation...reacting to the presence of the investigator
the Rosenthal effect
the experimenters beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter's expectations
switching the order in which stimuli are presented to a subject in a study
The Pygmalion effect
the experimenter falls in love with his/her own hypothesis and the experiment becomes a self-fulfilling prophecy
ahistoric therapy
any psychotherapeutic model that focuses on the here-and-now rather than the past
multiple treatment interference
if a subject receives more than one treatment, then it is often tough to discern which modality truly caused the improvements.
Random sampling
(like sticking your hand in a fish bowl to pick a winning ticket) each subject has the same probability of being selected, and the selection of one subject does not affect the selection of another subject. This procedure eliminates the researcher's tendency to pick a biased sample of subjects.
Cluster sampling
utilized when it is nearly impossible to find a list of the entire population. it solves the problem by using an existing sample or cluster of people or selects a portion of the overall sample.
Horizontal sampling
occurs when a researcher selects subjects from a single socioeconomic group. can be contrasted with "vertical sampling," which occurs when persons from two or more socioeconomic classes are utilized.