145 terms

NCE/Research and Program Evaluation

often used in casual-comparative studies, tests two/more groups while controlling for extraneous variables that are often called 'co-variates'
a test for three or more means, is used to test for differences among two or more groups, yields an F ratio.
Fatorial Analysis of Variance
Is used in the study of two or more variables. The 2x2 design, in which there are two independent variables each with two distinct values, is the most common, although multi-level designs can be used.
is used when there is more than one dependent variable.
Pearson Product Moment Correlation
used with interval and ratio data. Correlation describes a relationship or assocaition between variables
whether two means or correlation coefficients deffer significantly from each other
Chi-square Test
whether an obtained distribution is significantly different than an expected or theorectical distribution
Non-Parametric (Kruskal Wallis)
is a non-parametric method for testing equality of population medians among groups
Spearman Rho
if the data in the correlational study are expressed as ranks then this used, used to calculate a correlation coefficent, both variables are ordinal.
Non-Parametric (Mann-Whitney U)
used to determine whether 2 uncorr/unmatched means diff significantly
Career Decision Scale (CDS)
self reporting measure suitable for high school and college aged students
used with normally functioning individuals, most frequently college aged population
Kuder career inventory
interest inventory that makes the assumption that a person will find satifaction in an occupation where workers have similar interest patterns
standarized personality test designed to assess some major personality characteristics that affect personal and social adjusments
measure of personality dispostion gives a four letter code used for interpreting persoanlity type
Rorschach Inkblot
a projective test that utlizes ten 6 by 9 inch cards
self administered career interest assesement
compares a persons interests with those or persons who have been in their occupation for at least three yrs
a projective test made up or 30 cards of pictures and clients asked to make up story
most popular adult intelligence test, ages 16 to 89
measure intellectual functioning of children, ages 6 to 16
measure intellectual functiong of young children, ages 3 to 7
useful for assessing emotional disturbance most researched test in history
16 PF
measures key personality factors like assertiveness, emotional maturity and shrewdness associated with Raymond Cattell
Parametric tests
assume a normal distribution, selection of participant is independent data must represent interval and ratio scale, have more power than non parametric tests
Chi-square of independence
used when you have categorical/count data, is useful for considering rela b/t two categorical var, will seek therapy or wont. Diff b/t what you observe and what you expect (is formula for this).
Non-parametric tests
measure qualitative attributes,characteristics, and catergories, data is nominal or ordinal.
Solomon four-group design
considered a true experimental design sins each group is chosen via random sample, design for a group experiment. One control group receives a pre-test and one experimental group receives a pre-test and the other control/experimental group do not
Causal comparative research
groups are randomly assigned and current researcher didn't truly control the IV in the study
Wilcoxon singed-rank
test examines whether 2 corr means differ signif from each other
standard deviation
the square root of the variance
Z-scores/standard score
are the same as standard deviatations
has a mean of 50 with every 10 points landing at a standard deviation above of below the mean
Hawthorne effect
if subjects know they are part of an experiment their performance or if they are given more attention because of the experiment their performance imporves
Rosenthal effect
Known as experimenter effect, Pygmalion effect, experimenters beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenters expectations
Halo effect
Transfer of goodwill or positive feelings about one characteristic (such as pleasing appearance) of a product or person to another, possibly unrelated, characteristics (such as performance)
Correlational Research
This type of research involves one group of people who have two or more variables measured, research relates two or more variables to determine whether and to what degree a relationship exists between them, does not establish a cause and effect relationship, nor does it involve a treatment.
Causal Comparative Research
research is sometimes called ex post facto research because both the effect and the cause of the effect have already occurred. No treatment applied, can establish a cause and effect relationship.
Experimental Research
researcher will manipulate one or more IV in a controlled setting in order to determine the effect on the DV, is the same as causal comparative except that a treatment must be administered to the experimental group and not to the control group.
Independent Variable
It is the cause, experimenter manipulates the.....
Dependent Variable
it is the effect, test score or observation, experimenter has no control over.......
Action Research-Kurt Lewin
not generalizable, solve practical problems through the use of the scientific methods, "learning by doing" - a group of people identify a problem, do something to resolve it, see how successful their efforts were, and if not satisfied, try again
Historical Method Research
involves studying and explaining past events, purpose is to arrive at a conclusion concerning cause, effect or trends of past events that help explain current or predict future events.
Qualitative Method Reshearch
involves researcher observing people or events in their natural setting, main purpose is to understand the effect of the particular context on the events.
Threats to Internal Validity
History, Maturation, Testing, Insturmentation, Statistical Regression, Biases or Differential Selection, Mortality, Selection
affects the performance on the independent variable during th time of the treatment bust is not the independent variable.
the physical or mental changes that occur within subjects over time.
improved results on the post-test as a result of the subjects having taken a pre-test.
results from a problem with instruments used for assessment, changes in the measuring device or procedure over the course of a study.
Statisical Regression
the tendency for scores to regress toward the mean, when one of two groups scores rather high or low prior to the study, control by random assignment.
Biases or Differential Selection
when groups are different before the study begins due to the use of already formed groups.
Mortality (attrition)
occurs when subjects drop out of the study.
when already formed groups are used one may profit more from the treatment than the other.
Threats to External Validity
Pretest treatment interaction, Multiple treatment interference, Selection treatment interaction, Specificty of variables, Experimenter effects, Reactive arrangments.
Pretest Treatment Interaction
when subjects respond differently to treatment because they have been pre-tested, so results are only generalizable to other pre-tested sujects.
Multiple Treatment Interference
when more than one treatment is administered in an experiment, the earlier treatments may affect the later treatments.
Selection Treatment Interaction
occurs when subjects are not randomly selected from the population. If sample not representative of population, results cannot be generalized to the populaiton.
Specificity of Variables
while the populaiton and the sampling method must be described in detail, care should be taken that the experimental situation is still generalizable.
Experimenter Effects
when the researcher causes the results to be different from what they would have been.
Experimenter Bias Effect
when the researcher's expectations affect the researcher's behavior and it then will affect the outcomes.
Experimenter Personal Attributes
when the researcher's personal attributes such as temper, sex, age etc., affect the outcomes.
Random Sampling
a process of selecting individuals from the population so that each individual has an equal and independent chance of being selected.
Stratified Random Sampling
a process or selecting individuals from a population in such a way that the subgroups in the population are represented in the sample, the population is divided into subpopulations and random samples are taken of each stratum.
Cluster Sampling
a process in which groups rather than individuals are randomly selected.
Systematic Sampling
individuals are selected from a list by taking every "th" name, ex. taking every fourth namefrom the phone directory.
True Experimental Design
ex, pre-post-test control, post-test only, and Solomon Four group, provide excellent control, is one in which the researcher manipulates the Independent Variable (or variables) to observe its effect on (the dependent variable) to test casual relationship, uses random assignment to groups in order to control external factors from influencing the results.
Quasi-Experimental Design
subjects are not randomly assigned, manipulates one or more independent variables
Factorial Design
allows the testing of more than one hypothesis, two or more independent variables are manipulated simultaneously in order to study the independent effect of each variable on the dependent variable
lowest level, is qualitative, categorizes groups as male/female, freshman/seniors, etc. Expressed as names or numbers, mode is most appropriate.
2nd level, is quantitative, calculations of median and variability of quartile deviation, has magnitude only, and can be looked at as any set of data that can be placed in order from greatest to lowest but, there is no absolute zero and no equal intervals
3rd level, possesses magnitude and equal units, no absolute zero, Temperature is a classic example we know that each degree is the same distance apart and we can easily tell if one temperature is greater than, equal to, or less than another. Temperature, however, has no absolute zero because there is (theoretically) no point where temperature does not exist.
Highest level, has an absolute zero and equal intervals, physical scales of time, length, and wieght, one can be twice, three times as much as another, which means addition, subtraction, multiplication, and division can all be used.
Measure of Central tendency
used to describe the average score of a group of scores.
Measures of Variabiliy
describes how spread out a group of scores are or how they spread themselves from the cetral tendency. Examples the range and standard deviation,
Measures of Relative Position
compare a subject's performance to that of the other subjects. Examples percentile rank, standard score, z scores, t scores, stanines.
Measures of Relationship
describe the degree of relationship between tow sets of scores. Examples correlations, Spearman rho, Pearson r.
Standard Deviation
A measure of spread within a distribution (the square root of the variance). The most popular and most reliable measure of variability.
Standard Score
a measure of relative position when the test data are of interval or ration measurement, the distance between the raw score and the mean of the norm in terms of the standard deviation.
Raw Score
represents the number of points a student received for correctly answering questions on a test or for a content area.
Z Scores
raw score minus the mean then divided by the standard deviation, always has a mean of 0 and standard deviation equal to 1, same as standard score
T Scores
A standard score that sets the mean to fifty and standard deviation to ten. Used on a number of tests including the MMPI.
Non-Parametric(Median Test)
will determine if the proportion of subjects falling above and below the median is different from the control group.
Non-Parametric(Sign Test)
asign a + for direction above and a - for direction below an identifed point.
is used when the data is in the form of frequency counts and represents a nominal scale.
used for muliple comparsions, ex. if F-ratio is significant this is used to determine where the significance lies.
this situation is an experimental simulation of some aspect of the counseling process.
Control group
subjects not recieving the active treament.
Ex Post Facto Design
looks like a posttest, research takes place after groups have been formed, design where the independent variable cannot be manipulated, such as persoanlity, gender, and race
External Validity
refers to the gneralizability of the results.
F-Ratio Chart
chart is utilized to read the significance level for critical values for the F-test
Double Blind Study
the researcher or participants do not know the hypothesis or who belongs to the treatment or control groups.
Inferentaial Statistics
A procedure used to estimate parameters (characteristics of populations) from statistics (characteristics of samples).
A measurable characteristic of a population
A characteristic of objects or subjects that can take on different values.
Descriptive Statistics
Numbers used to describe information or data or those techniques used to calculate those numbers.
Pilot Study
mini-study which aids in the development of a full-blown research design and study.
Meta Analysis
a process of combining defferent studies involving similar variables by estimating the effect size in each study.
reffered to as the probability that a true relationship does exist.
as a tendency of scores to move toward the mean, is a statistical technique used to predict a variable (success) when a second variable (skill) is known, that is the relationship between success and skill.
True Score
score that the participant would have obtained if measurement was perfect—ie, we were able to measure without error, hypothetical score based upon no measurement of error
Type I (Alpha)
occurs when the null hypothesis is true, but the researcher rejects the null hypothesis.
Type II (Beta)
is when the null hypothesis is false, but the researcher accepts it.
Independent group comparison design
the two groups in an experiment are independent of each other (the change in one group did not influence the change in the other group)
Repeated-measures comparison design
research measures the same group of subjects without the IV and with the IV
T-Test, simplistic form of the analysis of variance
When comparing TWO sample groups this is used
Accept, Your computation must EXCEED the critical t in order to reject the Null.
If the t-value obtained statistically is lower than the t value (critical t) then you ____ the null hypothesis.
Analysis of Variance, ANOVA
If there are more than 2 groups, then the ____ is used
an F-statistic, If F obtained exceed the critical F-value, you reject the Null.
The results of ANOVA yield...
ANCOVA, analysis of covariance
To test 2 or more groups while controlling for extraneous variable (co-variates) you use the
When the data are nonparametric, the _____ is used (instead of the one-way ANOVA)
versions for various age brackets or levels of education
In testing, a vertical test would have...
In an ipsative measure the person taking the test must compare items to one another. The result is that...
you cannot legitimately compare two or more people who have taken an ipsative tests. (does not reveal absolute strengths) - it points out high and low between a single individual.
The Q-sort involves a procedure in which...
someone is given cards with statements and asked to place them in piles of "most like me" to "least like me." Then the subject compiles them to create the "ideal self," which can be compared to her current self-perception in order to assess self-esteem.
correlation coefficients; 1.00
Reliability and validity is expressed via_____ and the closer they are to,____ the better.
the Kuder-Richardson coefficients of equivalence. (KR-20 or KR-21 formulas)
A counselor is told by his supervisor to measure the internal consistency reliability of a test but not to divide the test in halves. The counselor would need to utilize...
the Kuder-Richardson coefficients of equivalence.
A counselor is told to measure the internal consistency reliability (i.e. homogeneity)of a test but not to divide the test in halves. The counselor would need to utilize...
inter-item consistency (if each item on the test is measuring the same thing as every other item.
internal consistency or homogeneity of items is also known as...
70% of the score is accurate while 30% is inaccurate. (70% is true variance, 30% error variance).
A reliability coefficient of .70 indicates that...
49% To demonstrate the variance of one factor accounted for by another you merely square the correlation. This principle is referred to as the coefficient of determination.
The same test is given to the same group of people using the test-retest reliability method. The correlation between the first and second administration is .70. The true variance is...
Raymond Cattell
The 16 PF reflects the work of
factor-analytic tests, theory-based tests
Tests and inventories like the 16PF that analyze data outside a given THEORY are called _______ ____ rather than _____-______ tests.
a causal comparative design (data from this design can be analyzed with a test of significance just like any true experiment)
an experiment that is a true experiment except for the fact that the groups were not randomly assigned is called...
withdrawal designs
ABA or ABAB paradigms are sometimes referred to as...
multiple-baseline design
when a researcher employs more than one target behavior, the term ______ is used.
ratio or interval data and rho is used for ordinal data
The Pearson Product-Moment correlation r is used for ____ data while the Spearman rho correlation is used for ____ data.
In the Solomon four-group design...
the researcher uses two control groups. Only one experimental group and one control group are pretested, the other groups are merely post-tested. (lets the researcher know if the results are influenced by pretesting)
a distribution with class intervals can be graphically displayed via a bar graph also called a
What the John Henry Effect?
is a threat to internal validity of an experiment that occurs when subjects strive to prove that an experimental treatment that could threaten their livelihood isn't all that effective.
the abscissa, the ordinate
the x-axis and y-axis are also called what respectively?
tail points right
positively skewed
tail points left
negatively skewed curves
two or more
Factorial designs include ____ IVs
peakedness in the curve. Mesokurtic: normal curve with peak in middle (meso)
kurtic in "mesokurtic" refers to....
the comparison group lowers their performance in an inept manner because they have been denied the experimental treatment.
Another control group phenomenon that threatens internal validity in research is the "Resentful Demoralization of the Comparison Group" - explain this.
The Rosenthal/Experimental Effect: the experimenter falls in love with her own hypothesis and the experiment becomes a self-fulfilling prophecy.
What is the Pygmalion Effect?
any psychotherapeutic model that focuses on the here-and-now rather than the past.
What is "ahistoric therapy"?
from a single socioeconomic group
Horizontal sampling occurs when a research selects subjects...
persons from two or more socioeconomic classes are utilized.
Vertical sampling occurs when...
systematic sampling (selecting every 10th person in a sample)
A researcher wants to run a true experiment but insists she will NOT use a random sample. You could safely say that she could accomplish this by using...
matched in regard to any variable the could be correlated with the DV.
In a matched design the subjects are literally...
Mann-Whitney U test
to determine whether 2 UNcorrelated/UNmatched means differ significantly from each other.
Wilcoxon signed-rank test
examines whether 2 correlated means differe significantly from each other.
independent groups
Unmatched/Uncorrelated could also be termed...
2 = t-test
If you were using a parametric test to examine a null hypothesis for two means, you could rely on a
3+ = ANOVA and the f-test
if you had 3 or more groups, then the ______ and the ____ would be required.