NCE Exam - Research and Program Evaluation

199 terms by clabbergirl 

Ready to study?
Start with Flashcards

Create a new folder

Advertisement Upgrade to remove ads

NCE exam stuff for research/eval

Experimental research

the process of gathering data in order to make evaluative comparisons regarding different situations.

The experiment

The most valuable type of research
(used to discover cause-and-effect relationships)

You need at least 30 individuals to conduct a 'true' experiment.

Correlational research requires 30 subjects per variable.

Quasi-experiment

Uses PRE-EXISTING groups, so the independent variable (IV) cannot be altered (i.e., gender or ethnicity), and can't state with any statistical confidence that the IV caused the dependent variable (DV).

Correlation research is quasi-experimental and does not yield _____ - ____ data.

cause-effect

Parsimony
(also known as Occam's Razor)

interpreting the results in the simplest ways
(Literally a tendency to be miserly and not overspend.)

Ex post facto study

A type of quasi-experiment (literally means 'after the fact') connoting a correlational study in which preexisting groups are utilized

Independent variable

the variable the researcher manipulates, controls, alters, or wishes to experiment with
(memory: 'I' manipulate the IV)

Dependent variable

expresses the outcome or the data regarding factors one wants to measure
(memory: 'D' in dependent and data)

Internal validity

Refers to whether the DVs were truly influenced by the experimental IVs or whether other factors had an impact.

Threats to internal validity

maturation of subjects (psychological and physical changes including fatigue due to time involved), mortality (subjects withdrawing), instruments used to measure the behavior or trait, or statistical regression (notion that extremely high or low scores would move toward the mean if utilized again)

External validity

Refers to whether the experimental research results can be generalized to larger populations (other people, settings, conditions).
[If the results of the study only apply to the population in the study then external validity is LOW.]

Causal Comparative design

a true experiment WITHOUT random assignment
(Data from the causal comparative ex post factor [after the fact] design can be analyzed with a test of significance [t test or ANOVA] just like any true experiment.)

Factor analysis

Statistical procedure to summarize MANY variables. (i.e., A test measuring a counselors ability may try to describe 3 important variables that make up an effective helper although hundreds exist.)

Chi-square

Nonparametric statistical measure that tests whether a distribution differs significantly from an expected theoretical distribution of scores.
(Memory: ''chi' like 'chi-a pet' that I expected more from)

Occam's Razor
(also known as Lloyd Morgan's 1894 Canon)

suggests experimenters interpret the results in the simplest manner.

William of Occam

14th century philosopher and theologian.
(Occam's Razor, aka 'parsimony' named for)

Bubbles in research

Considered flaws in research (i.e., rubbing a sticker on car and getting no bubbles - impossible)

Confounded or flawed variable

Undesirable variables that invalidate experiments.
(The only experimental variable should be the independent variable.)

Nondirective is to person-centered as

parsimony is to Occam's Razor
(both are synonymous)

Most counselors see themselves as practitioners, not researchers.

...

Confounding

occurs when an undesirable variable (also known as contaminating variable) which is not controlled by the researcher is introduced in the experiment.

An experiment is confounded when

undesirable variables are not kept out of the experiment.

Basic research

is conducted to advance our understanding of theory.

Applied research
(aka 'action research' or experience-near research)

is conducted to advance our knowledge of how theories, skills, and techniques can be used in terms of practical application.

In experimental terminology, IV stands for _____ ______, and DV stands for ______ _______.

independent variable, dependent variable

Variable

a behavior or circumstance that can exist on at least two levels or conditions.
(a factor that 'varies' or is capable of change)

The variable you manipulate/control in an experiment is the

IV or independent variable
("I am the researcher so I manipulate or experiment with the IV.")

Experimental ethics

subjects informed of risk, negative after effects removed, allow subjects to withdraw at any time, confidentiality of subjects is protected, results will be presented in an accurate format that is not misleading, and will use only techniques trained in.

Research is a necessary factor for professionalism in counseling.

...

To conduct an experiment with a hypothesis, one needs

a control group and an experimental group.

The control group

does not receive the IV
(same characteristics of the experimental group - the averages between the two groups should not differ significantly)

The experimental group

received the IV
(has the same characteristics of the control group the averages between the two groups should not differ significantly)

Surveys should include at least 100 people.

...

R. A. Fisher

pioneered hypothesis testing.

Hypothesis

a hunch or educated guess which can be tested utilizing the experimental model.
A statement which can be tested regarding the relationship between the independent (IV) and the dependent variables (DV).

Null hypothesis

suggests there WILL NOT be a significant difference between the experimental group which received the IV and the control group which did not.
(asserts the samples will not change - will stay the same - even after the experimental variable is applied.)
The IV DOES NOT affect the DV.

Hunch

The experimental or alternative hypothesis.

Experimental hypothesis

"There will be differences between the control group and the experimental group.

Alternative hypothesis
(aka 'affirmative hypothesis')

asserts th independent variable (IV) has indeed caused a change.

Percentile rank

is descriptive statistic telling the counselor what percentage of the cases fell below a certain level.

Percentage score

another way of stating a RAW score.

Use of tests of significance

to determine whether a difference in the groups' scores is significant or just due to change factors.

t test

Used to determine whether two sample groups are significantly different, simple form of the ANOVA, for comparing 2 sample groups
(for "two-groups" or "two-randomized groups" research design)

Independent group comparison design

In a study of two groups, change in one group DID NOT influence the other group.

Repeated-measures comparison design

Measuring the SAME group of subjects without the IV and then with the IV.

Between-subjects design

A research study uses different subjects for each condition.
(Each subject receives only one value of the IV)

Within-subjects design

Two or more values or levels of the IV are administered to each subject.

When you see the letter P in relation to a test of significance it means

Probability

Parameter

A value obtained from a population.
(Summarizes a characteristic of a population, i.e., average male height)

Statistic

A value obtained from a sample.

Traditionally, PROBABILITY in social science research is set at _____ or lower (i.e., 01, .001).

.05
(.05 indicates differences would occur via chance only 5 times in 100.

In social sciences, the accepted probability level is usually .05 or less.

Type I (alpha error) occurs when a researcher rejects the null hypothesis although it is true.

P.05 means

there is only a 5% chance that the difference between the control group and the experimental group is due to chance.
(differences truly exist; the experimenter will obtain the same results 95 out of 100 times.)

Other terms for 'level of significance'

level of confidence
confidence level
alpha level

A study that would best rule out chance factors would have a significance level of P=___.

.001
The smaller the value for P, the more stringent the level of significance.

Another name for 'Type I error'

Alpha error

Another name for 'Type II error'

Beta error

Type II error (beta error) occurs when a researcher accepts null even though it is false.

(memory: RA as in 'residence advisor'...
R - signifies reject when true
A - signifies accept when false

The probability of committing a Type I error equals the level of significance.

The level of significance is also called the 'alpha level'.

Power of a statistical test

1- beta
(Power connotes a statistical test's ability to correctly reject a false null hypothesis.)

Parametric tests have more power than nonparametric statistical tests.

Parametric tests are used ONLY with interval and ratio data.

Type I errors _____ null when it is _____.

reject, true.
(memory: RA - Reject when Applicable/true)

A Type II error is also called a ____ error and means you _____ null when it is _____.

beta, accept, false

Lowering the significance level LOWERS Type I errors, but it RAISES the risk of Type II errors.

Type I/Type II relationship is a seesaw.

Lower significance =

higher risk of Type II errors

The safest way to avoid Type I/Type II errors is to set alpha (significance level) at a very stringent level and use a large sample size for the study.

Differences revealed via large samples are more likely to be genuine than differences revealed using small sample size.

A counselor decides to increase the sample size in her experiment. This will ____

reduce Type 1 and Type II errors.
Raising the size of a sample helps lower the risk of chance/error factors.

If a researcher changes the significance level from .05 to .001, then

alpha errors decrease, but beta errors increase.

If t value is less than the t value in a statistical table

ACCEPT the null hypothesis
(computations must exceed the number cited in the table in order to reject null)

Analysis of variance (ANOVA)

Used for more than 2 sample groups, yields and F statistic

If F value exceeds the critical F value in a statistical table

REJECT the null hypothesis

Analysis of covariance (ANCOVA)

tests 2 or more groups while controlling for extraneous variables that are called covariates

Kruskal-Wallis

used instead of the ANOVA when data is nonparametric

Wilcoxon

signed rank test used in place of the t test when data are nonparametric and you wish to test whether 2 correlated means differ significantly
(memory: 'co' to remind you of correlated)

Mann-Whitney U-test

determines whether 2 uncorrelated means differe significantly when data are nonparmetric
(memory: the 'u' reminds you of 'uncorrelated')

Spearman correlation (also known as Kendall's tau)

used in place of the Pearson r when parametric assumptions cannot be utilized

Chi-square nonparametric test

examines whether obtained frequencies differ significantly from expected frequencies

A 1-way analysis of variance (ANOVA) is used for testing ONE IV.

A two-way analysis of variance (ANOVA) is used to test TWO IVs.
(Two IVs requires a two-way ANOVA, 3 IVs requires a 3-way ANOVA, etc.)

Multi-variate analysis of variance (MANOVA)

When a study has moe than one dependent variable.

Some researchers refer to the level of significance as where one _____ the ____, or as the ______ point.

draws, line, cutoff
(If a researcher sets the level of significance at .50, then the odds would be 50/50 that the results were due to pure chance.)

When a researcher uses ______, then there is no direct manipulation of the IV.

correlation

Correlation coefficient

A statistic that indicates the degree or magnitude of relationship between two variables, often abbreviated using the lower-case 'r'.
(Makes a statement regarding the association of two variables and how a change in one is related to the change in the other.)

Correlations range from 0.00 (no relationship) to 1.0 or -1.0 (perfect relationship).

A positive relationship is not stronger than a negative relationship of the same numerical value.
(i.e., .70 and -.70 are the same significance)

A positive correction

Evident when both variables change in the same direction (imagine a graphical representation of scores)

A negative correlation

Evident wen the variables are inversely associated (one goes up and the other goes down).

biserial correlation

One variable is continuous while the other is dichotomous.

Correlation is concerned with covariation.

Correlation does not imply causation!

Covary positively

When two variables vary together.

Covary negatively

When one variable increases while the other decreases.

Bivariate

When correlational data describes the nature of two variables .

Multivariate

When more than two variables are under scrutiny.

Another name for N=1

intensive experimental design (pioneered by Freud), also known as a case study
(N= the number of people being studied)

Idiographic studies

single case investigations
(Case studies are often misleading because the results are not necessarily generalizable.)

Single-blind study

The subject does not know whether they are in the control group, but the researcher does.
(helps eliminate 'demand characteristics')

Double-blind study

Neither the subject nor the researcher knows of the person is in the control group.
(Researcher is sometimes unaware of the null hypothesis too.)

Experimenter effects

Things that can flaw an experiment because the researcher unconsciously communicates intent or expectations to the subjects.

ABA model of research (also known as 'withdrawal design')

A - baseline secured
B - intervention implemented
A - outcome is examined via a new baseline

Multiple-baseline design

When a researcher employs more than 1 target behavior.

Experimental is to cause and effect as correlational is to _____ of _______.

degree of relationship.

Correlation coefficient is a

descriptive statistic which indicates the degree of 'linear relationship' between two variables.

Pearson Product-Moment correlation r

used for interval or ratio data.
(memory - Pearson r uses I and R for Info and Referral)

Spearman rho correlation

used for ordinal data

Pearson r is the most common _______ ________.

correlation coefficient.

Normal curve

theoretical notion often referred to as 'bell-shaped curve'. Bell is symmetrical.

68-95-99.7 rule (empirical rule)

Within a normal distribution, 68% of scores will fall within +/- 1 standard deviation (SD) of the mean; 95% within 2 SDs of the mean; and 99.7% within 3 SDs of the mean.
(Almost all scores will fall between 3 SDs of the mean.)

Most common measures of central tendency

Mode, Median, Mean

Mean

average of all scores

Median

middle scores in a distribution of scores
(The middle scores when data are arranged from highest to lowest.)

Mode

most frequently occurring scores and the least important measure of central tendency.
(The highest or maximum point of concentration on a curve.)

Bimodal distribution

has two modes
(graphically looks like a camel with 2 humps)

Multimodal distribution

Two or more peaks in a distribution curve

Unimodal

One peak in a distribution curve

Range

distance between the largest and the smallest scores.

The larger the range, the greater the dispersion or spread of scores from the mean.

The most useful measure of central tendency is the MEAN (i.e., average).
In skewed distributions, the median is the best choice.

The mean is misleading when ___ and ___

the distribution is skewed, there are extreme scores.

Skewed distribution

When a distribution of scores is not distributed normally (and symmetrically).

Factorial design

Used to ferret out the effects of more than one IV.

Factorial experiment

Several experimental variables are investigated and interactions can be noted.
Factorial designs include 2 or more IVs.

Solomon 4 group design

Researcher uses 2 control groups - only one experimental group and one control group are PRE-tested. The other control group and experimental group are merely post-tested. (Lets the researcher known if results are influenced by testing.)

Regardless of the shape, the ____ will always be the high point when a distribution is displayed graphically.

mode

In a graph, the tail indicates whether a distribution of scores is positively or negatively skewed.

(Tail to left - negatively skewed. Tail to right - positively skewed.)

The benefit of standard scores such as percentiles, t-scores, z-scores,stanines, or standard deviations over raw scores, is that a standard score allows you to analyze the data in relation to the properties of the normal bell shaped curve.*

...

Histogram

A distribution with class intervals graphically displayed on a bar graph.

X axis (also called 'abscissa')

The horizontal line drawn under a frequency distribution graph.
(horizontal axis plots the independent variable [IV])

Gaussian curve is said to be ______ because the peak is in the middle.

mesokurtic

Y axis (also known as 'ordinate')

Used to plot frequency of the DVs, plotting the experimental data.
(memory: Letter 'Y' is vertical like the line it represents in a graph)

If a distribution is bimodal, there is a good chance that the researcher is working with ____ distinct ______.

2, populations.

Reliable experiement

If an experiment can be replicated by others with almost identical findings.

The RANGE is the simplest way to measure the spread of scores.

The RANGE is usually calculated by subtracting the lowest score from the highest scores (i.e., 93-33=60.)
- If 'inclusive range' is specified on exam, then use the formula but add '1' to the final value after subtraction of the range.
-generally increases with sample size

Sociogram is to a counseling group as a scattergram is to _____.

a correlation coefficient.

Scattergram (also known as scatterplot)

Pictorial diagram or graph of two variables being correlated.

John Henry Effect (also known as 'compensatory rivalry of a comparison group')

threat to internal validity when subjects strive to prove an experimental treatment that might threaten their livelihood isn't really effective. (i.e., sabotage)

Resentful Demoralization of the Comparison Group (also called compensatory equalization)

Threat to validity in which comparison group lowers their performance or behaves inept in an attempt to make the experimental group look better than they should.
(Noted if the comparison group deteriorates throughout the experiment while the experimental group does not.)

Variance

A measure of dispersion of scores around some measure of central tendency; it is also the standard deviation squared.

Statistically speaking, 68.26% of scores fall within + or - one SD of the mean.

Statistically speaking, 95.74% of scores fall within + or - 2 SD of the mean.

Statistically speaking, 99.74% of scores fall within + or - 3 SD of the mean.

The greater the standard deviation of scores, the greater the spread of a plotted graph.

Z-score (often called standard score)

same as a standard deviation - the most elementary of standard score.
(memory: Z score is simply SD)
A Z-score of +1 or 1 SD would include about 34% of the cases in a normal population.

T-score (often called transformed score)

Mean of 50 with each SD of 10 [different from a Z-score]
(i.e., a Z score of -1.0 would be a T score of 40. A Z-score of -1.5 would be a T-score of 35, etc.)
- Not mathematically threatening because never expressed as a negative number.

Z-scores (aka standard scores) are the same as standard deviations, thus a Z-score of -2.5 means

2.5 SD below the mean

College Entrance Examination Board (also known as Educational Testing Services [ETS]) scores range from 200 to 800 with a mean of 500.

...

An IQ score on an IQ test which has 3 SDs above the mean would be near the ____ level.

genius

Kurtosis

Peakedness of a frequency distribution.

Platykurtic distribution

Flatter and more spread out than a normal curve.
(Memory: 'Plat' sounds like 'flat')

Leptokurtic distribution

Distribution curve is very tall, thin and peaked.
(Memory: Leptokurtic leaps tall buildings in a single bound.)

Stanine scores (contraction of 'standard' and 'nine')

Divides the distribution into 9 equal intervals with stanine 1 as the lowest 9th and 9 as the highest 9th - in this, 5 is the mean.

Four basic measurement scales (by. S. S. Steven)

Nominal - simplest type, strictly qualitative NOT quantitative
Ordinal
Interval
Ratio

Nominal scale

most basic, does not provide measurable info, merely classifies names, labels, or identifies by group, has NO TRUE ZERO point and DOES NOT INDICATE ORDER.
(i.e., street address, telephone #, gender, brand or therapy; adding/subtracting nominal categories is meaningless)

Ordinal scale (2nd level of measurement)

Rank-orders variables, though distance between the variables may not be equal - Provides relative placement or standing but does not delineate absolute differences
(adding/subtracting is no-no)
(Memory: 'ordinal' sounds like 'order')

See More

Please allow access to your computer’s microphone to use Voice Recording.

Having trouble? Click here for help.

We can’t access your microphone!

Click the icon above to update your browser permissions above and try again

Example:

Reload the page to try again!

Reload

Press Cmd-0 to reset your zoom

Press Ctrl-0 to reset your zoom

It looks like your browser might be zoomed in or out. Your browser needs to be zoomed to a normal size to record audio.

Please upgrade Flash or install Chrome
to use Voice Recording.

For more help, see our troubleshooting page.

Your microphone is muted

For help fixing this issue, see this FAQ.

Star this term

You can study starred terms together

NEW! Voice Recording

Create Set