60 terms

Reliability and validity should be considered...

before you start analyzing data

This explores the question "how do I know that the test, scale, instrument, etc. works every time I use it?"

reliability

This explores the question "how do I know that the test, scale, instrument, etc. measures what it is supposed to?"

validity

If data are not reliable or not valid, the results of any test or hypothesis

are not valid

Reliability and validity specifically relate to

the quality of the data themselves

The outcome variable in an analysis is called...

the dependent variable

The predictor variable in an analysis is called

the independent variable

This is the least precise level of measurement.

nominal

This is the most precise level of measurement.

ratio

This is the level of measurement where outcomes are based on some underlying continuum where it is possible to speak about how much more a higher performance is than a lower one:

interval level of measurement

This is the level of measurement where outcomes can only be placed into unranked categories

nominal level of measurement

This is the level of measurement where outcomes can be rank ordered.

ordinal level of measurement

This is the level of measurement where outcomes are based on some underlying continuum that also contains a true or absolute zero.

ratio level of measurement

Only this level of measurement has a true zero.

ratio

Moving from the nominal to the ratio levels of measurement, your data becomes...

more precise and detailed

Which of the following is concerned with monitoring estimates of present performance and predictions of future performance?

predictive validity

What type of validity is concerned with the adequate representation of test items?

content validity

If you want to know that a test measures some underlying psychological construct, what type of validity evidence would you want to collect?

constrict validity

If you correlate scores from your test with some other valid measure that assesses the same set of abilities, what type of validity evidence are your collecting?

criterion validity

If I can say that my weekly statistics quiz fairly assesses the material covered, what source of validity evidence should I have collected?

content validity

What is the key to establishing Criterion validity?

quality of criterion

Which of the following is NOT a measure of validity?

parallel forms validity

This type of validity is established by consultation with an expert on the topic focused upon by your instrument.

content validity

This type of validity is determined by determining the association between the test scores and some specified present or future criterion.

criterion validity

This type of validity is based on the judgment of how well a test reflects an underlying idea

construct validity

High school class rank is highly correlated with college GPA. This is an example of what type of validity

predictive validity

The score that you would actually record is the...

observed score

A measure of how stable a test is over time is an example of which of the following types of reliability?

test-re test

The correlation between scores from time 1 and time 2 is called:

test- re test reliability

The correlation between two forms of the same test is called:

parallel forms reliability

Cronbach'a Alpha is used to calculate the following

internal consistency reliaility

The number of agreements between your two raters divided by the total number of possible agreements is the way to calculate:

interrater reliability

This is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the present.

concurrent criterion validity

This is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the future.

predictive validity

This is the score that you would receive if the test contained no error.

true score

The actual or measured score is called:

the observed score

This is the difference between the observed and true scores:

the error score

Which coefficient reflects the occurrence of perfect reliability?

1.00

Two trained professionals observe the behavior of children in a classroom. They each rate observed behaviors using the same form and the number of items that were rated the same is calculated. This is an example of which type of reliability?

inter ratee

If the measures associated with a test are said to be consistent, you might conclude that the measure is ___________.

reliable

Which of the following is NOT a measure of reliability?

multiple forms reliability

When we calculate reliability, we know the observed score. What are the two unknown components of the reliability equation?

true and error scores

Inter-rater reliability measures consistency _________________.

from rater to rater

Which of the following is the Greek letter associated with Cronbach's Alpha, sometimes known as coefficient alpha?

A.

Test-Retest reliability is an example of which of the following?

measure of reliability over time

Parallel forms reliability is an example of which of the following?

measure of equivalence

Inter-rater reliability is an example of which of the following?

measure of agreement

What would the inter-rater reliability be for a 50 item measure in which the number of agreements between Rater 1 and Rater 2 was 45?

0.90

What are the four levels of measurement?

nominal, ordinal, interval, ratio

The maximum level of validity possible is equal to:

the square of the reliability coefficient

Which level of measurement is concerned only with the characteristics of an outcome that fits into a category?

nominal level of measurement

If you are interested in describing the order of variables along a continuum, what level of measurement would you use?

ordinal level of measurement

If the underlying continuum we are measuring assumed equal intervals, at what level of measurement is the associated variable being measured?

interval level of measurement

Which of the following levels of measurement provides the most information about a variable?

ratio level of measurement

Who is responsible for creating the formula for coefficient alpha?

Lee J. Cronbach

What is the assignment of values to outcomes known as?

measurement

What is the difference between the true score and the observed score?

observed score: what you actually get on the test

true score: the true , 100% accurate reflection of what you really know.

page 106

true score: the true , 100% accurate reflection of what you really know.

page 106

The outcome or predictive variable in an analysis is called the dependent variable. T/F

True

The treatment or predictor variable is called the independent variable. T/F

True

The less precise levels of measurement contain all the qualities of the scales above them. T/F

False