Advertisement Upgrade to remove ads

Reliability

The results are replicable. Describes stability or consistency of test performance. When administered properly, a test gives consistent results on repeated administrations or with different interpreters judging the same administration.

Types of Reliability

Interjudge
Test-retest
Alternate form
Split-half

Interjudge Reliability

Refers to agreement of two independent judges on the occurrence and type of responses performed by a client.
It is preferable for standardized tests to have interjudge reliability coeffients of .90 or above to show good agreement among examiners

Factors that affect interjudge reliability

1) incomplete or ambiguous definitions
2) training in the observation, coding, and interpretation of data
3) practice (with the test/protocol/procedures)
4) response complexity
5) live scoring versus tape analysis

2)Test-Retest Reliability

should be done within an interval of 2 weeks; should yield the same scores from the same individual

Alternate Form Reliability

two versions of the same test (e.g., forms A & B) should measure items in an equivalent manner

Split-Half Reliability

refers to internal consistency of the items on a test. Both halves of a test (e.g. every other item) should agree in terms of scores obtained.

Validity

refers to the extent to which a test measures what it sets out measure. Validity is highly related to the purpose for which a test is used; a particular test can be valid for one purpose and invalid for another.

Construct validity

the degree to which a test measures the theoretical construct it is intended to measure.

Content Validity

derived by a careful examination of the content of a test by an expert panel of judges. Established by examining three factors:
a. appropriateness of the types of items included
b. completeness of the item sample
c. the way in which the items assess content

Criterion-related validity

(two types)
a. concurrent validity - measure of how an individual's current score on one instrument can be used to make an estimate of his or her current score on some other criterion measure, typically another test in a related area
b. predictive validity - measure of how an individual's current score on one instrument can estimate scoring on a criterion measure taken at a later time

Factors /assumptions affecting validity:

a. test-taker has ability to respond in manner which required by the test
b. test-taker has had exposure to the items presented
c. administration has been properly followed
d. norms are accurately established

Correlation

A number (between -1.0 and +1.0) that describes the strength and direction of a relationship between two variables. Note: if two scores correlate, it does not mean that one score causes a change in another score (i.e., correlation does not imply causation).

Types of Correlation

1) positive correlation indicates that higher scores on one variable tend to be paired with higher scores on the other variable, and lower scores on one variable tend to be paired with lower scores on the other (e.g., weight and amount of food eaten has positive correlation)
2) negative correlation indicates that high scores on one variable tend to be paired with low scores on the other variable (e.g. weight and amount of exercise has negative correlation)

Standard Error of Measurement (SEM)

a statistic used to determine whether the observed score (raw score) of a client is reasonably close to his or her possible true score (ideal score that test taker would have earned if there were not errors in the measuring instrument; these errors may be associated with test scores, test items, and test times).
1) The SEM can be calculated, but most tests provide tables from which to arrive at the information. The SEM calculation involves:
a. estimate of the test's reliability
b. mean and SD of scores obtained by the normative sample to which the test taker's score is to be compared
c. the test taker's observed score

Confidence Intervals

the range in which a measurement (score) falls corresponding to a given probability.
1) tests usually provide a table for 90% or 95%
probability (confidence level)
2) Example: Standard score = 92
At 90% confidence level, SEM = +/- 4
Confidence interval = 88-96

Normed tests

Primary purpose is to compare individual client's score to the average score of the norming group. Using this comparison, the clinician may determine: a) if the client has a problem, b) if the problem is clinically significant, c) whether the problem warrants intervention.
 Not the same as a standardized test. Many norm-referenced tests are standardized, however, a test can be standardized without being norm-referenced.

Criterion Referenced Tests

a form of assessment in which the examiner selects target behaviors to be assessed and uses stimulus materials that are effective for and individualized to the particular client. Involves a qualitative evaluation of what is acceptable and what is unacceptable performance.

Normed vs Criterion Referenced Tests

The difference between norm-based and criterion referenced testing is that in norm-based, the client's performance is evaluated against age-based norms. The purpose of norm-based is to rank individuals. In criterion referenced testing, results are not compared against norms. Rather, skills are defined and individual performance is emphasized.

Advantages of Criterion Referenced Tests

 May prove to be especially helpful when the norms of standardized tests do not apply to a particular client, or when the available standardized tests do not assess specific client skills of interest to the clinician.
 May assess aspects or skills that standardized tests do not include, and may assess those skills in greater depth than standardized tests do
 Also helpful in assessing a client's progress in treatment.

Derived Scores

a raw score that has been converted so that it holds normative meaning on it's own.
Raw scores must be converted in order to hold meaning within the comparison group.



Distributions: yield measures of the client's performance compared to the performance of the norming sample. Two measures of distribution are standard deviation and mean.

most common types of derived scores

age equivalents, percentiles, standard scores.

Percentile Rank

converted scores that show the percentage of subjects who scored at or below a specific raw score. Percentile ranks use percentile points to express a client's score relative to the norming sample. 50th percentile = mean and median.

Age equivalency

scores showing the chornological age for which a raw score is the mean or average score in the standardized sample.

Distributions

yield measures of the client's performance compared to the performance of the norming sample. Two measures of distribution are standard deviation and mean.

Tests vs. Assessment

Tests - often standardized/normed or criterion referenced measures of a client's abilities, compare individual performance to that of a large and similar group, often yield percentile rank and standard deviation or ratings
Assessment - evaluation of client's abilities through various measures including:
• preassessment history: family, medical, educational, developmental, social, psychological
• interview
• informal assessment & observation of strengths/weaknesses
• language sample
• formal TESTS - just one small part of the overall assessment!
• dynamic assessment (test-teach-retest)
• oral-facial exam
• hearing eval

Dynamic Assessment

-TEST-TEACH-RETEST, a form of authentic assessment, looks at client's ability to learn with help, uses time in a diagnostic session in order improve the clinician's understanding of the client's potential performance during therapy, clients are not penalized for lack of exposure. The direction for treatment is often determined through dynamic assessment.
Dynamic assessment relies heavily on Vygotsky's Zone of Proximal Development, which defines a range of performance that a child can produce with assistance from adults or peers.

static vs dynamic assessment

Static Assessment = snapshot of performance during a given moment in time
Dynamic Assessment = how performance improves with help over time

Sentence memory tests (2)

• Clinical Evaluation of Language Fundamentals-4 (CELF-4)- Recalling Sentences Subtest
• Boston Aphasia Battery

Receptive & Expressive vocabulary (7)

1. Peabody Picture Vocabulary Test (PPVT-4) (Receptive)
2. Receptive One Word Picture Vocabulary Test (ROWPVT) (Receptive)
3. Expressive One Word Picture Vocabulary Test (EOWPVT) (Expressive)
4. Expressive Vocabulary Test (EVT) (Expressive)
5. Test of Word Finding-2 (TWF-2) (Expressive)
6. Test of Adolescent Word Finding (TAWF) (Expressive)
7. CELF-4 Expressive Vocabulary Subtest (Expressive)

Following directions (3)

1. Preschool Language Scale-4 (PLS-4)
2. Fullerton Test of Adolescent Language- Oral Commands Subtest
3. CELF-4 Concepts and Following Directions Subtest

Understanding paragraphs

1. CELF-4 Understanding Spoken Paragraphs Subtest
2. Listening Comprehension Test-2

phonemic & phonological awareness

• Phonological Awareness Test-2 (PAT-2)
• Comprehensive Test of Phonological Processing (CTOPP)
• CELF-4 Phonological Awareness Subtest
• Test of Language Development Primary-3 (TOLD-3)
• Hodson Test of Phonological Patterns-3

Rapid automatic naming (RAN)

• CELF-4 Rapid Automatic Naming Subtest
• Test of Rapid Automatic Naming

Comprehension of specific word structures

• CELF-4 Word Structure Subtest
• Comprehensive Assessment of Spoken Language (CASL)- Grammatical Judgment Subtest
• The Structures Photographic Expressive Language Test-3 (SPELT-3)
• PLS-4

pragmatics

• CASL Pragmatic Judgment Subtest
• Wesby Play Scale (Symbolic Play Scale Checklist)
• Test of Pragmatic Language (TOPL)

Preschool assessments (8)

1. CASL (3-21); comprehensive
Speech: expressive and receptive
Areas: Morphology, syntax, semantics, pragmatics
2. CELF-P2 (3-6); comprehensive
Speech: expressive and receptive
Areas: Phonology, morphology, syntax, semantics, pragmatics
Extra: phonological awareness
3. EOWPVT-3 (2-18); vocabulary
Speech: expressive
Areas: Semantics
4. OWLS (3-21); comprehensive
Speech: expressive and receptive
Areas: Morphology, syntax, semantics, pragmatics
Extra: written section
5. PLS-4 (0-6); comprensive
Speech: expressive and receptive
Areas: Phonology, morphology, syntax, semantics, pragmatics, nonverbal comm.
Extra: attention, cognitive, supplemental language sample, interview, artic screening
6. PPVT-3 (2.5-90): vocabulary
Speech: receptive
Areas: Semantics
7. ROWPVT-3 (2-18); vocabulary
Speech: receptive
Areas: Semantics
8. TELD-3 (2-7); comprehensive
Speech: expressive and receptive
Areas: Morphology, syntax, semantics, pragmatics

Elementary school age assessments (10)

1. CASL (see preschool)
2. CELF-2 (see preschool)
3. EOWPVT-3 (see preschool)
4. ITPA-3 - Illinois Test of Psycholinguistic Abilities; (5-12); comprehensive
Speech: expressive and receptive
Areas: Phonology, morphology, syntax, semantics, pragmatics
Extra: phonemic awareness and literacy
5. OWLS (see preschool)
6. PPVT-3 (see preschool)
7. ROWPVT-3 (see preschool)
8. TELD-3 (see preschool)
9. CELF-4
Type: Language
Age: 5-21 years
Speech: expressive and receptive
Areas: language structure, language content, memory
10. LPT-3: Language Processing Test (5-11)
Type: Language
Areas: labeling, functions, associations, categorizations, similarities, differences, multiple meanings

Secondary school age assessments (7)

1. CASL (see preschool)
2. CELF-4 (see preschool)
3. EOWPVT-3 (see preschool)
4. OWLS (see preschool)
5. PPVT-3 (see preschool)
6. ROWPVT-3 (see preschool)
7. TOAL-3 - Test of Adolescent and Adult Language
Type: adolescent/adult language (12-24)
Speech: expressive and receptive
Areas: Syntax, semantics
Extras: writing subtest

Aphasia assessments (7)

1. BDAE
Used to try to classify aphasia into types.
2. WAB - Western Aphasia Battery
Used to try to classify aphasia into types.
3. MTDDA - Minnesota Test for Differential Diagnosis of Aphasia
4. NCCEA - Neurosensory Center Comprehensive
5. BAT - Bilingual Aphasia Test
Comprehensive test that evaluates skills in 40 languages.
6. PICA - Porch Index of Communicative Ability
Downside - need training to administer test
7. Aphasia Diagnostic Profile

TBI assessments (4)

1. Glasgow Coma Scale
Helps make an initial assessment of eye opening, motor responses (e.g. flexing the body in response to pain), and verbal responses.

2. Brief Test of Head Injury
Brief screening test to evaluate adults with severe head injury
Assesses orientation, following commands, and other relevant behaviors of patient with head injury
3. Galveston Orientation and Amnesia Test
Assesses amnesia, orientation, and memory
Disability Rating Scale
Assess changes in patients with head injury
Assesses eye opening, verbal responses, motor responses, feeding, toileting etc.
4. Rancho Los Amigos Scale of Cognitive Levels
Assesses cognition and behavior at the levels of no response, generalized, response, localized response, confused-agitated, confused-appropriate, automatic-appropriate, and purposeful-appropriate.

Language sample length

minimum of 50-100 distinct utterances
better is 200 or more different utterances

recommendations for obtaining reliable & valid speech/language sample

1. length
2. unobtrusive; minimize interruptions/distractions
3. wait for client to talk/don't talk to fill in
4. vary subject matter of sample
5. seek out multiple environments
6. alter context (narration, conversation, responses to pictures)
7. ask open ended questions (tell me about..) & avoid yes/no questions

transcription of lang. sample

1. transcribe entire sample
2. indicate speaker for all utterances
3. use phonetic symbols only for unintelligible words
4. keep punctuation to a minimum
5. indicate end of utterance with a slash
6. number client's utterances
7. transcribe utterances consecutively from the recording.

analysis of language sample: form

• Does the child primarily use single words, phrases, or sentences?
• Are the sentences of the subject-verb-object for exclusively?
• Are there mature negative, interrogatives, and passive sentences?
• Does the child elaborate the noun or verb phrase?
• Is there evidence of embedding and conjoining?

analysis of language sample: semantics

• Does the child respond appropriately to the various question forms (what, where, who, when, why, how)?
• Does the child confuse words for different semantic classes

analysis of language sample: language use

• Does the child display a range of functions such as asking for information, help, and objects; replying, making statements, providing information?
• Does the child take conversational turns?
• Does the child signal the status of the communication and make repairs?

analysis of language sample: rate

• Is the rate of speech too slow or too fast?
• Are there notable pauses between turns or between utterances?
• Are there frequent word substitutions?
• Does the child use fillers frequently or pause before producing certain words?

analysis of language sample: sequencing

• Does the child relate events in a sequential fashion based on the order of occurrence?
• Can the child discuss the recent past or recount stories?

MLU up to age 5

MLU was created by Brown; Generally, a normal child's age (up to age 5) will correspond closely to his/her MLU. For example, a normally developing 4 year, 3-month-old child will often exhibit an MLU of approximately 4.3 (plus or minus a few tenths).

What to count as ONE morpheme

1. Grammatical morphemes that are whole words (nouns, verbs, articles, prepositions)
2. Auxiliaries (is, will, have, must, would)
3. Diminutives (mommy, doggy)
4. Uninflected lexical morphemes (run, fall)
5. Inflections (possessives -'s, third person singular -s, regular past tense -ed, progressive -ing)
6. Irregular past tense
7. Contractions (I'll, can't)
8. Plurals that do not occur in singular form (us, clothes)
9. Gerunds and participles that are not part of a verb phrase (She was tired. Swimming is fun.)
10. Stuttered words (My, my, my)
11. Compound words (birthday, see-saw, belly-button)
12. Single words or phrases (Hi. No. Yeah.)
13. Proper nouns
14. Ritualized reduplications (quack-quack, choo-choo)

What to count as more than one morpheme

1. Inflected forms: regular and irregular plural nouns, possessive nouns, third person singular verbs, present participle and past participle when part of the verb phrase, regular past tense verbs, reflexive pronouns, comparative and superlative adverbs, adjectives.
2. Contractions only when one or both of the constituent parts occurs separately elsewhere in the sample (It's if it or is occurs elsewhere).
3. Repeated words only if the word is produced for emphasis (No, no, no! is counted as 3 morphemes)

What no NOT count as a morpheme

1. Partial utterances
2. Imitations that immediately follow a model utterance
3. Elliptical answers to questions
4. Unintelligible utterances
5. Rote passages (nursery rhymes, songs)
6. False starts or reformations
7. Noises, unless they are meaningfully integrated into the utterance
8. Fillers
9. Counting, sequences, or other enumerations

Please allow access to your computer’s microphone to use Voice Recording.

Having trouble? Click here for help.

We can’t access your microphone!

Click the icon above to update your browser permissions above and try again

Example:

Reload the page to try again!

Reload

Press Cmd-0 to reset your zoom

Press Ctrl-0 to reset your zoom

It looks like your browser might be zoomed in or out. Your browser needs to be zoomed to a normal size to record audio.

Please upgrade Flash or install Chrome
to use Voice Recording.

For more help, see our troubleshooting page.

Your microphone is muted

For help fixing this issue, see this FAQ.

Star this term

You can study starred terms together

NEW! Voice Recording

Create Set