Upgrade to remove ads
Only $35.99/year

Psych 309 - Exam 2

Terms in this set (98)

Define and be able to apply the broad definition of validity.
Validity can be defined as the agreement between a test score or measure and the quality it is believed to measure.
The degree to which a test actually measures what it purports to measure.
Committee definition: evidence for interferences that can be made about a test score.
What are the three main types of validity evidence?
Construct-related, Criterion-related, Content-related
What prerequisites exist for validity?
Reliability is necessary (but NOT sufficient) for validity
Define Face Validity. How does it differ from other aspects of validity?
The mere appearance that a measure has validity.
It's not really validity because has no evidence to support conclusions drawn from test scores.
One may or may not want a test to have high face validity b/c it may increase the risk of measurement error, and it may also help motivate b/c the test appears relevant to the subject.
Define Content Validity: How is it measured?
The degree to which a test or measure adequately represents the conceptual domain it is designed to measure. Evidence that the content of a test represents the entire breadth of the construct it is designed to cover.
- Is the difficulty consistent with the expected mastery level? Are terms meaningful & familiar to test-takers?
- Often determined by expert judgment.
How do construct underrepresentation and construct-irrelevant variance relate to content validity?
Construct underrepresentation: failure to capture important components of a construct (ex: measure of depression that omits sleep symptoms).
Construct-irrelevant variance: Scores are influenced by factors irrelevant to the construct (ex: final exam grade influenced by anxiety, poor sleep, disgusting person next to you, etc.).
What is the content validity ratio and how is it calculated?
CVR uses a panel of judges who rate items on a 3-point scale (essential, useful but not essential, not necessary).
- The more panelists consider the item essential, the higher the content validity.
What is a criterion? What is Criterion-related Validity?
Criterion: the standard against which a test is compared
Criterion-related validity: How well a test score corresponds with a particular criterion, the test or measure acts as a stand-in for the measure we are interested in.
Name and define the three subtypes of criterion related validity. Be able to give examples.
Predictive validity: the accuracy with which test scores predict a criterion obtained at a later time (ex. SAT & is a predictor and college GPA is the criterion)
Concurrent Validity: the degree to which the test scores are related to the criterion and can be measured at the same time (ex: job samples - screening applicatns)
Postdictive Validity- the accuracy with which a test score predicts a previously obtained criterion (ex: a test for adult memories of childhood events - probably wouldn't work)
What is the validity coefficient? What is the meaning of a squared validity coefficient?
It is the correlation between a test & a criterion It is R.
Squared validity coefficient: The percentage of variation in the criterion we can expect to know in advance because of our knowledge of the test scores.
- If r is low, it can sometimes be useful like when it has to do with mortality, increasing earnings, or saving taxpayers money.
What is incremental validity?
Incremental validity: The unique information gained through using the test. A form of evidence is how much info a test contributes beyond some simpler method for making the same prediction.
What is a construct? What is Construct-related Validity?
Construct: an idea or theory
Construct-related validity: the degree to which a set of measurement operations measures hypothesized constructs, aka fidelity... It is inextricably linked to theory - does a test behave as the theory predicts the construct will behave?
What are the two types of evidence in construct validity?
Convergent: a demonstration of similarity, when a measure correlates well with other tests believed to measure the same construct, convergent evidence for validity is obtained.
Discriminant: a demonstration of uniqueness, to demonstrate discriminant evidence for validity, a test should have low correlations with measures of unrelated constructs, or evidence for what the test does not measure.
What is the relationship between reliability and validity?
Reliability is necessary but not sufficient for validity.
Which two types of validity are logistical and not statistical? Why?
Content Validity & Face Validity... Requires good logic, intuitive skills, and perseverance.
Which type of validity has been referred to as "the mother of all validities", or "the big daddy" and why?
Construct Validity - it subsumes all activities used in other types of validity.
Be able to identify and/or give examples of the different types of validity.
...
What role does the relationship between examiner and test taker play?
The way an interviewer responds affects the content of responses in interview studies.
Reinforcement, inconsistent feedback, rewards, stereotype threat, expectancy effects (Rosenthal Effects)
Test scores can be affected by examiner behavior & relationships.
What is the relationship between test examiner race and intelligence scores?
Effects increase when examiners have more discretion about the test use
• Paraprofessionals (not psychologists) showed a race effect with whites & African Amer.
Why would examiner race effects be smaller on IQ tests than on other psychological tests?
When it is highly structured, such as exact words to be used and standardization are used, ethnicity doesn't really play a roll.
What is the standard for test takers who are fluent in two languages?
Give the test in their "best" language. Use interpreters with caution.
Define expectancy effects and know whose name is associated with these effects
Rosenthal Effects: when data is affected by what an experimenter expects to find.
A review of many studies showed that expectancy effects exist in _______ situations
some but not all
What types of situations might require the examiner to deviate from standardized testing procedures
Certain groups, such as the blind, deaf, and old, etc.
What advantages and disadvantages were mentioned in lecture and the text regarding computer administered tests?
Pros: excellent standardization, can give individually tailored sequential administration, precision of timed responses, releases human testers for other duties, controls bias, allows test taker to not be rushed, objective & cost effective, & people are more honest behind a screen.
Cons: can't replace clinical judgment, can cause harm if misinterpreted, could have poor internet or old computer, also some people are just slower on computers.
What subject variables can impact testing?
Any independent variable that can affect testing results. Ex: hungry, tired, motivation, illness, test anxiety
What are the three major problems in behavioral observation studies?
Reactivity, drift, and expectancies
What is reactivity? How does performance change when people are not being observed or checked?
Reactivity is the "reaction" to being checked. Reliability and accuracy are highest when rater is checked.
What is drift? How does it relate to contrast effect? How can drift be addressed?
Drift: adopting idiosyncratic definitions of behavior, drifting from the standardization training
Contrast effect: the tendency to rate the same behavior differently when observations are repeated in the same context.
How well do people do in detecting deception/lies?
Very poor. Basically just by chance. Validity coefficient - r = .13 (WAY low!)
What is the Halo Effect? Be able to think of examples where the Halo Effect applies.
Ascribing attributes to someone on the basis of something other than the trait. Ex: people who are attractive are perceived as being kind, such as certain politicians.
What does a good interviewer know how to do?
A good interviewer knows how to provide a relaxed and safe atmosphere through social facilitation
a. Remain in control and set the tone
b. Either you're good at it or you're not
How are interviews similar to tests?
Gather data, reliability and validity, individual vs. group, many tests cannot be used without an interview and vice versa
Define interpersonal influence and interpersonal attraction. How are they interrelated?
Interpersonal influence: The degree to which one person can influence another
Interpersonal attraction: The degree to which people share a feeling of understanding, mutual respect, and similarity...
Interrelated: You need to establish rapport, so it doesn't matter so much about what you say but how you feel! Need to be warm, open, concerned, involved, committed, & interested.
Define and note what types of statements should be avoided to elicit as much information as possible?
Evaluative, judgmental, and probing statements; hostile responses, false reassurance
What is the main goal in interviewing?
To keep the interaction flowing - one speaks, other responds, minimal effort to maintain convo, face-to-face contact with interest, requires flexibility while following general guidelines
Define transitional phrase. If it fails, what responses should be used to continue the theme? Define.
A response should be made that continues the "theme" using one of the following:
Verbatim playback - repeat the exact words
Paraphrasing & restatement - repeats the response using diff. words
Summarizing - pulls together the meaning of several responses
Clarification response - clarifies the response
Empathy & understanding - communicates interviewer knows how they feel
When should direct questions be used in an interview?
Maybe with children, nonverbal, intellectually limited, uncooperative people, time is limited, data can't be obtained any other way.
What are the advantages and disadvantages of using structured clinical interviews?
Advantages: Same questions in same order, specified rules so all handled the same, offers reliability
Disadvantages: Sacrifices flexibility, requires cooperation, relies exclusively on the respondent making the assumptions questionable
What is the purpose of a mental status examination? What areas are typically covered?
Purpose: used to evaluate and screen psychosis, brain damage, and other major psychiatric and neurological difficulties. Requires an understanding of psychiatric disorders & brain damage.
Covers: a person's appearance, attitude, behavior, & emotion.
Define general standoutishness. How does appearance play a role?
When one trait or quality stands out above any other one & causes you to remember them based off of the one characteristic. Ex: Always remembering a girl with a big mole b/c of her mole.
How much higher is interview reliability for structured interviews?
2x higher.
What is a major criticism of structured interviews?
Not as flexible, based on self report data that you can deviate from - not broad data.
What is social facilitation? Be prepared for examples.
•Social facilitation: we tend to act like the models around us
o Very important in interviewing
o If you project a mood, then the interviewee responds in kind.
What is the largest source of error in interviews?
Judgment! - first impressions, generalizing from small "samples" of behavior, etc.
What were the three independent research traditions identified by Taylor to study human intelligence?
Psychometric Approach (examines elemental structure of a test), Information-Processing Approach (examines processes that underlie how we learn & solve problems), Cognitive Approach (focuses on how humans adapt to real world demands)
Through what 3 facilities did Binet believe intelligence expressed itself? What two major concepts guided him?
Judgement, Attention, Reasoning
Two major concepts guiding him: age differentiation, mental age
Know age differentiation, mental age, general mental ability, and positive manifold.
Age Differentiation: differentiating older from younger children by the former's greater capacities
Mental Age: age capabilities of a child regardless of chronological age, obtains through age differentiation
General Mental Ability: total product of the various separate and distinct elements of intelligence, remains a large part of the modern concept of IQ
Positive Manifold: all tests, no matter how diverse, measure g.
Binet searched for tasks that could be completed by what percentage of children in a particular age group?
66.67 to 75%
What concept did Spearman introduce? What does this concept mean?
General mental ability (Spearman's G): intelligence consists of one general factor (g) plus a large number of specific factors... Spearman's psychometric g was based on positive manifold-all tests, no matter how diverse, measure g.
What statistical method did Spearman develop to support his notion of g?
Factor analysis - reducing a set of variables to a smaller number of factors.
According to the gf-gc theory, what are the two basic types of intelligence? How do they differ?
Fluid intelligence: abilities that allow us to learn and acquire information
Crystal intelligence: the knowledge and understanding we have acquired, actual learning that has occurred
Know and be able to calculate IQ using mental age and chronological age
Mental age divided by chronological age x 100 (MA/CA)
What is a deviation IQ and how was it used in the Stanford-Binet scale?
Mean of 100, SD 16 (today it is 15).
If below mean= disability. Two standards above= genius. Two below=v impaired.
Based on standard group score, scores standardized. Rejects intelligent quotient
Define and differentiate basal and ceiling. Be able to give an example of each.
Basal: minimum criterion number of correct responses is obtained before moving forward, ex: base
Ceiling: a specified number of incorrect responses (indicating that they have reached their peak performance)
What factors did Wechsler focus on that those before him had not?
Believed intelligence comprised specific elements that one could individually define and measure but that were interrelated. He also focused on non-intellective factors in intelligence (i.e. anxiety, motivation, vision, hearing, etc.). Designed the tests for adults.
What were some criticisms of the Binet scale by Wechsler?
a. Couldn't be used for adults
b. Wechsler didn't like the idea of a single score
c. Wanted to de-emphasize timed portions
d. Did not take into account non-intellective factors
What is the age range of the Wechsler scales?
WAIS IV: 16-90
WPPSI: 2.5-7.25 years (originally 4-6)
WISC - 6 months to 16 years, 6 months
Overall: 2-90
Why is the inclusion of a point scale a significant improvement? What did a performance scale add?
Point scale: you get points for your responses, easier to group together
Performance scale: can be a bit more subjective, based on observations of the participant's performance
The Vocabulary Subtest
Ability to define words - one of the best measures if intelligence
Deterioration because of emotion or brain damage, vocab is one of the last to be affected
Can measure mental capacity before emotional illness or brain injury or trauma
The Similarities Subtest
Paired items of increasing difficulty
Example: in what way are bread and water alike?
The Arithmetic Subtest
15 simple problems in increasing difficulty
The Digit Span Subtest
Requires subjects to repeat digits forward and backward
Measures short-term memory
Anxiety can affect this test performance, however
The Information Subtest
Ex: How many members are there in the US Congress?
Linked to alertness in environment & cultural opportunities
The Comprehension Subtest
3 types of questions: what should be done in a given situation, provide a logical explanation for some rule of phenomenon, define proverbs such as "a journey of 1000 miles begins with the first step"
Measures everyday practical situations, or common sense
The Letter-Number Sequencing Subtest
Supplementary on the working memory index
Related to working memory and attention
The Digit Symbol-Coding Subtest
Requires subjects to copy symbols
Measures ability to learn an unfamiliar task, visual-motor dexterity, degree of persistence, and speed of performance
The Matrix Reasoning Subtest
Presented with nonverbal, figural stimuli
Task is to identify a pattern or relationship between the stimuli
Good measure of fluid intelligence, information processing and abstract reasoning skills
The Symbol Search Subtest
Core measure in processing speed index scale
Subject is shown two target geometric figures, then recognize these figures in another group
What are the mean, standard deviation, and range for scaled scores, standard scores, and index scores?
Scaled score mean 10, SD 3, range 1-19
Standard score 100, SD 15, range 50-150
Index 100, SD 15
How are the IQ scores calculated?
Raw subtests scores for the index are converted to age-corrected scaled scores which are summed, and then compared with standardization sample (Mental age divided by chronological age x 100)
Name the index scores and what the purpose of each index is.
Verbal comprehension index - vocab (one of last function to be affected from brain damage, measure of crystallized intelligence)
Perceptual reasoning index - visual puzzles (measure of fluid intelligence)
Working memory index - working memory (info we actively hold in our minds)
Processing speed index - how quickly the mind words
What is pattern analysis? What are the concerns when using such a method?
Evaluates large patterns and differences between subtests. Looking at pattern and performance. Concern: Doesn't account for individual variability at all. Inconclusive and contradictory
What is a hold subtest?
When a subtest is not sensitive to brain damage or cerebral dysfunction. Scores that hold despite injury or mental illness/cerebral dysfunction. They're valid.
Which subtests are most sensitive to cerebral dysfunction? Which are considered "hold" subtests?
Cerebral dysfunction: block design sensitive
Vocab holds well.
How would you differentiate the WAIS-IV subtests that measure crystallized intelligence from those that measure fluid intelligence?
Degree to how abstract it is.
Crystalized- held in mind, solid. Vocab.
Fluid- recently learned things. Block design.
Where do traditional intelligence tests fail in studying "normal" abilities?
Measuring baseline for "normal people". 2 standard deviations above or below the mean, doesn't account for outliers/extremes. Ex: blind people might not do well, not because they aren't intelligent, but because they're blind
What are the disadvantages of alternative intelligence tests when compared to Binet and Wechsler?
Disadvantages:
- Weak psychometrics: less validity documentation, weaker standardization sample, test manual limitations, less stable
- Scores are not interchangeable with Binet or Wechsler
- Specificity limits the range of functions/abilities measured
What are the advantages of alternative intelligence tests when compared to Binet and Wechsler?
Advantages:
- Used for specific populations and purposes (limitations, cultural deprivation, foreign-born and/or non-English speaking people
- Nonverbal administration or minimized verbal
- Not as dependent on complex visual-motor integrations
- Useful for screening, supplement, and reevaluations
- Not as reliant on learning/achievement
What theme in relation to future intelligence do you notice about infant development tests?
Not useful in predicting IQ!
Compare and contrast surveillance and screening. Be able to identify either.
Screening: a brief test that determines if something else needs to take place, such as developmental disorder
Surveillance: process of recognizing children who may be at risk of developmental delays. 5 areas- taking care of parents concerns, document and maintain developmental history, accurate observations of child, identifying risks and protective factors, maintaining an accurate record and documenting the process and findings.
Which two infant development tests were discussed in class? What are some disadvantages of each of these tests?
Brazelton Neonatal Assessment Scale
• The Good:
- "newborn competence"
- Interactive
- Ages 3 days to 4 weeks
- 47 scores for reflexes & behavior: not a single score
- Useful for research
• The Bad - This is a bad test!!
- No norms
- Poor test-retest reliability
- Does not predict later IQ!!!!!!!!!!!
The Bayley Scales of Infant Development-II
• Mental & motor scores
- How does motor relate to mental?
• Ages 1-42 months
• Great standardization; great psychometrics
- Problems
• Psychometrics break down at lower ages
• Does not predict later IQ!!!!!!!!!!!
• But: extreme low scores do predict MR later in life
Sensitivity
Accuracy of the test in identifying delayed development - Finding True positives
Specificity
Accuracy of the test in identifying individuals who are not delayed
• Avoiding False Positives
• True negative
False Positives
When the condition is negative, but the test outcome is positive (Type II error)
True Negatives
When the condition is negative and the test outcome is negative
What are acceptable sensitivity and specificity levels for developmental screening tests?
70-80%
How is a learning disability currently defined in the school systems? Is this a good method? Why or why not?
Two standards of deviation, if your difference is different from the normal difference in potential and achievement, then that's a learning disability
Discrepancy hypothesis: between the child's potential to achieve and actual school achievement. It's bad bc science doesn't support it.
For what was the Woodcock Johnson-III designed? (i.e., what does it assess?)
It evaluates learning disabilities in educational settings. It measures general intellectual ability, specific cognitive abilities, scholastic aptitude, oral language, and achievement
- Most testers use only achievement
- It is a TEST OF ACHIEVEMENT
Should test scores be used alone to define developmental or learning disabilities?
No!
What is the most likely cause of expectancy effects?
subtle, usually unintentional, reinforcement that guides the examinee toward the preferred (or expected) response
The phenomenon where a study rater provides more accurate ratings when they know their rating will be observed or checked is called...
Reactivity
T or F: The early Wechsler tests were criticized for being too verbal and not having a non-verbal scale
False
Review: Name one statistic used to measure internal consistency
alpha, KR20, split half method
The concept of Spearman's g refers to what?
Broad intellectual ability
You are in an interview with Oprah and find that her demeanor is causing you to feel relaxed, engaged and willing to share more information than usual. What process is at work?
Social facilitation
True or False. Interviewers who conceal their emotions are rated "best" by interviewees.
False
Knowledge and understanding you have learned is called
A. IQ
B. Crystallized intelligence
C. Fixed intelligence
D. Permanent intelligence
B. Crystallized intelligence
Students also viewed