Predictors are typically used to predict or estimate performance on a criterion measure.
Why is a predictor's criterion-related validity of particular interests in organizations?
1.Conduct a job analysis
2. Select or Develop the Predictors
3. Administer the Predictor and the Criterion.
4. Correlate Predictor and Criterion scores
5. Check for Differential Validity and Unfairness
6. Evaluate Incremental Validity
7. Cross- Validate
What are the 7 steps in establishing criterion-related validity
Conduct a job analysis- Identify KSAO's (knowledge, skills, abilities and other characteristics) required for successful job performance.
What is the first step in establishing criterion-related validity?
Select or Develop the Predictors that measures the attributes identified by the job analysis.
What is the second step in establishing criterion-related validity?
Administer the Predictor to a sample of job applicants (predictive validity) or current employees (concurrent validity); obtain Criterion information for all individuals in the sample.
What is the third step in establishing criterion-related validity?
Correlate Predictor and Criterion scores- calculate a correlation coefficient to determine if there is a statistically significant relationship between predictor and criterion scores.
What is the fourth step in establishing criterion-related validity?
Check for Differential Validity and Unfairness - determine if the predictor has different levels of validity for different subgroups (differential validity) or if the predictor scores consistently underestimate the criterion performance of any subgroup (unfairness)
What is the fifth step in establishing criterion-related validity?
Evaluate Incremental Validity- Determine if use of the predictor increases the proportion of correct decisions
What is the sixth step in establishing criterion-related validity?
Cross- Validate perform steps 3-6 with a new sample. Because all of the chance factors that maximized the original validity coefficient are not present in the second sample, the cross-validation coefficient is usually smaller than the original coefficient. This is called shrinkage.
What is the seventh step in establishing criterion-related validity?
Because all of the chance factors that maximized the original validity coefficient are not present in the second sample, the cross-validation coefficient is usually smaller than the original coefficient. This is called shrinkage.
Refers to the increase in decision-making accuracy and employer will achieve by using the predictor to make selection decisions. Also, the slopes of the regression lines for two different groups differ.
Validity coefficient, selection ratio and base rate.
What factors impact incremental validity?
Helps increase decision making accuracy.
How does a validity coefficient impact incremental validity?
Yes in certain conditions since the usefulness of a predictor also depends on other factors such as selection ratio and base rate.
Can a predictor with a validity coefficient as low as .20 or .30 increase decision making accuracy?
The ratio of job openings to job applicants. (ex. 1:50 indicates there is one job opening for every 50 applicants). This can be used to help the usefulness of a predictor by setting a higher cutoff score that can reduce the risk of hiring false positives.
A low selection ratio (many applicants for one or few jobs).
What type of selection ratio is preferred since it allows an employer to be more selective when making hiring decisions?
Raise the predictor cutoff score.
When having a low selection ratio, the employer can do what and thereby reduce the risk of hiring false positives?
Another factor that affects a predictor's incremental validity is this. It is the percent of employees who are performing satisfactorily without use of the proposed predictor and it ranges from 0 to 1.0.
Moderate base rates (close to .50).
Which base rates are associated with the greatest incremental validity?
When most of the employees being hired using the current techniques are either highly successful or unsuccessful (i.e. when the base rate is very high or very low), the addition of a new predictor will have little effect on the quality of the work force.
Why are moderate base rates close to .50 better than base rates that are very low or high?
These can be used to estimate the percent of new hires that will be successful as employees given various combinations of validity coefficients, selection ratios and base rates.
When the selection ratio is low and the base rate is moderate a predictor with a low validity coefficient can improve decision making accuracy.
According to the Taylor- Russell tables the percentage of satisfactory employees will be larger when a test has a validity coefficient of .30, the selection ratio is 1:100 and the base rate is .45 than when a test has a validity coefficient of .40, the selection ration is 1:10 and the base rate is .25. What is the reason for this?
Title VII legislation of the Civil Rights Act
Prohibit discrimination in hiring, placement, training, promotion and retention on the basis of race, color, religion, sex or national origin and more recently on the basis of age or disability. It is enforced by the EEOC. These guidelines apply to "any measure, combination of measures or procedures used as a basis for employment decisions and pertain to selection procedures ranging from "traditional paper and pencil tests, performance tests, training programs or probationary periods and physical education and work experience requirements through informal or casual interviews and unscored application forms.
Adverse Impact (always on exam!)
This enforced by the EEOC and according to the Uniform Guidelines an employment procedure might be discriminating against a minority group when it has this. This occurs when a use of a procedure results in a substantially different selection, placement or promotion rate for members of a subgroup
The 80% (4/5th rule)
This is used to determine if a procedure is having an adverse impact.
The hiring rate for the majority group is multiplied by 80% to determine the minimum hiring rate for the minority group.
How is the 80% rule applied?
90% x 80% rule= 72%> 50% African American hiring rate.
If a selection tests results in 50% hiring rate for African-Americans and 90% rate for whites. Would the test have adverse impact? How would it be calculated?
It can be used to determine if a person has been discriminated against only when he or she is a member of a specific group or a finding of adverse impact, the 80% rule requires that a pattern of discrimination be established.
What is a shortcoming of the 80% rule?
Differential Validity and Unfairness
What are the two most common causes of employment procedures that result in adverse impact?
When a measure is valid for one group but is not valid or significantly less valid) for another group. (See figure 1a. and 1.b page 10 I/O book). It causes adverse impact on the other group.
What is the characteristic that distinguishes the groups (gender, race age) in differential validity (when a measure is valid for one group but not another?
This occurs when members of one group consistently obtain lower scores on a predictor than members of another group, but the differences in predictor scores are not reflected in differences in actual job performance (ex. Did bad on employment sales test, but performing great on the job). The predictor has the same degree of validity for both groups, but one group consistently scores lower on the predictor then other group members if the same predictor cutoff score is used for both groups.
Business necessity and bona fide occupational qualification (BFOQ)
What are two situations in class-action suits when a defendant (the employer) can continue to use a procedure that when it is having adverse impact?
It must prove that the measure is job-related and fairly screens out poor performers but also that its use if required for the safe and efficient operation of the business.
When using the defense of business necessity that continued use of a procedure that has adverse impact, what must the employer prove?
When it is an essential determinant of the genuineness of the job (ex. Interview men for a male role in play) or when a person's gender must be taken into consideration to maintain community standards of morality or propriety (e.g. women for the position of a women's restroom attendant).
When using the defense of bona fide occupational qualification (BFOQ) what does the defendant have to show?
Americans with Disabilities Act
Type of legislation that effects the selections process. Legislation that requires companies with 25 or more employees to avoid using procedures that discriminate against people with physical or mental disabilities and when a disabled person is able to perform the essential functions of a job, to consider the person qualified and to make "reasonable accommodations". Also pre-employment procedures be directly related to job requirements (ex. Medical exam not allowed before job offer and if done after must be made to all applicants). Exception is drug tests.
Multiple regression, Multiple cutoff and Multiple (Successive) Hurdles.
Since the complexity of most jobs, one predictor is likely to be completely adequate for the job selection. What are the different ways multiple predictors can be combined?
One way to use multiple predictors. Predictor scores are weighted and summed to yield an estimated criterion score, with each predictor's weight being determined by its correlation with the criterion and with the other predictors. It is compensatory since exceptional performance on one predictor can offset poor performance on another predictor.
A non-compensatory method of using multiple predictors. A minimum score on each predictor must be obtained before a job applicant will be considered for selection. It can be used with multiple regression by first selection only those people who score above the minimum cutoff point on all predictors and then using multiple regression equation to predict each selected applicant's criterion score.
Multiple (Successive) Hurdles
A technique where predictors are administered one at a time in a predetermined order, with each predictor being administered only if the previous one has been successfully passed or completed.
It saves time and money since all the predictors are not administered to all applicants.
What is an advantage of the Multiple (Successive) Hurdles?
Objective tests (General Mental (Cognitive) Ability tests, Job Knowledge tests, Personality Tests, Interest Tests), Application forms (Biodata), Interviews, work samples and Assessment Centers.
What are the different methods commonly used for selection and other employment decisions?
General Mental (Cognitive) Ability tests (GMA)
Which of the various predictors used for selection produce the highest validity coefficients across jobs and job settings?
.51 and .56
What is the correlation coefficient of GMA and job performance and GMA and performance in job training programs?
They vary somewhat depending on the objectivity of the criterion measure. Hunter and Hunter report and average correlation coefficient of .53 when the criterion is performance ratings and .75 when the criterion is performance on a work sample.
For measures of GMA, how do validity coefficients do?
They are good predictors of job performance with validity coefficients being similar to or even larger than those for GMA, however, unlike GMA, measures of job knowledge are specific and are useful only when applicants have had previous job training or experience.
What are the pros and cons of a Job Knowledge tests in terms of predictors of job performance?
Meta-analysis show that personality tests can be useful, although studies are inconsistent with regard to the relative validities of global and specific personality measures. (of the big 5 only conscientiousness has been found to be the best predictor of job performance across difft jobs, settings and criterion measures).
How do Personality tests fair as predictors in organizational settings?
Neuroticism, extraversion, openness to experience, agreeableness and conscientiousness. Conscientiousness is the best predictor.
What are the Big Five personality traits and of these traits, which one has been to be the best predictor of job performance across different jobs, job settings and criterion measures?
Applicants whose interest profiles resemble those of successful employees will perform best on the job.
What is the basis in which Interest tests have been used in organizations in selection?
Strong-campbell Interest Inventory (SCII) and the Kuder Occupational Interest Survey (KOIS)
What are the two most widely used interest tests?
Interest tests are very susceptible to faking. It is more valid for predicting job choice, satisfaction and persistence than job success.
Although the Strong-campbell Interest Inventory (SCII) and the Kider Occupational Interest Survey (KOIS) are useful for career and vocational counseling, their use as selection tools are questionable for these reasons.
The use of this as a selection tool is based on the assumption that the best predictor of future behavior is past behavior and this assumption has been confirmed by studies finding that this to be among the most valid predictors across organizations, occupations and samples.
When the items are derived empirically (ex. The Biographical Information Blank (BIB)).
When is the validity of biodata maximized?
The Biographical Information Blank (BIB)
This type of empirically-derived form contains multiple-choice questions that not only assess an applicant's job history but also his or her family background, economic history, health, attitudes, preferences, values, etx.
They often contain items that lack face validity- items not clearly related to job performance. As a consequence, applicants may resist them because they perceive them to be an invasion of privacy.
What is a disadvantage of the The Biographical Information Blank (BIB) and other empirically-derived forms?
These are the most commonly used method for obtaining information about job applicants, although they generally have lower levels of reliability and validity than many other selection techniques.
1.Accuracy of interviews increased by training interviewers in observation and interpersonal skills. 2. The reliability and validity of interviews is maximized when a structured interview is used (all asked the same question in the same way). 3. Past-oriented interviews (behavioral descriptions) are more valid than future-oriented (situational interviews).
What are three ways that reliability and validity can be increased in interviews as a selection tool?
Some recommend use of panels, but there is evidence that structured interviews are the most reliable.
What does the evidence state that the most reliable, valid and cost effective type of interview?
Work (job) samples
This requires an individual to perform tasks similar to or identical to those actually performed on the job and are most commonly used to select applicants for jobs requiring clerical, mechanical, or technical skills. They are most often used with experienced applicants to assess their current ability to perform a job, can also be used with w/those w/o prev training to assess capacity to benefit from training.
What is it called when work samples are used with w/those w/o prev training to assess capacity to benefit from training?
Realistic job preview (RJP)
Work samples are sometimes used as a part of this which may also include a written description of the job, a video, discussions with current workers and other strategies.
Disillusionment with the job is a primary cause of turnover. RJPs reduce unrealistic job expectations and thereby increase job satisfaction and commitment and lower turnover rates.
What is the reason use of Realistic job previews are done?
These are used to evaluate managerial-level personnel, most often to determine if lower-level managers should be promoted or to identify a manager's need for training and personal development. It is a team of evaluators that look at a number of dimensions including leadership, human relations skills, oral and written communication, resistance to stress, decision-making and flexibility.
A number of techniques including structured interview, written tests and situational tests (work samples) with best known in-basket test.
What techniques do assessment centers incorporate?
This is a technique used to increase diversity and reduce adverse impact in the workplace. It involves specifying a range of scores on a test that are considered equivalent for selection purposes and then considering applicants who obtain scores within that range. Then other job-related and/or diversity factors are then used to select among the applicants who obtain scores within that range.