Research Bias

Created by jimlee 

Upgrade to
remove ads

Method bias leads to

Type I and Type II errors
(Burton-Jones, 2009)

Measurement Method Elements

1. Rater
2. Instrument
3. Procedure
(Burton-Jones, 2009)

Evolutionary critical realism

1. Reality exists independent of researcher
2. Knowledge of reality is fallible
3. Theories and methods help understand reality
4. Researchers seek better theories and methods over time
(Burton-Jones, 2009)

Extrinsic trait

Property of something outside of human mind

Intrinsic trait

Property of something inside human mind

Trait score

The real score of a property
(Burton-Jones, 2009)

Measured score

The score obtained via measurement method in attempt to capture the trait score
(Burton-Jones, 2009)


Difference between trait score and measured score
(Burton-Jones, 2009)

Method bias

Difference between trait score and measured score that is due to measurement method
(Burton-Jones, 2009)

Common method bias

Difference between trait score and measured score that is due to use of common method to measure traits
(Burton-Jones, 2009)
(Podsakoff, Mackenzie, Lee, & Podsakoff, 2003)
(Malhotra, Kim, & Patil, 2006)

Knowledge bias

Difference between trait score and measured score that is due to the lack of the rater's knowledge of the trait score
(Burton-Jones, 2009)

Science of the sophomore (Gordon et al, 1986)

Rating bias

Difference between trait score and measured score that occurs when the rater does not provide an accurate estimate of the trait score
(Burton-Jones, 2009)

Consistent bias

Difference between trait score and measured score that occurs the same way each time the trait is measured - systematic error
(Burton-Jones, 2009)

Inconsistent bias

Difference between trait score and measured score that varies each time it is measured - random error
(Burton-Jones, 2009)

Common method variance

Change in the dispersion of measured scores around an expected value due to common method bias
(Burton-Jones, 2009)
(Podsakoff, Mackenzie, Lee, & Podsakoff, 2003)

Sources of method bias

Knowledge bias
Rating bias
(Burton-Jones, 2009)

Rating biases

Consistency motif
Illusory correlations
Social desirability
Mood states
(Podsakoff et al, 2003)

Sources of common method bias

Common Rater
Item Characteristics
Item Context
Measurement Context
(Podsakoff et al, 2003)

What are the main types of bias?

Method bias
Knowledge bias
Rater bias
Consistent bias
Inconsistent bias

Consistent Bias

Bias that occurs the same way each time a trait is measured (Burton-Jones, 2009)

Could indicate validity issues

Inconsistency bias

Bias that varies each time the trait is measured and that averages out to zero in large sample. Random error. (Burton-Jones, 2009)

Consistency motif

Respondents maintain consistency to avoid cognitive dissonance (Podsakoff et al., 2003)

Common Method Bias References

Podsakoff, MacKenzie, Lee, & Podsakoff, 2003
Richardson, Simmering, & Sturman, 2009
Burton-Jones, 2009
Warkentin, Shropshire, & Johnston, 2006
Boss et al., 2009

Spector (2006)

Argues that CMV doesn't exist
1. Method alone is insufficient to produce bias
2. All constructs measured with the same method share teh same biases

Correlation Marker Technique

Used to detect CMV by examining correlation between substantive variable and a priori marker variable that is theoretically separated from substantive variable. Any correlation is due to

What are the statistical remedies for CMV?

1. Unmeasrued latent method factor
2. Correlation Marker
3. CFA Marker
4. Directly measured latent method factor (measure source of bias)

What method of CMV detection does Richardson, Simmering, & Sturman 2009 recommend

CFA Marker because it can detect if CMV is present or not in congeneric and non-congeneric situations

What method of CMV correction does Richardson, Simmering, & Sturman 2009 recommend

None, because all techniques increase the probability of Type I and Type II errors if no CMV exists

What makes a good CFA marker?

1. Theoretically unrelated to substantive variables
2. Susceptible to the same CMV forces as substantive variables
- latent variable
- same scale
- same anchors

Describe Harman's Single-factor test

1. Load all variables into EFA
2. Examine unrotated solution
3. If CMV is present, then a single factor will emerge or one factor will account for the majority of covariance

What are the limitations of Harman's Single factor test?

1. Does not partail out method effects
2. Single factor result may stem from lack of discriminant validity
3. CMV would have to account for 100% of covariance to be regarded as a problem

What are the three variations of partial correlation procedures to control for method biases?

1. Partialling out social desirability or general affectivity
2. Partialling out "marker" variable
3. Partialing out general factor score

What are statistical remedies for CMV?

1. Harman's single factor test
2. Partial correlation procedure
3. Controlling for the effects of a directly measured latent methods factor
4. Controlling for the effects of an unmeasured latent methods factor
5. Multiple method factors

What are the three perspectives on CMV?

1. CMV does not exist
2. Noncongeneric perspective - CMV exists and has an equal effect on all constructs
3. Congeneric perspective - CMV exists, but does not effect all constructs equally
(Richardson, Simmering, & Sturmam, 2009)

Support for congeneric CMV perspective

Williams & Anderson (1994)
Williams et al. (2003)
Rafferty & Griffin (2004)
Cote & Buckley (1987)

Cote & Buckley's (1987) findings

CMV depends on type of measure
41% for attitudes
25% for personality
23% performance and satisfaction

Correlation Marker Technique

Partials out method variance equally from all constructs
A non-theoretical related variable is measured
The variance between the marker variable and substantive variables is thought to be caused by CMV.

CFA Marker Technique

1. Compares the change in fit between a model with marker construct-substantive items loading freely to one which they are constrained to zero
2. Models at item level, so accounts for congeneric and noncongeneric CMV
(Richardson, Simmering, Sturman 2009)

Sharma, Yetton, & Crawford (2009)

CMV in TAM research

Procedural CMV remidies

1. Measure predictor and criterion measures from two different raters
2. Temporal separation - time between measures
3. Proximal - physical distance between measures (item 1 vs. item 20)
4. Psychological separation - cover story to reduce linkage between predictor and criterion variables
5. Eliminate common scale properties - anchors, scale types, polarity
6. Eliminate ambiquity
7. Reduce social desirability
8. Balance positive and negative items
Podsakoff, et al. (2003)
Podsakoff, Mackenzie, Podsakoff (2012)

Disadvantages of temporal separation

1. Assumes relationship is stable over time
2. Determining appropriate amount of delay
3. Non-method factors may change criterion variable
4. Respondent attrition
5. Increases complexity of design
Podsakoff, Mackenzie, Podsakoff (2012)

Disadvantages of proximal separation

1. Increase instrument length
2. If filler items are conceptually related, then context effects could increase method bias
Podsakoff, Mackenzie, Podsakoff (2012)

Disadvantages of psychological separation

1. Effectiveness is dependent upon credibility of cover story
Podsakoff, Mackenzie, Podsakoff (2012)

Please allow access to your computer’s microphone to use Voice Recording.

Having trouble? Click here for help.

We can’t access your microphone!

Click the icon above to update your browser permissions above and try again


Reload the page to try again!


Press Cmd-0 to reset your zoom

Press Ctrl-0 to reset your zoom

It looks like your browser might be zoomed in or out. Your browser needs to be zoomed to a normal size to record audio.

Please upgrade Flash or install Chrome
to use Voice Recording.

For more help, see our troubleshooting page.

Your microphone is muted

For help fixing this issue, see this FAQ.

Star this term

You can study starred terms together

NEW! Voice Recording

Create Set