1. Are important independent variables left out of the model?
-Leaving important variables out of a regression model can bias the coefficients of other variables and lead to spurious conclusions.
-Important variables are those that affect the dependent variable and are correlated with the variables that are the focus of the study.
2. Does the dependent variable affect any of the independent variables?
-If the dependent variable in a regression model has an effect on one or more independent variables, any or all of the regression coefficients may be seriously biased.
-Non-experimental data rarely tell us anything about the direction of a causal relationship. You must decide the direction based on your prior knowledge of the phenomenon you're studying.
-Time ordering usually gives us the most important clues about the direction of causality.
3. How well are the independent variables measured?
-Measurement error in independent variables leads to bias in the coefficients. Variables with more measurement error tend to have coefficients that are biased toward 0.
-The degree of measurement error in a variable is usually quantified by an estimate of its reliability, a number between 0 and 1. A reliability of 1 indicates that the variable is perfectly measured, whereas a reliability of 0 indicates that the variation in the variable is pure measurement error.
4. Is the sample large enough to detect important effects? In small samples, the approximations used to calculate p values may not be very accurate, so be cautious in interpreting them.
5. Is the sample so large that trivial effects are statistically significant?
-In large samples, even trivial effects may be statistically significant.
-You need to look carefully at the magnitude of each coefficient to determine whether it is large enough to be substantively interesting.
-When the measurement scale of the variable is unfamiliar, standardized coefficients can be helpful in evaluating the substantive significance of a regression coefficient.
6. Do some variables mediate the effects of other variables?
-If you're interested in the effect of x on y, but the regression model also includes intervening variables w and z, the coefficient for x may be misleadingly small.
-You have estimated the direct effect of x on y, but you have missed the indirect effects through w and z.
-If intervening variables w and z are removed from the regression model, the coefficient for x represents its total effect on y. The total effect is the sum of the direct and indirect effects.
7. Are some independent variables too highly correlated?
-If two or more independent variables are highly correlated, it's difficult to get good estimates of the effect of each variable controlling for the others. This problem is known as multicollinearity.
-When two independent variables are highly collinear, it's easy to incorrectly conclude that neither has an effect on the dependent variable.
8. Is the sample biased?
-As with any statistical analysis, it's important to consider whether the sample is representative of the intended population.
-A probability sample is the best way to get a representative sample.
-If a substantial portion of the intended sample refuses to participate in the study, regression analysis may produce biased estimates.