Upgrade to remove ads
Only $35.99/year

Field Work and Data Quality Issues

Terms in this set (26)

Why is Data quality is a major concern in research?
if you have bad quality data, there is nothing you can do about it anymore afterwards with statistical analyses.
What is interview evaluation?
which is when people feel they are being evaluated (and thus change their answers)
What are the 2 types of errors in research?
1. Sampling error
2. Non-sampling error:
What is a non sampling error?
all errors in a survey except those due to the sampling plan/sample size
- nonresponse errors
- data-gathering errors
- data-handling errors
- data-analysis errors
- interpretation errors

These all occur at the data collection stage.
At what stage do most dramatic errors happen?
At the data collection stage. They are not only (a) more common in that stage, because data collection is difficult, they also (b) have large implications for the research results.
What are the two data collection errors?
1. Fieldworker error: errors committed by the persons who administer the questionnaires. Depends largely on the resources they have, the training they had, and the level of professionalism... but fieldwork errors occur not only in do-it-yourself surveys, but also with experienced fieldworkers!

2. Respondent error: errors committed on the part of the respondent. These occur regardless of the method of data collection, but some data collection methods have greater risk for errors.
What is an Intentional fieldworker error?
Willful violation of the data collection requirements set forth by the researcher.

-Interviewer cheating: intentionally misrepresents respondents
-Leading respondents: influences respondents answers by body language, wording, etc.
Why high intentional fieldworker errors happen?
Because of the compensation system: fieldworkers do not often get paid highly and they may resort to convenience, such as filling out surveys themselves or having someone who is not selected fill the survey out.
Why can't we fix the compensation system?
Because there are long periods of inactivity, this would be very expensive.
What is an Unintentional fieldworker error?
-interviewer believes he/she is performing correctly
-Interviewer personal characteristics: (accent, gender, demeanor, etc.)
-Interviewer misunderstanding
-Fatigue-related mistakes
What may cause Interviewer misunderstanding?
There is a considerable education gap between researcher who design the questionnaires and also collect the data, versus when a survey administer collects the data (especially if they are inexperienced).
What is an Intentional respondent error?
Respondents willfully misrepresent themselves in surveys through:
1. Falsehoods:e.g. embarrassed, protect privacy, etc. Some topics are more prone to false answers: income categories, sometimes even age, sexual orientation, etc. or even out of boredom and want survey to end faster.

2. Nonresponse
-Refusal
-Break-off
-Item omission
What is an Unintentional respondent error?
Respondent gives a response that is not valid but he/she believes it is the truth.

-Respondent misunderstanding
-Guessing
-Attention loss
-Distractions (e.g., interruptions)
-Fatigue
How do we control Intentional fieldworker error?
1. Supervision
2. Validation
→ this strategy is aimed at the falsification/cheating problem
What is an example of supervision?
Most telephone interviewing companies tap into interviewer's interview phone calls (with the respondent and the interviewer being aware that calls may be monitored)

NOTE: Impossible to supervise every interview that occurs (high cost). Train people in ethics and hope they don't do anything wrong
What is an example of validation?
e.g., recontact the respondents to find out if they really took part in the survey (e.g. Industry standard: select 10% of the sample and contact them again)
How do we control Unintentional field worker error?
-selection and training of interviewers

-orientation sessions: meetings in which the researcher introduces and explains the questionnaire & the requirements to the fieldworkers. The more competent the field worker is through training, the smaller the chance for unintentional errors.

-taking breaks and alternate surveys (prevent tiredness and boredom)
How do we control intentional respondent errors?
Anonymity: name is not disclosed at all, name is not registered

Confidentiality: name will not be made public /to third parties
Prevent respondents from lying

Validation checks: quite subjective judgements by the fieldworker in order to detect wrong answers/lies (e.g., old respondent who says he's young)

Third person technique (projective techniques: balloon test, sentence completion, etc.)
How do we control Unintentional respondent errors?
-well drafted questionnaires (questions and responses)
-omitting prompters: "we're almost done"
-Reduce guessing by leaving a "no response/ I don't know" option
-Likert scales: you can insert one reversed item in the negative way to see if they are consistent in their profile
What is a nonresponse error?
It includes:
-Refusals to participate in survey
-Break-offs during the interview
-Refusals to answer certain questions (item omissions)
What causes nonresponse error?
-Invasion of privacy
-Consumer skepticism regarding research
-The use of research as a disguise for telemarketing
What is response rate?
The % of the total sample who completed the survey/interview

e.g. US Census bureau now has about 55% response rate (so 45% nonresponse) from their 25.000 households
Willingness to participate in surveys differs strongly by demographic characteristics!
How do you define a completed survey/interview?
Almost all surveys will have some omission of items, or breakoffs. Based on your judgement, this will vary with each MR project.
How do you deal with a nonresponse error beforehand?
Mail surveys:
-Advance notification
-(Monetary) incentives

Follow-up mailings
- Telephone surveys:
-Callback attempts
How do you deal with a nonresponse error afterwards?
Adjusting data by:

-Oversampling: more expensive! Drawing a sample that is larger than the group to be analyzed (similar to quota sampling)

-Weighing (quota): you apply weights to subgroups that are unevenly presented. e.g., weigh underrepresented groups more heavily than overrepresented groups.
What is preliminary questionnaire screening?
Unsystematic & systematic checks of completed questionnaires

You want to conduct raw data inspection before you start analyzing:
-Incomplete questionnaires
-Non-responses to specific questions (item omissions)
-Yes/no saying patterns
-Middle-of-the-road patterns
-Unreliable responses (Chronbach's alpha)
Sets found in the same folder