Advertisement Upgrade to remove ads

Research Methods

Biases in Using Existing Data

Selective deposit and selective survival

Dimensions of Data Collection Methods (4)

1) Structure
2) Quantifiability
3) Researcher obtrusiveness
4) Objectivity


Useful for quantitative researchers to qualitize their data to provide a richer understanding of phenomena or test their interpretations. Researchers can create a mini case study designed to "give life" to the patterns emerging in the quantitative analysis, to extract more info from the data and to aid in the interpretation.


Qualitative researchers can sometimes profit from quantitizing their data to generate new meaning or to document and confirm their conclusions. Also re-presenting data and lives


Most nursing studies use this data collection. Is data obtained by directly questioning people about the phenomena of interest. Are strong with respect to their directness and versatility. The major drawback is the potential for respondents' deliberate or unconscious misrepresentation.

Projective Techniques

Data collection methods that rely on people's projection of psychological traits in response to vaguely structured stimuli.
Examples of -
Rorschach test
Expressive methods like drawings - people express their needs, motives and emotions by working with or manipulating materials.

Expressive Projective Methods

Projective technique that take the form of play, drawing or role playing.

Observational Methods

Obtaining data through the direct observation of phenomena. A wide variety of human activity and traits are amenable to observations (i.e. characteristics and conditions of individuals, verbal and nonverbal communication, activities and behavior, skill attainment and performance, environmental conditions.Observation is subject to observer biases and to distorted behavior by study participants (reactivity).
Also- observers can only observe those behaviors occurring at the time of the study.


Distorted behavior by study participants who know they are being observed. Eliminated if the observations are made without people's knowledge through some type of concealment - i.e. one-way mirrors

Biophysiologic Measures

Yield high-quality data that are objective and valid, and often cost-efficient for nurse researchers.

Data Collection Plan

A plan typically developed by quantitative researchers before they actually begin to collect their data.


For highly structured data, researchers often use formal data collection instruments.

Data Collection Plan 1 -
Identify data needs

Identification and prioritization of all data needs for addressing the following -
a) Testing the hypotheses or addressing the research questions.
b) Describing sample characteristics
c) Controlling extraneous variables
d) Analyzing potential biasies
e) Understanding subgroup effects
f) Interpreting results
g) Checking the manipulation (did manipulation occur)
h) Obtaining administrative information.

Data Collection Plan 2 -
Selecting Types of Measurements

Select a data collection method for each variable. Combo of several methods in one study is common. May combine several methods that vary in terms of the four basic dimensions. Decisions about methods also driven by ethical considerations.

Data Collection Plan 3 -
Selecting and Developing Instruments

Existing measures of the variables should be sought for use or adaptation. The construction of new instruments requires considerable time and skill and should be undertaken as a last resort. Additional criteria that may affect researchers' decisions in selecting an instrument are resources available, availability and familiarity, norms and comparability, population and appropriateness, administration issues, reputation. Also worded as, selection should be based on such considerations as conceptual suitability, expected data quality, cost, population appropriateness, and reputation.

Data Collection Plan 4 -
Pretest the Data Collection Package

Need to be done even when existing instruments are used to determine its length, clarity and overall adequacy.
Also listed - Identify parts of the instrument package that are difficult for participants to read or understand, find objectionable or offensive, determine if sequencing of questions or instrument is sensible, determine needs for training data collection staff, and whether the measures yield data with sufficient variability.

Data Collection Plan 5 -
Developing Data Collection Forms and Procedures

When researchers cannot collect the data without assistance, they should carefully select data collection staff and formally train them. Also need to develop data collection protocols that include conditions that must be met for collecting data, specific procedures for collecting the data (including requirements for sequencing instruments and recording info), info to provide participants and procedures to follow in the event that a participant becomes distraught or disoriented or other reasons data collection cannot continue.

Data Collection Plan -
Qualitative Studies

Typically adopt data collection plans that are flexible and that evolve as the study progresses.

Qualitative Data Collection

Self reports are the most frequently used type of data collection method in qualitative studies, followed by observation.

Ethnographer's Data Collection

Combine self-report and observation data sources with others such as the products of the culture (e.g. photographs, documents, artifacts).

Qualitative Fieldwork Issues

1) Gaining participants' trust,
2) Pacing data collection to avoid being overwhelmed by the intensity of data
3) Avoiding emotional involvement with participants,
4) Maintaining reflexivity

"Going Native"

Qualitative researchers try to avoid this. It's the emotional involvement with participants.


Qualitative researchers need to maintain this, which is an awareness of the part they play in the study and possible effects on their data.

Qualitative Data Collection Plan

Qualitative researchers need to plan in advance for how their data will be recorded and stored. If technical equipment is used (e.g., audio recorders, video recorders), care must be taken to select high-quality equipment that functions properly in the field.
Selecting research personnel - take into consideration experience, congruity with sample characteristics, unremarkable appearance, personality, availability.
Training Data Collectors - good foundation for training manual

Observer Biases

Objective observations may be interfered or result in erroneous classification or conclusions by -
1) Emotions, prejudices, attitudes, and values of observers
2) Personal interest and commitment
3) Anticipation of what is to be observed
4) Hasty decisions before adequate info is collected

Structure (Data Collection Dimension)

What information is to be gathered and how to gather it. Structured methods give participants limited opportunities to qualify their answers or to explain the underlying meaning of their response. Qualitative studies, rely almost exclusively on unstructured or loosely structured methods of data collection.

Quantifiability (Data Collection Dimension)

Data that will be analyzed needs to be quantified. All variables must be quantitatively measured - even though they may be abstract and intangible phenomena that represent qualities of humans, such as hope, pain, and body image.

Researcher Obtrusiveness (Data Collection Dimension)

Degree to which people are aware of their status as participants Behavior or responses given by participants may not be "normal", and distortions can undermine the values of the research. However, ethical problems may arise if participants are not aware they are being studied.

Objectivity (Data Collection Dimension)

The degree to which two independent researchers can arrive at similar "scores" or make similar observations regarding the concepts of interest. Make judgments regarding participant's attributes or behavior that are not biased by personal feelings or believes. Some data collection approaches require higher degree of subjectivity/objectivity than others.
Positivist Paradigm strive for a reasonable amount of objectivity.
Naturalistic Paradigm - subjective judgment of the investigators is considered an asset because subjectivity is viewed as essential for understanding human experiences.

Qualitative Data Recording and Storing

Taking detailed notes, audio and/or video notes. Notes tend to be incomplete.

Unstructured Interviews

Methods of collecting qualitative self-report data - are conversational discussions on the topic of interest. Informally asking a broad question (GRAND TOUR QUESTION)

Semistructured (Focused) Interviews

Methods of collecting qualitative self-report data - which interviewers are guided by a topic guide of questions to be asked.

Focus Group Interviews

Methods of collecting qualitative self-report data - involve discussions with small homogeneous groups about topics covered in a topic guide. Interviewer (often called the MODERATOR) guides the discussions according to a written set of questions or topics to be covered.

Joint Interviews

Methods of collecting qualitative self-report data - simultaneously talking with members of a dyad (e.g., two spouses)

Life Histories

Methods of collecting qualitative self-report data - used to gather personal recollections of events and their perceived causes and consequences. Narrative self-disclosures about individual life experiences (frequently used by Ethnographers).

Critical Incidents Technique

Methods of collecting qualitative self-report data - Involves probes about the circumstances surrounding a behavior or incident that is critical to an outcome of interest.

Diaries and Journals

Methods of collecting qualitative self-report data - Respondents are asked to maintain daily records about some aspect of their lives.

Think-Aloud Method

Methods of collecting qualitative self-report data - Involves having people use audio-recording devices to talk about decisions as they are making them.

Photo Elicitation

Methods of collecting qualitative self-report data - Is stimulated and guided by photographic images.

Solicited or unsolicited narrative

Methods of collecting qualitative self-report data - Communication of the Internet.

Preparation for In-Depth Interview

Researchers -
a) learn about the language and customs of participants,
b) formulate broad questions
c) make decisions about how to present themselves
d) develop ideas about interview settings
e) take stock of equipment needs.

Conducting Good In-Depth Interview

a) considerable interviewer skill in putting people at ease
b) developing trust
c) listening intently
d) managing possible crises in the field.

In-Depth Self-Report Methods

Advantages - Yield data of considerable richness and are useful in gaining an understanding about little-researched phenomena
Disadvantage - They are time-consuming and yield a large volume of data that are challenging to analyze.

Participant Observation

Unstructured observational data collected by qualitative researchers sometimes. Participant observers obtain information about the dynamics of social groups or cultures within members' own frame of reference.

Phases of Participant Observation

1) Are primary observers and active listening - getting a preliminary understanding of the site
2) As time passes, researchers become more active participants
3) Observations tend to become more focused over time, ranging from DESCRIPTIVE OBSERVATION to FOCUSED OBSERVATION and then to SELECTIVE OBSERVATIONS.

Descriptive Observation

Participant Observations that are more broad observations

Focused Observation

Participant observations that are more carefully selected events or interactions.

Selective Observations

Participant observations that are designed to facilitate comparisons.

Single Positioning

Participant observation where events are to be observed through a fixed location.

Multiple Positioning

Participant observation where events to be observed are done by moving around the site to observe different locations.

Mobile Positioning

Participant observation where events are to be observed through following a person around a site.

Methods for Recording Unstructured Observational Data

Logs and field notes. Field notes are both descriptive and reflective.

Descriptive Notes

Also called Observational notes - are detailed, objective accounts of what transpired in an observational session. Observers strive for detailed, thick description.

Reflective Notes

a) Methodologic Notes - that document observers' thoughts about their strategies.
b) Theoretical Notes/Analytic Notes - that represent ongoing efforts to make sense of the data
c) Personal Notes - Document observers' feelings and experiences.

Phenomenological interview.

Produce rich descriptions of the experience under study - describe from the inside, like a state of mind. Focus on a particular example or incident, try to focus on an example that stands out for its vividness, Attend to hwo the body feels, how things smelled and sounded.

Grounded Theory Interview

Techniques change as theory develops. Initially are similar to open-ended conversations using unstructured interviews. Later ask more direct questions.

Ethnographic interviews

Unstructured - 3 types of questions are used to guide interviews -
1) Descriptive - ask to describe their experiences in their own language (backbone of ethnographic interviews.
2) Structural Questions - more focused and help to develop the range of terms in a category or domain.
3) Contrast Questions - are asked to distinguish differencesin the meaning of terms and symbols.

Oral Histories

Qualitative researchers use to gather personal recollections of events and their perceived causes and consequences. Typically focus on describing important themes rather than individuals. Are a method for connecting individual experiences with broader social and cultural contexts.

Evolving Participant Observer's Role
(Leninger & McFarland 2006)

1) Primarily observation and active listening.
2) Primarily observation with limited participation
3) Primarily participation with continued observation.
4) Primary reflection and reconfirmation of findings with informants.

Windshield Survey (Windshield Tour)

Many times used by Ethnographers which involves an intensive exploration to "map" important features of the community under sudy. Mapping include documenting community reesources, community liabilities, and social and environmental characteristics.

Unstructured Observational Data

1) The physical setting
2) The participants
3) Activities and interactions
4) Frequency and duration
5) Precipitating factors
6) Organization
7) Intangible factors.

Structured Data Collection Methods

Place constraints on both those collecting the data and those providing it.

Structured Self-Report Instruments

Interview schedules or questionaires

Open-ended Questions

Permit respondents to reply in narrative fashion

Closed-ended (Fixed Alternative) Questions

Offer response options from which the respondents must choose.

Dichotomous Questions

Type of closed-ended question which require a choice between two options.

Multiple-Choice Questions

Type of closed-ended question which offer a range of alternatives.

Cafeteria Questions

Type of closed-ended question which respondents are asked to select a statement that best represents their view.

Rank-Order Questions

Type of closed-ended question which respondents are asked to rank a list of alternatives along a continuum.

Forced-Choice Questions

Type of closed-ended question which require respondents to choose between two competing positions.

Rating Questions

Type of closed-ended question which ask respondents to make judgments along an ordered, bipolar dimension.

Checklists or Matrix Questions

Type of closed-ended question which several questions requiring the same response format are listed.

Visual Analogue Scales (VAS)

Type of closed-ended question which are continua used to measure subjective experiences such as pain.

Event history Calendars

And Diaries are used to capture information about the occurrence of events.

Composite Psychosocial Scales

Multiple item self-report tools for measuring the degree to which individuals posses or are characterized by target traits or attributes.

Likert Scales

Comprise a series of statements worded favorably or unfavorably toward a phenomenon. Respondents indicate a degree of agreement or disagreement with each statement; a total score is computed by the summing item scores, each of which is scored for the intensity and direction of favorability expressed.

Summated Rating Scales

Another name for Likert Scales

Semantic Differentials (SD)

Consist of a series of bipolar rating scales on which resondents indicate their reaction toward some phenomenon; scales can measure an evaluative (e.g. good/bad), activity (e.g. active/passive), or potency (e.g. strong/weak) dimension.

Cognitive Tests

Measuring various aspects of cognitive functioning, including intelligence, aptitude or neuropsychological functioning.

Q sorts

People sort a set of card statements into piles according to specified criteria. Can be used to measure attitudes, personality, and other psychological traits.


Brief descriptions of an event or situation to which respondents are asked to react. They are used to assess respondents' hypothetical behaviors, opinions, and perceptions.


ADV over Interviews -
1) Less costly and time consuming than interviews
2) Offer the possibility of anonymity
3) Run no risk of Interviewer Bias
1) Interviews tend to yield higher response rates
2) Be suitable for a wider variety of people
3) Yield richer data than questionaires

Interviewer Skills

1) Put respondents at ease
2) Build rapport with them
3) Need to be skillful at probing for additional information when respondents give incomplete or irrelevant responses.

Questionnaire Administration

1) Group administration - Most convenient and economical way to distribute questionnaires.
2) Mail - risk low response rates, which can result in biased sample
3) Internet Distribution - by an email attachment or as a web-based survey that participants access through a hypertext link and respond directly. Internet surveys rarely yield representative samples because response rates are low and not everyone has access to computers.

Questionnaire Techniques

Follow-up reminders and good cover letters.

Structured Self-Reports

Valuable research tool but are vulnerable to the risk for reporting biases.

Response Set Biases

Structured Self-Report bias that stems from the tendency of some people to respond to questions in characteristic ways independent of content.

Social Desirability Response Bias

Stems from a person's desire to appear n a favorable light.

Extreme Response Set

Results when a person characteristically endorses extreme response alternatives.

Acquiescence Response Set

A YEA-SAYER'S tendency to agree with statements regardless of their content. A converse problem arises when people (NAY-SAYERS) disagree with most statements.

Structured Observational Methods

Impose constraints on observers to enhance the accuracy and objectivity of the observations and to obtain an adequate representation of the phenomena of interest.

Molar Approach

Unit of observation - entails observations of larger segments of behaviors as integral units.

Molecular Approach

Units of observation - treats small, specific actions as separate entities.


Tools for recording the occurrence or frequency of predesignated behaviors, events, or characteristics.

Category Systems

Checklists are based on category systems for encoding observed phenomena in discrete categories. Some checklists categorize exhaustively all behaviors of a particular type (e.g., body movements) in an ongoing fashion, whereas others record particular behaviors while ignoring others.

Rating Scales

Another record-keeping device for structural observations. Observers are required to rate phenomena along a dimension that is typically bipolar (e.g., passive/aggressive); ratings are made either at specific intervals (e.g., every 15 minutes) or after observations are completed.

Time Sampling

Structured observations that involves the specification of the duration and frequency of both observational periods and intersession intervals.

Event Sampling

Structural observations that select integral behaviors or events of a special type for observation.

Technological Advances

Have greatly augmented researchers' capacity to collect, record and preserve observational data. Such devices as audiotape recorders and videotape cameras permit behaviors and events to be described or categorized after their occurrence.

Observational Bias

One of th most prevalent observer biases.Observational methods are an excellent method of operationalizing some constructs, but are subjectt to various biases. Thje greater the degree of observer inference, the more likely that perceptual errors and distortions will occur.

Enhancement of Contrast Effect

One of the most prevalent observer biases.

Central Tendency Bias

One of the most prevalent observer biases.

Halo Effect

One of the most prevalent observer biases.

Assimilatory Biases

One of the most prevalent observer biases.

Errors of Leniency

One of the most prevalent observer biases.

Errors of Severity

One of the most prevalent observer biases.

In Vivo Measurements

Biophysiologic measures that are performed within or on living organisms, like blood pressure measurement.

In Vitro Measurements

Biophysiologic measures that are performed outside the organisms body, such as blood tests.

Biophysiologic Measures

Advantage of being objective, accurate and precise, BUT CARE MUST BE TAKEN IN USING SUCH MEASURES with regard to practical, technical, and ethical considerations.


Involves the assignment of numbers to objects to represent the amount of an attribute, using a specified set of rules. Researchers strive to develop or use measurements whose rules are isomorphic with reality.



Sources of Measurement Error

Situational contaminants, response-set biases, and transitory personal factors, such as fatigue.

Obtained Scores

From an instrument consist of TRUE SCORE component and an error component or ERROR OF MEASUREMENT.

True Score

The value that would be obtained for a hypothetical perfect measure of the attribute.

Error of Measurement

Error component of measurement that represent measurement accuracies. Added to True score equals obtained score.


One of two primary criteria for addressing a quantitative instrument, is the degree of consistency or accuracy with which an instrument measures an attribute. The higher the reliability of an instrument, the lower the amount of error in obtained scores.

Reliability Coefficient

There are many different methods for assessing an instrument's reliability and computation of the reliability coefficient. Is based on the computation of correlation coefficient.
Usually range from .00 to 1.00, with higher values reflecting greater reliability.
Reliability coefficients reflect the proportion of true variability in a set of scores to the total obtained variability.

Correlation Coefficient

Indicates the magnitude and direction of a relationship between 2 variables.
Can range from -1.00 (a perfect negative relationship) through zero to +1.00 ( a perfect positive relationship).


Aspect of reliability, which concerns the extent to which an instrument yields the same results on repeated administration, evaluated by TEST-RETEST RELIABILITY.

Test-Retest Reliability

Test the stability aspect of reliability.

Internal Consistency Reliability

Refers to the extent to which all the instrument's items are measuring the same attribute. Assessed by using Cronbach's Alpha method.

Cronbach's Alpha

Method used to test an instruments internal consistency.

Interrater (interobserver) Reliability

Estimate of reliability assessment that focuses on equivalence between observers in rating or coding behaviors.

Kappa Statistic

Reliability assessment in which the consensus measure capturing interrater agreement within a small number of categories is desired.

Intraclass Correlation Coefficient

Reliability assessment that focuses on consistency of ratings.


The degree to which an instrument measures what it is supposed to be measuring.

Face Validity

Refers to whether the instrument appears, on the face of it, to be measuring the appropriate construct.

Content Validity

Concerned with the sampling adequacy of the content being measured.

Content Validity Index (CVI)

Expert ratings on the relevance of items can be used to compute the CVI Information.

Item-level CVIs (I-CVIs)

Represent the percentage of experts rating each item as relevant.

Scale-Level CVIs

Using the average calculation method (S-CVI/Ave) are the average of all I-CVI values.

Criterion-Related Validity

Includes both PREDICTIVE VALIDITY and CONCURRENT VALIDITY. Focuses on the correlation between the instrument and an outside criterion. The instrument is said to be valid if its scores correlate highly with scores on the criterion. The key issue is whether the instrument is a useful predictor of other behaviors, experiences, or conditions.

Construct Validity

An instrument's adequacy in measuring the focal construct, is primarily a hypothesis-testing endeavor. Is a key criterion for assessing the quality of a study, and construct validity has most often been addressed in terms of measurement issues.

Known-Group Technique

Construct validation method which contrasts scores of groups hypothesized to differ on the attribute

Factor Analysis

Construct validation method that is a statistical procedure for identifying unitary clusters of items or measures. Identifies dimensions underlying a central construct. Each dimension is called a FACTOR. The procedure is used to identify and group together different items measuring an underlying attribute. Confirmatory Factor Analysis.

Multitrait-Multimethod Matrix Technique (MTMM)

Construct validation approach which is based on the concepts of convergence and discriminability.


Refers to the evidence that different methods of measuring the same attribute yield similar results.


Refers to the ability to differentiate the construct being measured from other, similar concepts.

Psychometric Assessment

Of a new instrument is usually undertaken to gather evidence about vailidy, reliability, and other quality criteria.


Important criteria for screening and diagnostic instrument. Is the instrument's ability to identify a case correctly (i.e., its rate of yielding true positives)


Important criteria for screening and diagnostic instrument. is the instrument's ability to identify noncases correctly (i.e., its rate of yielding true negatives).

Positive Predictive Value (PPD)

Predictive values are posterior probabilities - the probability of an outcome after the results are known. PPV is the proportion of people with a positive result who have the target outcome or disease.

Negative Predictive Value (NPV)

Predictive values are posterior probabilities - the probability of an outcome after the results are known. NPV is the proportion of people who have a negative test result who do not have the target outcome or disease.

Likelihood Ratios

Summarizes the relationship between specificity and sensitivity in a single number. The likelihood ratio addresses the question, "how much more likely are we to find that an indicator is positive among those with the outcome of concern compared with those for whom the indicator is negative?

Receiver Operating Characteristic Curve (ROC Curve)

Sensitivity is plotted against specificity to determine the optimum cutoff point for caseness. The score (cutoff point) that yields the best balance between sensitivity and specifricity can then be determined. The optimum cutoff is at or near the shoulder of the ROC curve.

Predictive Validity

Type of criterion-related validity refers to the adequacy of an instrument in differentiating between people's performance on a future criterion (i.e. incoming student and high school grades).

Concurrent Validity

Type of criterion-related validity refers to an instrument's ability to distinguish individuals who differ on a present criterion. i.e. psych test used to differentiate those patients who may or may not be released from mental hospital.

Hypothesized Relationships

Method of construct validation involves testing hypothesized relationships, often on the basis of theory. i.e. According to theory, construct X is positively related to construct Y.
Instrument A is a measure of construct X; instrument B is a measure of construct Y.
Scores on A and B are correlated positively, as predicted.
Therefore, it is inferred that A and B are valid measure of X and Y.


Other criteria for assessing quantitative measures. Use of Spearman-Brown formula to estimate how reliable the scale would be with fewer items. Efficiency is more characteristic of certain types of data collection procedures than others.

See More

Please allow access to your computer’s microphone to use Voice Recording.

Having trouble? Click here for help.

We can’t access your microphone!

Click the icon above to update your browser permissions above and try again


Reload the page to try again!


Press Cmd-0 to reset your zoom

Press Ctrl-0 to reset your zoom

It looks like your browser might be zoomed in or out. Your browser needs to be zoomed to a normal size to record audio.

Please upgrade Flash or install Chrome
to use Voice Recording.

For more help, see our troubleshooting page.

Your microphone is muted

For help fixing this issue, see this FAQ.

Star this term

You can study starred terms together

NEW! Voice Recording