Dimensions of Data Collection Methods (4)
3) Researcher obtrusiveness
Useful for quantitative researchers to qualitize their data to provide a richer understanding of phenomena or test their interpretations. Researchers can create a mini case study designed to "give life" to the patterns emerging in the quantitative analysis, to extract more info from the data and to aid in the interpretation.
Qualitative researchers can sometimes profit from quantitizing their data to generate new meaning or to document and confirm their conclusions. Also re-presenting data and lives
Most nursing studies use this data collection. Is data obtained by directly questioning people about the phenomena of interest. Are strong with respect to their directness and versatility. The major drawback is the potential for respondents' deliberate or unconscious misrepresentation.
Data collection methods that rely on people's projection of psychological traits in response to vaguely structured stimuli.
Examples of -
Expressive methods like drawings - people express their needs, motives and emotions by working with or manipulating materials.
Expressive Projective Methods
Projective technique that take the form of play, drawing or role playing.
Obtaining data through the direct observation of phenomena. A wide variety of human activity and traits are amenable to observations (i.e. characteristics and conditions of individuals, verbal and nonverbal communication, activities and behavior, skill attainment and performance, environmental conditions.Observation is subject to observer biases and to distorted behavior by study participants (reactivity).
Also- observers can only observe those behaviors occurring at the time of the study.
Distorted behavior by study participants who know they are being observed. Eliminated if the observations are made without people's knowledge through some type of concealment - i.e. one-way mirrors
Yield high-quality data that are objective and valid, and often cost-efficient for nurse researchers.
Data Collection Plan
A plan typically developed by quantitative researchers before they actually begin to collect their data.
Data Collection Plan 1 -
Identify data needs
Identification and prioritization of all data needs for addressing the following -
a) Testing the hypotheses or addressing the research questions.
b) Describing sample characteristics
c) Controlling extraneous variables
d) Analyzing potential biasies
e) Understanding subgroup effects
f) Interpreting results
g) Checking the manipulation (did manipulation occur)
h) Obtaining administrative information.
Data Collection Plan 2 -
Selecting Types of Measurements
Select a data collection method for each variable. Combo of several methods in one study is common. May combine several methods that vary in terms of the four basic dimensions. Decisions about methods also driven by ethical considerations.
Data Collection Plan 3 -
Selecting and Developing Instruments
Existing measures of the variables should be sought for use or adaptation. The construction of new instruments requires considerable time and skill and should be undertaken as a last resort. Additional criteria that may affect researchers' decisions in selecting an instrument are resources available, availability and familiarity, norms and comparability, population and appropriateness, administration issues, reputation. Also worded as, selection should be based on such considerations as conceptual suitability, expected data quality, cost, population appropriateness, and reputation.
Data Collection Plan 4 -
Pretest the Data Collection Package
Need to be done even when existing instruments are used to determine its length, clarity and overall adequacy.
Also listed - Identify parts of the instrument package that are difficult for participants to read or understand, find objectionable or offensive, determine if sequencing of questions or instrument is sensible, determine needs for training data collection staff, and whether the measures yield data with sufficient variability.
Data Collection Plan 5 -
Developing Data Collection Forms and Procedures
When researchers cannot collect the data without assistance, they should carefully select data collection staff and formally train them. Also need to develop data collection protocols that include conditions that must be met for collecting data, specific procedures for collecting the data (including requirements for sequencing instruments and recording info), info to provide participants and procedures to follow in the event that a participant becomes distraught or disoriented or other reasons data collection cannot continue.
Data Collection Plan -
Typically adopt data collection plans that are flexible and that evolve as the study progresses.
Qualitative Data Collection
Self reports are the most frequently used type of data collection method in qualitative studies, followed by observation.
Ethnographer's Data Collection
Combine self-report and observation data sources with others such as the products of the culture (e.g. photographs, documents, artifacts).
Qualitative Fieldwork Issues
1) Gaining participants' trust,
2) Pacing data collection to avoid being overwhelmed by the intensity of data
3) Avoiding emotional involvement with participants,
4) Maintaining reflexivity
Qualitative researchers try to avoid this. It's the emotional involvement with participants.
Qualitative researchers need to maintain this, which is an awareness of the part they play in the study and possible effects on their data.
Qualitative Data Collection Plan
Qualitative researchers need to plan in advance for how their data will be recorded and stored. If technical equipment is used (e.g., audio recorders, video recorders), care must be taken to select high-quality equipment that functions properly in the field.
Selecting research personnel - take into consideration experience, congruity with sample characteristics, unremarkable appearance, personality, availability.
Training Data Collectors - good foundation for training manual
Objective observations may be interfered or result in erroneous classification or conclusions by -
1) Emotions, prejudices, attitudes, and values of observers
2) Personal interest and commitment
3) Anticipation of what is to be observed
4) Hasty decisions before adequate info is collected
Structure (Data Collection Dimension)
What information is to be gathered and how to gather it. Structured methods give participants limited opportunities to qualify their answers or to explain the underlying meaning of their response. Qualitative studies, rely almost exclusively on unstructured or loosely structured methods of data collection.
Quantifiability (Data Collection Dimension)
Data that will be analyzed needs to be quantified. All variables must be quantitatively measured - even though they may be abstract and intangible phenomena that represent qualities of humans, such as hope, pain, and body image.
Researcher Obtrusiveness (Data Collection Dimension)
Degree to which people are aware of their status as participants Behavior or responses given by participants may not be "normal", and distortions can undermine the values of the research. However, ethical problems may arise if participants are not aware they are being studied.
Objectivity (Data Collection Dimension)
The degree to which two independent researchers can arrive at similar "scores" or make similar observations regarding the concepts of interest. Make judgments regarding participant's attributes or behavior that are not biased by personal feelings or believes. Some data collection approaches require higher degree of subjectivity/objectivity than others.
Positivist Paradigm strive for a reasonable amount of objectivity.
Naturalistic Paradigm - subjective judgment of the investigators is considered an asset because subjectivity is viewed as essential for understanding human experiences.
Qualitative Data Recording and Storing
Taking detailed notes, audio and/or video notes. Notes tend to be incomplete.
Methods of collecting qualitative self-report data - are conversational discussions on the topic of interest. Informally asking a broad question (GRAND TOUR QUESTION)
Semistructured (Focused) Interviews
Methods of collecting qualitative self-report data - which interviewers are guided by a topic guide of questions to be asked.
Focus Group Interviews
Methods of collecting qualitative self-report data - involve discussions with small homogeneous groups about topics covered in a topic guide. Interviewer (often called the MODERATOR) guides the discussions according to a written set of questions or topics to be covered.
Methods of collecting qualitative self-report data - simultaneously talking with members of a dyad (e.g., two spouses)
Methods of collecting qualitative self-report data - used to gather personal recollections of events and their perceived causes and consequences. Narrative self-disclosures about individual life experiences (frequently used by Ethnographers).
Critical Incidents Technique
Methods of collecting qualitative self-report data - Involves probes about the circumstances surrounding a behavior or incident that is critical to an outcome of interest.
Diaries and Journals
Methods of collecting qualitative self-report data - Respondents are asked to maintain daily records about some aspect of their lives.
Methods of collecting qualitative self-report data - Involves having people use audio-recording devices to talk about decisions as they are making them.
Methods of collecting qualitative self-report data - Is stimulated and guided by photographic images.
Solicited or unsolicited narrative
Methods of collecting qualitative self-report data - Communication of the Internet.
Preparation for In-Depth Interview
a) learn about the language and customs of participants,
b) formulate broad questions
c) make decisions about how to present themselves
d) develop ideas about interview settings
e) take stock of equipment needs.
Conducting Good In-Depth Interview
a) considerable interviewer skill in putting people at ease
b) developing trust
c) listening intently
d) managing possible crises in the field.
In-Depth Self-Report Methods
Advantages - Yield data of considerable richness and are useful in gaining an understanding about little-researched phenomena
Disadvantage - They are time-consuming and yield a large volume of data that are challenging to analyze.
Unstructured observational data collected by qualitative researchers sometimes. Participant observers obtain information about the dynamics of social groups or cultures within members' own frame of reference.
Phases of Participant Observation
1) Are primary observers and active listening - getting a preliminary understanding of the site
2) As time passes, researchers become more active participants
3) Observations tend to become more focused over time, ranging from DESCRIPTIVE OBSERVATION to FOCUSED OBSERVATION and then to SELECTIVE OBSERVATIONS.
Participant observations that are more carefully selected events or interactions.
Participant observation where events are to be observed through a fixed location.
Participant observation where events to be observed are done by moving around the site to observe different locations.
Participant observation where events are to be observed through following a person around a site.
Methods for Recording Unstructured Observational Data
Logs and field notes. Field notes are both descriptive and reflective.
Also called Observational notes - are detailed, objective accounts of what transpired in an observational session. Observers strive for detailed, thick description.
a) Methodologic Notes - that document observers' thoughts about their strategies.
b) Theoretical Notes/Analytic Notes - that represent ongoing efforts to make sense of the data
c) Personal Notes - Document observers' feelings and experiences.
Produce rich descriptions of the experience under study - describe from the inside, like a state of mind. Focus on a particular example or incident, try to focus on an example that stands out for its vividness, Attend to hwo the body feels, how things smelled and sounded.
Grounded Theory Interview
Techniques change as theory develops. Initially are similar to open-ended conversations using unstructured interviews. Later ask more direct questions.
Unstructured - 3 types of questions are used to guide interviews -
1) Descriptive - ask to describe their experiences in their own language (backbone of ethnographic interviews.
2) Structural Questions - more focused and help to develop the range of terms in a category or domain.
3) Contrast Questions - are asked to distinguish differencesin the meaning of terms and symbols.
Qualitative researchers use to gather personal recollections of events and their perceived causes and consequences. Typically focus on describing important themes rather than individuals. Are a method for connecting individual experiences with broader social and cultural contexts.
Evolving Participant Observer's Role
(Leninger & McFarland 2006)
1) Primarily observation and active listening.
2) Primarily observation with limited participation
3) Primarily participation with continued observation.
4) Primary reflection and reconfirmation of findings with informants.
Windshield Survey (Windshield Tour)
Many times used by Ethnographers which involves an intensive exploration to "map" important features of the community under sudy. Mapping include documenting community reesources, community liabilities, and social and environmental characteristics.
Unstructured Observational Data
1) The physical setting
2) The participants
3) Activities and interactions
4) Frequency and duration
5) Precipitating factors
7) Intangible factors.
Structured Data Collection Methods
Place constraints on both those collecting the data and those providing it.
Closed-ended (Fixed Alternative) Questions
Offer response options from which the respondents must choose.
Type of closed-ended question which respondents are asked to select a statement that best represents their view.
Type of closed-ended question which respondents are asked to rank a list of alternatives along a continuum.
Type of closed-ended question which require respondents to choose between two competing positions.
Type of closed-ended question which ask respondents to make judgments along an ordered, bipolar dimension.
Checklists or Matrix Questions
Type of closed-ended question which several questions requiring the same response format are listed.
Visual Analogue Scales (VAS)
Type of closed-ended question which are continua used to measure subjective experiences such as pain.
Composite Psychosocial Scales
Multiple item self-report tools for measuring the degree to which individuals posses or are characterized by target traits or attributes.
Comprise a series of statements worded favorably or unfavorably toward a phenomenon. Respondents indicate a degree of agreement or disagreement with each statement; a total score is computed by the summing item scores, each of which is scored for the intensity and direction of favorability expressed.
Semantic Differentials (SD)
Consist of a series of bipolar rating scales on which resondents indicate their reaction toward some phenomenon; scales can measure an evaluative (e.g. good/bad), activity (e.g. active/passive), or potency (e.g. strong/weak) dimension.
Measuring various aspects of cognitive functioning, including intelligence, aptitude or neuropsychological functioning.
People sort a set of card statements into piles according to specified criteria. Can be used to measure attitudes, personality, and other psychological traits.
Brief descriptions of an event or situation to which respondents are asked to react. They are used to assess respondents' hypothetical behaviors, opinions, and perceptions.
ADV over Interviews -
1) Less costly and time consuming than interviews
2) Offer the possibility of anonymity
3) Run no risk of Interviewer Bias
1) Interviews tend to yield higher response rates
2) Be suitable for a wider variety of people
3) Yield richer data than questionaires
1) Put respondents at ease
2) Build rapport with them
3) Need to be skillful at probing for additional information when respondents give incomplete or irrelevant responses.
1) Group administration - Most convenient and economical way to distribute questionnaires.
2) Mail - risk low response rates, which can result in biased sample
3) Internet Distribution - by an email attachment or as a web-based survey that participants access through a hypertext link and respond directly. Internet surveys rarely yield representative samples because response rates are low and not everyone has access to computers.
Response Set Biases
Structured Self-Report bias that stems from the tendency of some people to respond to questions in characteristic ways independent of content.
Extreme Response Set
Results when a person characteristically endorses extreme response alternatives.
Acquiescence Response Set
A YEA-SAYER'S tendency to agree with statements regardless of their content. A converse problem arises when people (NAY-SAYERS) disagree with most statements.
Structured Observational Methods
Impose constraints on observers to enhance the accuracy and objectivity of the observations and to obtain an adequate representation of the phenomena of interest.
Unit of observation - entails observations of larger segments of behaviors as integral units.
Tools for recording the occurrence or frequency of predesignated behaviors, events, or characteristics.
Checklists are based on category systems for encoding observed phenomena in discrete categories. Some checklists categorize exhaustively all behaviors of a particular type (e.g., body movements) in an ongoing fashion, whereas others record particular behaviors while ignoring others.
Another record-keeping device for structural observations. Observers are required to rate phenomena along a dimension that is typically bipolar (e.g., passive/aggressive); ratings are made either at specific intervals (e.g., every 15 minutes) or after observations are completed.
Structured observations that involves the specification of the duration and frequency of both observational periods and intersession intervals.
Structural observations that select integral behaviors or events of a special type for observation.
Have greatly augmented researchers' capacity to collect, record and preserve observational data. Such devices as audiotape recorders and videotape cameras permit behaviors and events to be described or categorized after their occurrence.
One of th most prevalent observer biases.Observational methods are an excellent method of operationalizing some constructs, but are subjectt to various biases. Thje greater the degree of observer inference, the more likely that perceptual errors and distortions will occur.
In Vivo Measurements
Biophysiologic measures that are performed within or on living organisms, like blood pressure measurement.
In Vitro Measurements
Biophysiologic measures that are performed outside the organisms body, such as blood tests.
Advantage of being objective, accurate and precise, BUT CARE MUST BE TAKEN IN USING SUCH MEASURES with regard to practical, technical, and ethical considerations.
Involves the assignment of numbers to objects to represent the amount of an attribute, using a specified set of rules. Researchers strive to develop or use measurements whose rules are isomorphic with reality.
Sources of Measurement Error
Situational contaminants, response-set biases, and transitory personal factors, such as fatigue.
From an instrument consist of TRUE SCORE component and an error component or ERROR OF MEASUREMENT.
Error of Measurement
Error component of measurement that represent measurement accuracies. Added to True score equals obtained score.
One of two primary criteria for addressing a quantitative instrument, is the degree of consistency or accuracy with which an instrument measures an attribute. The higher the reliability of an instrument, the lower the amount of error in obtained scores.
There are many different methods for assessing an instrument's reliability and computation of the reliability coefficient. Is based on the computation of correlation coefficient.
Usually range from .00 to 1.00, with higher values reflecting greater reliability.
Reliability coefficients reflect the proportion of true variability in a set of scores to the total obtained variability.
Indicates the magnitude and direction of a relationship between 2 variables.
Can range from -1.00 (a perfect negative relationship) through zero to +1.00 ( a perfect positive relationship).
Aspect of reliability, which concerns the extent to which an instrument yields the same results on repeated administration, evaluated by TEST-RETEST RELIABILITY.
Internal Consistency Reliability
Refers to the extent to which all the instrument's items are measuring the same attribute. Assessed by using Cronbach's Alpha method.
Interrater (interobserver) Reliability
Estimate of reliability assessment that focuses on equivalence between observers in rating or coding behaviors.
Reliability assessment in which the consensus measure capturing interrater agreement within a small number of categories is desired.
Refers to whether the instrument appears, on the face of it, to be measuring the appropriate construct.
Content Validity Index (CVI)
Expert ratings on the relevance of items can be used to compute the CVI Information.
Using the average calculation method (S-CVI/Ave) are the average of all I-CVI values.
Includes both PREDICTIVE VALIDITY and CONCURRENT VALIDITY. Focuses on the correlation between the instrument and an outside criterion. The instrument is said to be valid if its scores correlate highly with scores on the criterion. The key issue is whether the instrument is a useful predictor of other behaviors, experiences, or conditions.
An instrument's adequacy in measuring the focal construct, is primarily a hypothesis-testing endeavor. Is a key criterion for assessing the quality of a study, and construct validity has most often been addressed in terms of measurement issues.
Construct validation method which contrasts scores of groups hypothesized to differ on the attribute
Construct validation method that is a statistical procedure for identifying unitary clusters of items or measures. Identifies dimensions underlying a central construct. Each dimension is called a FACTOR. The procedure is used to identify and group together different items measuring an underlying attribute. Confirmatory Factor Analysis.
Multitrait-Multimethod Matrix Technique (MTMM)
Construct validation approach which is based on the concepts of convergence and discriminability.
Refers to the evidence that different methods of measuring the same attribute yield similar results.
Refers to the ability to differentiate the construct being measured from other, similar concepts.
Of a new instrument is usually undertaken to gather evidence about vailidy, reliability, and other quality criteria.
Important criteria for screening and diagnostic instrument. Is the instrument's ability to identify a case correctly (i.e., its rate of yielding true positives)
Important criteria for screening and diagnostic instrument. is the instrument's ability to identify noncases correctly (i.e., its rate of yielding true negatives).
Positive Predictive Value (PPD)
Predictive values are posterior probabilities - the probability of an outcome after the results are known. PPV is the proportion of people with a positive result who have the target outcome or disease.
Negative Predictive Value (NPV)
Predictive values are posterior probabilities - the probability of an outcome after the results are known. NPV is the proportion of people who have a negative test result who do not have the target outcome or disease.
Summarizes the relationship between specificity and sensitivity in a single number. The likelihood ratio addresses the question, "how much more likely are we to find that an indicator is positive among those with the outcome of concern compared with those for whom the indicator is negative?
Receiver Operating Characteristic Curve (ROC Curve)
Sensitivity is plotted against specificity to determine the optimum cutoff point for caseness. The score (cutoff point) that yields the best balance between sensitivity and specifricity can then be determined. The optimum cutoff is at or near the shoulder of the ROC curve.
Type of criterion-related validity refers to the adequacy of an instrument in differentiating between people's performance on a future criterion (i.e. incoming student and high school grades).
Type of criterion-related validity refers to an instrument's ability to distinguish individuals who differ on a present criterion. i.e. psych test used to differentiate those patients who may or may not be released from mental hospital.
Method of construct validation involves testing hypothesized relationships, often on the basis of theory. i.e. According to theory, construct X is positively related to construct Y.
Instrument A is a measure of construct X; instrument B is a measure of construct Y.
Scores on A and B are correlated positively, as predicted.
Therefore, it is inferred that A and B are valid measure of X and Y.
Other criteria for assessing quantitative measures. Use of Spearman-Brown formula to estimate how reliable the scale would be with fewer items. Efficiency is more characteristic of certain types of data collection procedures than others.
Other criteria for assessing quantitative measures. Subjects and researchers should be able to comprehend the behaviors required to secure accurate and valid measures.
Other criteria for assessing quantitative measures. An instrument should discriminate between people with different amounts of an attribute as precisely as possible.
Other criteria for assessing quantitative measures. For most instruments, researchers should allow adequate time to obtain complete measurements without rushing the measuring process.
Other criteria for assessing quantitative measures. The instrument should be capable of achieving a meaningful measure from the smallest expected value of the variable to the largest.
Other criteria for assessing quantitative measures. A researcher normally strives to construct measurres that are equally accurate and sensitive over the entire range of values.
Other criteria for assessing quantitative measures. The instrument should, insofar as possible, avoid affecting the attribute being measured.
Sound conceptualization of the construct to be measured, including its dimensionality is the beginning of scale development.
Decide on type of scale then items to be generated
Common sources for items include existing instruments, the research literature, concept analyses, in-depth studies, focus groups and clinical observations.
Domain Sampling Model
Classical measurement theory, this is often assumed in which the basic notion is to sample a homogeneous set of items from a hypothetical universe of items.
a) How many items to generate
b) What to use as the continuum for the response options
c) How many response options there should be
d) Whether to include positive and negative item stems in the scale
e) How intensely worded the items should be
f) What to do about references to time.
After items are generated, they should be inspected for clarity, length, inappropriate use of jargon and good wording' the scale's readability should also be assessed.
External Review of preliminary pool of itesm
Should be undertaken, including refiew by members of the target population.
Should be built into the scale through careful efforts to conceptualize the construct and through content validation procedures by a panel of experts - including the calculation of a quantitative index such as the CVI to summarize the experts' judgments of the relevance of the scale items.
Administer scale to a development sample.
After content validity has been established at a satisfactory level, the scale must be administered to a development sample - Typically 200 or more respondents who are representative of the target population.
Data collected from the development sample are then analyzed using this technique (e.g., a scrutiny of inter-item correlations and item-scale correlations.
Exploratory Factor Analysis EFA)
Data collected from the development sample are then analyzed using this technique. Is used to reduce a large set of variables into smaller set of underlying dimensions, called factors. Mathematically, each factor is linear combination of variables in a data matrix.
Data collected from the development sample are then analyzed using this technique
The first phase of factor analysis identifies clusters of items that are strongly intercorrelated and is used to define the number of underlying dimensions in the items empirically.
2nd phase of factor analysis which enhances the interpretability of the factors by aligning items more distinctly with a particular factor. Rotation can either be orthogonal or oblique.
Factor rotation which maintains the independence of factors. Factors are kept at right angles to one another. Maintain the independence of factors- that is, orthogonal factors are uncorrelated with one another.
Factor rotation which allows intercorrelated factors. Permit rotated axes to depart from a 90-degree angle.
Factor loadings of the items on the rotated factor matrix are then used to interpret and name the factors.
Confirmatory Factor Analysis (CFA)
After the scale is finalized based on the preliminary analyses, a second study is often undertaken to validate the scale, using a variety of validation techniques. CFA is one technique.
CFA involves tests of a measurement model which stipulates the hypothesized relationship between latent variables and manifest variables.
Cut-Off points or norms
Some scales - espeically ones that will be used to make sassessments of diagnoses of individuals- benefit in interpretation through the establishemnt of cut-off points or norms.
Well-constructed scales with good psychometric properties are increasingly likely to be translated for use in other cultures. Translations are often CENTERED on the original language, but a DECENTERED approach, which would allow modifications to the wording of items in the original scale, may be preferred which it is anticipated during the development phase that the scale will be used in two languages.
Both conceptual equivalence and semantic equivalence are critical to the success of translated effort. The "gold standard" for semantic equivalence involves back-translation, in which the scale is first translated from the source language into the target language then translated back to the source language by translators blind to the original wording. The next step typically involves a committee that convenes with the goal of arriving at a consensus translation.
Evidence for semantic equivalence and psychometric soundness comes from the pretests of both original and translated scales with a sample of bilingual people and from comparison of reliabilities, factor structures and other validity estimates between the 2 scales.
Asymmetric - the scale is translated into another language, with no effect on the wording of the original instrument - loyalty to the original scale item is maintained. - usually occur after scales has been validated and used.
Involves the possibility of modification to items on the original scales. Such decentered (or symmetric) translations often reflect the goal of replacing culturally exclusive language with more universally understood language.
Structural Equation Modeling (SEM)
CFA is a subset of an advanced class of statistical techniches. LISREL (Linear STructural Relation Analysis). Method for anlyzing covariance in structures.
Maximum Likelihood Estimation
SEM the most frequently used estimation procedure - Maximum likelihood estimators are ones that estimate the parameters most likely to have generated the observed measurements.
CFA involves the testing of a measurement model, which stipulates the hypothesized relationships among underlying latent variables and the manifest variables (the items).
The anlyaisi would provide loadings of the observed variables on the latent variables, the correlation between the two latent variables and the correlations between the error terms. The anlaysis would also indicate whether the overall model fit is good based on a goodness-of-fit statistic.