How can we help?

You can also find more resources in our Help Center.

Face validity

-Not recognized by all theorists

-"on the face' of it approach

-"on the face' of it approach

Content validity

Answers the question: How much a measure covers the meanings within a concept?

Construct Validity

-Logical relationships among variables

-Offers evidence of whether your measure does or does not address the construct

-Offers evidence of whether your measure does or does not address the construct

Criterion Validity

-Degree to which a measure relates to some exterior criterion

-Sometimes known as predictive validity

-I.e. PCAT scores

-Sometimes known as predictive validity

-I.e. PCAT scores

Concurrent Validity

-Degree to which a measure relates to other similarly validated tests

-E.g. The Test of Functional Health Literacy (TOFHLA) was compared to REALM

-E.g. The Test of Functional Health Literacy (TOFHLA) was compared to REALM

External Validity

-Generalizability

-Test results across different populations

-Applicability of results

-Test results across different populations

-Applicability of results

Generalizability theory

-Introduced by Cronbach and colleagues

-Bridging reliability and validity

-Use of ANOVAs

-Item difficulty vs. Person by item interaction

-Bridging reliability and validity

-Use of ANOVAs

-Item difficulty vs. Person by item interaction

Survival Analysis

Studies that involve prospective analysis i.e. following individuals from one point in time to outcome (usually death)

-Can be used for any end-point/outcome: e.g. smoking cessation, tumor recurrence etc.

-End-point and follow-up time must be documented

-Can be used for any end-point/outcome: e.g. smoking cessation, tumor recurrence etc.

-End-point and follow-up time must be documented

Censoring

-value of a measurement or observation is only partially known

-don't have exact event time

-don't have exact event time

Reasons for having censored data

-Subject does not experience event before study ended

-Loss of follow-up

-Withdrawal from study

-Loss of follow-up

-Withdrawal from study

Life table analysis

-Can be thought of as enhanced frequency distribution table

-Divide observation interval into smaller periods of time to estimate the outcome during each period

-Divide observation interval into smaller periods of time to estimate the outcome during each period

Kaplan-Meier Curves

-Similar to life tables

-Uses exact times that events occurred instead of intervals of follow-up

-P(e) = Number of events at that time divided by Number at risk at that point in time

-Uses exact times that events occurred instead of intervals of follow-up

-P(e) = Number of events at that time divided by Number at risk at that point in time

Cox proportional hazards model

-Uses proportional hazards

-Key to interpretation is use of odds ratios or in some cases, hazard ratios

-Key to interpretation is use of odds ratios or in some cases, hazard ratios

Problems with open ended questions in surveys

-Need to be coded for analysis

-Coding is interpretation and may lead to researcher -bias

-Irrelevant answers may be a problem

-Coding is interpretation and may lead to researcher -bias

-Irrelevant answers may be a problem

Important considerations for closed ended questions in surveys

-Important that categories be mutually exclusive

-For multiple responses to a problem - coding difficulties arise

-For multiple responses to a problem - coding difficulties arise

Key factors of question construction in surveys

I. Make items clear

II. Avoid double barreled questions:

III. Respondents must be competent to answer

IV. Respondents must be willing to answer

V. Questions should be relevant

VI. Short items are preferable

VII. Avoid negative items:

IX. Avoid biased items and terms

II. Avoid double barreled questions:

III. Respondents must be competent to answer

IV. Respondents must be willing to answer

V. Questions should be relevant

VI. Short items are preferable

VII. Avoid negative items:

IX. Avoid biased items and terms

Thurstone Scaling

-One of the first scaling theorists

-Method of equal appearing intervals, successive intervals and paired contrasts

-Method of equal appearing intervals, successive intervals and paired contrasts

Likert Scaling

-Develop the focus: e.g. generate statement about specific attitudes that people may have toward

-Item generation: 1-5 or 1-7, Agree to Disagree scale

-Rate items: group of judges rate items from strongly unfavorable to strongly favorable to concept

-Select items: throw out unfavorable items (may use item correlation - easily done in SPSS)

-Administer items: 5 point likert scale

-Item generation: 1-5 or 1-7, Agree to Disagree scale

-Rate items: group of judges rate items from strongly unfavorable to strongly favorable to concept

-Select items: throw out unfavorable items (may use item correlation - easily done in SPSS)

-Administer items: 5 point likert scale

Guttman Scale

-Cumulative scaling

-Develop focus e.g. attitudes of patients toward immigrant physicians

-Develop items: e.g. I would feel comfortable discussing my health problems with an immigrant physician

-Rate items

-Develop cumulative scale: Construct matrix

-Develop focus e.g. attitudes of patients toward immigrant physicians

-Develop items: e.g. I would feel comfortable discussing my health problems with an immigrant physician

-Rate items

-Develop cumulative scale: Construct matrix

Reliability

reproducibility, repeatability or consistency

Inter-rater reliability

-Aka Inter-observer reliability

-To check for consistency from one observer to another

-To check for consistency from one observer to another

For measuring reliability a general rule of thumb is ______indicates good level of agreement

Kappa >0.6

Test-retest reliability

-Administer test on two different occasions

-Time is important: shorter time gap - higher correlation and vice versa

-Time is important: shorter time gap - higher correlation and vice versa

Parallel forms reliability

-Create two parallel forms

-Correlation indicates level of reliability

-Limitation: need a large set of items

-Correlation indicates level of reliability

-Limitation: need a large set of items

Statistical Decision Theory (SDT)

This answers the question: How often are we making good decisions?How often are we making bad decisions?

Type I Error or alpha

When you reject the Null Hypothesis but it is actually true Probability you say BS when H0 is really true

Type II error or beta

Accept the Null Hypothesis when its False (Say nothing is going on when something really is going on)

Power

1 minus beta

Basic Factors that Influence Power

A) Size of the difference between means

B) Significance Levels

C) Sample Size

D) Variance or standard deviation

E) Experimental Design

B) Significance Levels

C) Sample Size

D) Variance or standard deviation

E) Experimental Design

Why do large samples increase power?

1) You decrease the "appropriate sd"sdiff, sm, sm, etc

2) You reduce the effects of outliers and sampling error (central limit theorem) less likely to get weird sample with bigger n

2) You reduce the effects of outliers and sampling error (central limit theorem) less likely to get weird sample with bigger n

How do you decrease variance in your sample

1) Increase Sample Size

2) Decrease Error in your Research Design

3) Use a Homogeneous sample

2) Decrease Error in your Research Design

3) Use a Homogeneous sample

Small effect size

d = .20

Medium effect size

d = .50

Large effect size

d = .80

Kappa

Po - Pe / (1-Pe)

Total sum of squares

measures the total scatter of scores around the grand mean. The grand mean is the mean of all the subject's scores regardless of the group to which they belong.

Between groups sum of squares

measures the total scatter of the group means with respect to the grand mean.

Within Group sum of squares

measures the scatter of scores within each group with respect to the mean of that particular group. .

df in Anova between groups

Number of groups - 1

df in Anova within groups

Total number of people - total number of groups

df Total in Anova

Total number of people - 1

F is calculated this way

MSBW/MSWG

Post hoc analysis

-Test all pairs of means to see if they are diff from each other

-Done only after you find there is a significant difference in ANOVA

-Done only after you find there is a significant difference in ANOVA

You need this information to find the q value in post hoc analysis

df of between groups and number of means in ANOVA

Two-Way ANOVA

-Uses F statistics to compare the differences among means for 2 independent variables (categorical)

-Interested in the effects of multiple independent variables on a dependent variable

-Interested in the effects of multiple independent variables on a dependent variable

Assumptions for Two-Way ANOVA

1) Interval or Ratio Data

2) Subject must be randomly assigned to cells

3) Populations from which samples are drawn must be n normally distributed

4) Populations from which samples are drawn must have equal variance

2) Subject must be randomly assigned to cells

3) Populations from which samples are drawn must be n normally distributed

4) Populations from which samples are drawn must have equal variance

df when setting up t- in a correlation

df = N-2

Coefficient of Determination = r2

-tells us how much of the variance in one variable is accounted for by the variance on another variable.

-What do changes in X tell us about the value of Y.Essentially the "effect size" for the relationship between 2 variables

-What do changes in X tell us about the value of Y.Essentially the "effect size" for the relationship between 2 variables

Phi Coefficient

Use this stat if X is categorical Y is categorical for correlation

Point Biserial Correlation

Use this stat if X is categorical Y is continuous

Spearman's Rank order Correlation Coefficient

Use this stat if X is ordinal and Y is ordinal for correlation

Homoscedasticity

This assumption means that the

variance around the regression

line is the same for all values of

the predictor variable (X).

variance around the regression

line is the same for all values of

the predictor variable (X).

Two approaches to research logic

Deduction - scientific method

Induction - start from observations

Induction - start from observations

Types of qualitative research

Case Study

Grounded Theory

Phenomology

Ethnography

Historical

Grounded Theory

Phenomology

Ethnography

Historical

Case Study

Can be used to study a complex issue or extend experience and knowledge about a particular process

Emphasis is on detailed contextual analysis

Has often been dismissed by critics as a exploratory tool

Emphasis is on detailed contextual analysis

Has often been dismissed by critics as a exploratory tool

Grounded Theory

-Theory developed inductively from data

-Case perspective approach

-Cases that are similar on many variables are compared to reveal necessary causes

-Methods: May include reading large volumes of textual data, observed behavior etc.

-Case perspective approach

-Cases that are similar on many variables are compared to reveal necessary causes

-Methods: May include reading large volumes of textual data, observed behavior etc.

Phenomology

Field of philosophy

Study of structures, experience or consciousness

In other words, the study of phenomena

Studies in the first person to describe a wide range of experiences/emotions

Study of structures, experience or consciousness

In other words, the study of phenomena

Studies in the first person to describe a wide range of experiences/emotions

Ethnography

Field of Sociology/ Cultural anthropology

Close field observation of socio-cultural phenomena

Ethnographer: one who conducts ethnographic studies

Works in a community, selects informants to interview, interviews informants multiple times etc.

Close field observation of socio-cultural phenomena

Ethnographer: one who conducts ethnographic studies

Works in a community, selects informants to interview, interviews informants multiple times etc.

Historical research

Identifying problem of interest that is historical

Conducting data collection and analysis based on interviews, examination of historical documents, biases in previous information collected

Quantitative facts if any

More than one point of view

Conducting data collection and analysis based on interviews, examination of historical documents, biases in previous information collected

Quantitative facts if any

More than one point of view

Qualitative Research Methods

Participant Observation:

Direct Observation:

Unstructured Interviewing

Direct Observation:

Unstructured Interviewing