Upgrade to remove ads
Only $35.99/year

Research Methods: Chapter 13

Terms in this set (94)

Inferential statistics allow researchers to (3) and question.
1. assess likelihood that their findings would still occur if their study was repeated over and over
2. make inferences about the true difference in the population on the basis of the sample data
3. give the probability that the difference between means reflects random error rather than a real difference
- can we infer that the difference in sample means = true difference in population means?
inferential statistics are necessary because results of a given study are based only on data obtained from a _____ _____ of participants
single sample
Why will there always be some difference in the sample means even when all principles of experimental design are rigorously followed?
Because dealing with samples not populations. Therefore random or chance error will be responsible for difference in means even if IV had no effect on DV
Null hypothesis
- population means are equal - the observed difference is due to random error
- Assume that there is no difference between the populations from which the samples were drawn
- states IV had no effect
Alternative hypothesis (research hypothesis)
- population means are not equal.
- states IV did have an effect
- there is a difference between the populations (the drug had an
effect)
Logic of the null hypothesis: If we can determine that the null hypothesis is incorrect, then we ...
accept the alternative hypothesis as correct.
- acceptance of the alternative hypothesis means the IV had an effect on the DV
Statistical significance
- indicates that there is a low probability that the difference between the obtained sample means was due to random error
Probability
likelihood of the occurrence of some event or outcome
want to specify the probability that an event will occur if there is no difference in the population. question is, what is the probability of obtaining this result if only random error is operating?
- if the probability is very low, we reject the possibility that only random chance or error is responsible for the obtained difference in means
alpha level
- probability required for significance
- most common is .05
.05 alpha
- outcome of the study is considered significant when there is a .05 or less probability of obtaining the result; that is, there are only 5 out of 100 chances that the results were due to random error in one sample from the population
If it is very unlikely that random error is responsible for obtained results, the null hypothesis is ____
rejected
sampling distributions
- probability distribution based on the assumption that the null hypothesis is true
when obtained results are highly unlikely if you are sampling from the distribution specified by the null hypothesis, you conclude that the null hypothesis is _____
incorrect
sample size
total number of observations on determinations of statistical significance
As your sample size increases, you are more confident that your outcome is actually different from the
null hypothesis expectation
t test
- commonly used to examine whether two groups are significantly different from each other
- Tests the significance of the difference between sample means
- ex. experiment asking whether the mean of a group differs from another group
To use a statistical test you must .. (2)
1. specify the null and alternative hypothesis
2. specify LOS you will use to decide whether to reject null hypothesis
If the obtained t has a low probability of occurrence, ex. 05 or less, then then null hypothesis is
rejected
t value ratio =
group difference/within-group variance
- group difference is difference between your obtained means (under null hypothesis you'd expect this diff to be zero)
- within-group variance is amount of variability of scores about the mean (denominator essentially indicator of amount of random error in your sample)
value of t increases as the difference between your ______ ______ _____ increases
obtained sample means
Ex. using a LOS of .05, the critical value from the sampling distribution of t is 2.101.
- obtained value is 4.02 so...
because obtained value is larger than the critical value, we can reject the null hypothesis and conclude that the difference in means obtained in the sample reflects a true difference in population
degrees of freedom
- number of scores free to vary once the means are known
when comparing two means, you assume the DOF are
equal to n1 + n2 - 2
one-tailed test
research hypothesis specified a direction of difference between the groups (group one will be
two-tailed test
research hypothesis did not specify a predicted direction of difference (ex. group 1 will differ from group 2)
Whether to specify a one-tailed or two-tailed test will depend on whether you...
originally designed your study to test a directional hypothesis
Analysis of variance (F test)
- extension of the t test
- more general statistical procedure
- used when there are more than two levels of an IV and when a factorial design with two or more IV have been used
F test is appropriate for the ______ experimental design, as well as the more _____ designs
simple; complex
what is more a common procedure, t test or F test
F test
F statistic is a ratio of two types of variance:
1. systematic variance
2. error variance
systematic variance
- (between group variance) variability of scores between groups
- deviation of the group means from the grand mean, or the mean score of all individuals in all groups
- systematic variance is small when the differences between group means is small and increases as the group mean differences increase
Error variance
- variability of scores within groups
- the deviation of the individual scores in each group from their respective group means
the larger the F ratio is, the more likely that the results are
significant
effect size r (used for a t test) is a type of correlation coefficient that can range from
0.00 to 1.00
- where .69 is considered a alrge effect size
Cohen's d
- another effect size estimate used when comparing two means
- expresses effect size in terms of SD units
- d value of 1.0 tells you that the means are 1 SD apart, a d of .2 indicates means are separated by .2 SD
Effect size: Cohen's d and r provide info on the
size of the relationship between variables studied
Both effect size estimates have a value of ____ when there's no relationship. The value of r has a maximum value of ______ but d has ______
- 0.00
- 1.00
- no maximum value
Confidence intervals
- interval of values defines the "most likely" range of actual population values
- ex. a 95% CI indicates we are 95% sure that the population value lies within the range, 99% interval would provide greater certainty but range of values would be larger
Size of interval is related to both
1. size of sample
2. confidence level
- as sample size increases, confidence interval narrows
higher confidence is associated with a
larger interval
If you want to be almost certain that the interval contains the true population mean (ex. 99% CI) you will need to include more _____
possibilities
Statistical significance overview (4)
1. the goal of the test is to allow you to make a decision about whether your obtained results are reliable
2. the LOS you choose indicates how confident you wish to be when making the decision (a LOS of .05 says you're 95% sure of the reliability in your findings however there's a 5% chance you could be wrong)
3. You are most likely to obtain significant results when you have a large sample size because larger sample sizes provide better estimates of true population values
4. you're most likely to obtain significant results when the effect size is large (ex. when differences between groups are large and variability of scores within groups is small)
Type I Error
- made when we reject the null hypothesis but the null hypothesis is actually true
- so decision is that the population means are not equal when they actually are equal
- IV had no effect but you conclude it did
- FALSE ALARM
Type I errors occur when, simply by chance, we obtain a large value of ___ or ___
- t or F
error is determined by the choice of significance or _____ level
- alpha
- so when significance level for deciding whether reject null hypothesis is .05, probability of a type I error (alpha) is .05
Probability of making a Type I error can be changed by either decreasing or increasing the
LOS
- if we use a lower alpha level of .01, there is less chance of making a Type 1 error
Type II error
- occurs when the null hypothesis is accepted although in the population the research hypothesis is true
- population means are not equal, but the results of the experiment do not lead to a decision to reject the null hypothesis
- IV had an effect but you conclude that IV had no effect
- MISS
Type II error probability is called
beta
Probability of making a type II error is related to three factors:
1. LOS; if we set a low significance level to decrease the chances of a Type I error, we increase the chances of a Type II error (if we make it very difficult to reject null hypothesis, the probability of incorrectly accepting the null hypothesis increases)
2. sample size: true differences are morel likely to be detected if the sample size is large
3. effect size: if the effect size is large, type II error is unlikely. a small effect size may not be significant with a small sample
Examples of how Type I and Type II errors occur in everyday context
- can be applied to decisions people frequently make in everyday life
- juror in criminal trial
- decision that a doctor makes to operate or not operate
- (type I error is more serious than a Type II error for a juror but for a doctor Type II error may be more serious)
Significance level chosen by researchers usually is dependent on the consequences of making a Type __ error versus a type __ error
I; II
Is the consequences of making a Type I or Type II error more serious?
- researchers generally believe that the consequences of making a Type I error are more serious than those associated with a Type II error
- ex. If null hypothesis is rejected, researcher might publish results in a journal and the results might be reported by others in textbooks or in newspaper or magazine articles. (publish false results) Where the consequences of a Type II error are not seen as being very serious. So researchers want to be very careful to avoid Type I errors when their results may be published but under certain circumstances they're not as serious.
You shouldn't accept the null hypothesis just because
the results are nonsignificant; nonsignificant results do not necessarily indicate that the null hypothesis is correct
Must be circumstances in which we can accept null hypothesis and conclude that the two variables are in fact not related: (3)
1. look for well designed studies with sensitive dependent measures and evidence from a manipulation check that the IV manipulation had its intended effect
2. research should have reasonably large sample to rule out possibility that the sample was too small
3. evidence that the variables are not related should come from multiple studies
power of the statistical test
-related to probability of Type II error
- Power = 1 - p (type II error)
Type II error is related to (3)
1. LOS
2. sample size
3. effect size
Researchers usually use a power between .___ and .___ when using this method to determine sample size
.70 and .90
importance of power analysis
if researcher is studying a relationship with an effect size correlation of .20, a fairly large sample size is needed for statistical significance at the .05 level. An inappropriately low sample size in this situation is likely to produce a non-significant finding
rich understanding of any phenomenon comes form the results of _____ studies investigating the same variables
numerous
Computer analyses of data
1. input the data
- data entered into columns
- data for research participant are the rows of the matrix
- columns contain each participants score on one or more measures and an additional column may be needed to indicate a code to identify which condition the individual was in
2. provide instructions for statistical analyses
- each program uses somewhat different steps
Bivariate Research
- research studying two variables and whether they are related or not
- can be IV and DV or variable X and Y
Testing the null hypothesis question:
are the sample means significantly different?
(example)
- Null hypothesis:

- Alternative (Research) hypothesis:

-

- if the probability of obtaining our result when the null
hypothesis is true (p-value) is less than .05, _____ the
null hypothesis

- Test ___ at the .05 level of significance
- Null hypothesis:
H0: μE = μC

- Alternative (Research) hypothesis: H1: μE ≠ μC

- Test the null hypothesis

- if the probability of obtaining our result when the null
hypothesis is true (p-value) is less than .05, reject the
null hypothesis

- Test H0 at the .05 level of significance
(t-test) compute t based on (3)
1. sample means
2. SD's
3. sample sizes
What is the probability of obtaining that value of t by chance if the samples were drawn from identical populations?
- make decision based on p-value
For any statistic we compute (t, F, etc.) , we can compute it's ___ _______
p value
p-value
- Probability of obtaining that value or a more extreme value of the statistic if the null hypothesis is true.
- probability of obtaining tobs by chance
Making decisions about statistical significance
When doing a t-test in SPSS: (4)
- Select alpha (level of significance): .05
- Enter your data in SPSS, click on t-test
- SPSS computes tobs and tells you the probability of obtaining
tobs by chance (p-value)
- Determine if the p-value (probability of obtaining tobs by chance) is less than alpha
If you are doing a one-tailed
(directional) test how do you obtain p
divide p by 2
What can you conclude if p < .05
the sample means are
significantly different (IV had an effect)
What can you conclude if p > .05?
the sample means are not
significantly different (IV had no effect)
In SPSS, there are different t-tests for _________ _______ and _______ _______ - interpret the __ _____ the same way
- independent groups and repeated measures
- p-value
probability of Type I error =
alpha (α) = LOS
probability of Type II error =
beta (β)
probability of correctly deciding H0 is false =
1 - β = power
Ex. type I and type II error
- Researcher believes new teaching method can improve math ability
in grade school children more than the current teaching method

- H0: New teaching method is no better than old one
- H1 : New teaching method is better than old one

- Type I error: Researcher concludes that new teaching method is better than old one
In reality, there is no difference between old and new teaching methods (false alarm)

- Type II error: Researcher concludes that new teaching method is no better than old one
In reality, the new teaching method is more effective than the old method (miss)
What type of error is worse?
- If you make a Type I error and conclude that new teaching
method is better but it's really the same, there is no cost

- If you make a Type II error and conclude that new teaching method is no better, you lose out on a beneficial teaching method
Type II error seems worse than Type I error

- BUT, what if the new teaching method is expensive? A Type I error is more costly, maybe worse than a Type II error
t-tests (2)
- independent-groups t test
- related-samples t test
ANOVA (2)
- independent-groups ANOVA (IV manipulated between groups)
- related-samples/repeated measures ANOVA (IV manipulated within subjects)
chi-square
compare counts for nominal data
correlation
describe relationship between two continuous variables
Effect size r equation
√(t^2)/(t^2 + df)
df (Degrees of freedom) are related to ______ _____. df for paired samples t test and independent samples
sample size
- df = N - 1 for paired samples t test
- df = N1 + N2 - 2 for independent samples
Effect size increases as ____increases
t
Small, medium, and large effect size
small: r = .15
medium:r = .30
large: r = .40

0 ≤ r ≤ 1
Interpreting non-significant results (we fail to reject H0 rather than to accpet H0)
...
Reasons results may be non-significant even though H0 is false (Type II Error): (2)
1. Level of significance (α, probability of a Type I error) is very low...Increases probability of a Type II error
2. Sample size is too small for effect size
A statistically significant result has little practical significance when (4)
- the study has poor external validity
- the effect size is very small
- the treatment is too costly to implement
- the effect size is comparable to that for an existing treatment
If you perform multiple statistical tests, there is a high probability that, by chance, at least one result will be _________ __________
statistically significant
Hypothesis testing
probability of committing a Type I error (false alarm) when LOS is .05
Multiple tests
Probability of committing a Type I error in AT LEAST ONE OF TWO TESTS, each at the .05 level
If you repeatedly obtain significant results (you reject H0) in replications of a study, it is VERY unlikely that you are committing a
Type I error
Replication
Probability of committing a Type I error in BOTH TESTS when you perform two tests, each at the .05 level
Recommended textbook explanations

A Survey of Mathematics with Applications

10th EditionAllen R. Angel, Christine D. Abbott, Dennis C. Runde
6,365 explanations

Probability

1st EditionJim Pitman
464 explanations

The Practice of Statistics for the AP Exam

5th EditionDaniel S. Yates, Daren S. Starnes, David Moore
2,433 explanations

A First Course in Probability

9th EditionSheldon Ross
944 explanations