Hypothesis Testing

Outline the logical basis for a hypothesis test
Click the card to flip 👆
1 / 36
Terms in this set (36)
Clearly if we had seen 5 HIV cases in the vaccine group and 120 in the placebo group we would conclude the vaccine worked. Alternatively if we had seen 62 HIV cases in the vaccine group and 61 in the placebo group, we would conclude the vaccine did not work

However, at what stage should we begin to pay attention to the imbalance? Does the observed 51 versus 74 split supply evidence that the vaccine works, or could these results reasonably be ascribed to chance?

An hypothesis (or significance) test helps us to answer this question and assess the strength of the evidence provided by these data
The research question is first formulated in terms of a null hypothesis

The null hypothesis usually refers to no difference between groups or no association and researchers are generally keen to disprove this

More specifically in a clinical trial, the null hypothesis (H0) may be that the true difference in a parameter (mean or proportion) on treatment or non placebo is 0
The probability of observing a test statistic (result) as more extreme than the observed in your sample, in hypothetical repetitions of the study assuming that the null hypothesis is true

The P-value is compared with the significance level (chosen prior to the study, sometimes referred to as alpha, usually at 5%)

If P<0.05, the test is "statistically significant" at the 5% level and the null hypothesis is rejected

Alternatively if P>0.05, the test is not "statistically significant" at the 5% level and there is insufficient evidence to reject the null hypothesis
The outcome and the compassion being conducted

1. Formulae hypothesis

In a trial, the null hypothesis (H0) often that the true difference in mean/proportion on treatment or on placebo is 0

2. Conduct study

3. Use sample data to calculate a P-value

The P-value: the probability of observing a test statistic (results) as, or more, extreme than that observed in your sample, in hypothetical repetitions of the study assuming that the null hypothesis is true

4. Make decision based upon the P-value
A Type I error

Occurs if the null hypothesis is rejected when it is true

The probability of a Type 1 error, the rejection of a true null hypothesis, equals the significance level (and is therefore usually 5%)
Saying the treatment works when it doesn't

A Type II error

Occurs if the null hypothesis is not rejected when it is false

The calculation of the probability of a Type II error is beyond the scope of this course, but as a given significance level is related to sample, larger studies will have reduced risk of Type II error
A Chi-squared test for a 2x2 table is used to compare proportions between two independent groups

It is commonly used in randomised controlled trials comparing an outcomes with two categories (eg.dead/alive) in a placebo and treatment group

The Null Hypothesis (H0) is that population (or true) proportion in group one equals the population (or true) proportion in in group two (π1=π2)

The Alternative Hypothesis (H1) is that the population (or true) proportion in group one does not equal the population (or true) proportion in group two (π1=/=π2)
Test Statistic:

In this course, we are not concerned with calculating tests by hand but for transparency, the test statistic value is given by

x^2 = E [ (IO-EI - 0.5)^2 / E

...with 1 degree of freedom

Where O is observed cell count and E is the expected cell count (if the null hypothesis was true given by ([row total x column total] / overall total) in each cell of the 2x2 table

To obtain a P-value the test statistic is compared against the Chi-squared distribution with 1 degree of freedom

The test is only considered reliable if expected cell values are greater than 5

Other tests such a Fisher's Exact Test (not covered in this course) are required if this criteria is not met