RE: SG018 primary endpoints17 Jan 2022 12:16
Hi Matterhorn.
The bit on statistical significance isn't quite right.
In a nutshell, the p-value is the probability that the results were a fluke. In this case 0.05. The 95% significance level is asking the question, if the trial was repeated 100 times, would you draw the same conclusion at least 95 times?
You do not need 95% of the sample to show improvement to be 95% confident that the data is not a fluke. You could for example be 95% confident that 20% of the sample would show improvement.
The smaller the % of the population showing improvement, the larger the sample needed for it to be statistically significant.
Take the flip of a coin:
n=number of flips h=heads t=tails.
n=2 t=0 h=2 Would not be statistically significant at the 95% level to say that heads and tails had different probabilities. The probability of this event is 0.25 (>p=0.05).
However,
n=1000 t=600 h=400 would be statistically significant. You would be able conclude that tails was more likely than heads given a 95% significance level. (I haven't worked out the exact probability of 600 or more tails out of 1000 tosses, but trust me it will be very small. Much less than 0.05).