Calculate p-value

Link: P-value

The basic idea §

P-values are determined by adding up probabilities. It’s used to test null hypothesis e.g. the coin is a fair coin.

1. The “normal” probability of this result. E.g. The chance is 0.25 for getting 2 heads if flipping a coin twice.
2. The probability of getting something else that has the equal chance. E.g. getting 2 tails = 0.25.
3. The probability of getting something more rare. E.g. zero in this case because getting 2 heads/tails are the lowest probability out of all outcome.

p-value = sum of above e.g. 0.5

So we fail to reject the Null hypothesis: the coin is fair. aka the coin is not special.

The intuitive behind the idea §

The 2nd part of p-value: there are other things that are equally special. When theses exist, they make this event less special.

Similar logic applies to 3rd part of p-value: there are even other things that are more special, which also makes this even less special and rare.

Example: Getting 4 heads and 1 tails from flipping coins 5 times §

1. $5/32$: getting 4 heads, 1 tails
2. $5/32$: getting 1 heads, 4 tails
3. $2/32$: getting 5 heads

p-value = 0.375. We still fail to reject the null hypothesis.

Calculate p-value for continous value §

We usually use statistical distribution to calculate probabilities and p-values if it’s continous.

So, we add up the area to calculate p-value. It will tell us if the measurement comes from this distribution.

Example: Measure heights §

Null hypothesis: “someone who’s 142cm could come from this distribution”.

p-value = 0.025 + 0.025 = 0.05

Inconclusive: It’s at the threshold, so it’s hard to say whether it comes from this distribution. They could come from other distribution too.

BTW, the p-value = 0.03 if the height is 141cm and we can reject the hypothesis, which suggests it may comes from other distribution.

One-sided p-value §

1. Decide which direction we want to see change happen.
2. Only use the area that is in the desired direction (non bi-directional probability add-up)

The intuition about the difference between one-sided and two-sided p-value §

Two-sided p-value would detect the unusual events happened on the both end, unlike the one-sided, which only detect one side.

Therefore, one-sided p-value is usually avoided and should be treated carefully.