Someone recently challenged me to write the short version to the question ‘What is Hypothesis Testing ?’ so it can be used by someone in a practical way. Don’t get too caught up in the definition of the word ‘Hypothesis’ … I think we all know a Hypothesis when we see one ! It is of course some sort of statement made as an explanation, or proposal or what we suppose the truth to be….
Hypothesis Testing tells us the Probability that the Evidence we have is Sufficient to support our Hypothesis or not.
The Evidence is the data that we collect or measure to support our Hypothesis. Without data as Evidence then we cannot do a Hypothesis Test.
The Hypothesis Test is the mathematical analysis of the data which tells you the Probability that you would be Wrong to accept the Hypothesis. The mathematical analysis can be done with one of ( approx) 12 different analysis tests that are all different types ( ‘flavors’ ) of test. Some examples of the names of these analysis tests are ‘1-sample t-test’, ‘Moods-Median Test’ or ‘F-test’, to name just 3. The correct analysis test to choose depends on the type of data you have and the form of your Hypothesis. ( To make the correct choice of test needs more understanding and practice ).
The Probability number that is produced by the mathematical analysis is called a ‘p-value’ and is a decimal number from 0 to 1. A p-value of 0 would mean that the data indicates that you have a zero percent chance that your Hypothesis is wrong. A p-value of 1 would mean that the data indicates that it is 100% certain that your Hypothesis is wrong. In practice we never actually get p-values of exactly 0 or 1, but we can be very close to those numbers. So depending on your p-value, you make a decision if you want to go ahead and do something in practice…such as spend some money to make a change. Usually we have some Risk Level ( called the alpha level of risk ) that we compare the p-value to, and if the p-value is less than the Risk Level, then we would be prepared to do something in practice or make a change. We are looking for a low p-value, usually less than 0.05, which would indicate that there is less than a 5% chance that the evidence ( data ) does not support the Hypothesis.
I hope this helps someone, somewhere to have a better understanding, without going into so much details that it become more confusing or the person switches off and starts reading their Facebook ! Please feel free to send me questions if you are still puzzled. —- John
