IN.gov - Skip Navigation

Note: This message is displayed if (1) your browser is not standards-compliant or (2) you have you disabled CSS. Read our Policies for more information.

Indiana State Department of Health

Epidemiology Resource Center Home > Surveillance and Investigation > Surveillance and Investigation Division > Newsletters > Indiana Epidemiology Archived Newsletters > Easy Epidemiology for Everyone Easy Epidemiology for Everyone

Tracy Powell, MPH
ISDH Field Epidemiologist, District 4

The last couple of newsletters described case control and cohort studies.  The next step is looking behind the studies at the p-value and confidence interval.  This month focuses on the p-value.

You might ask about the p-value when discussing which variables or exposures are significant in a study or outbreak investigation.  In public health, the p-value, or probability value, is used to determine if the observed differences between groups (such as ill and not ill) are true differences. Epidemiologists are taught to look at the p-value when evaluating significance and testing hypotheses. Although the significance level is arbitrary, a p-value of less than or equal to 5 percent (0.05) is widely accepted.  Smaller or larger percentages may be used if justification, such as previous studies or clinical significance, is warranted.  For the purpose of this article, a p-value of ≤0.05 will be considered statistically significant, as most public health professionals use this value as a standard. 

The p-value indicates how often you would expect to see the observed outcomes just as a result of chance.  P-values less than or equal to 5 percent are considered “statistically significant”, which means the result is not likely attributable to chance. Events that have a high probability of occurring (p >0.05) are considered more common and, therefore, not significant.  P-values range from 0-1 and are usually expressed as p <0.05. 

P-value Example

This example will assist in understanding how to interpret a p-value.  The table contains hypothetical data on Disease X rates for 2007 per 100,000 people in Indiana by county.  The far right column displays the p-value for the disease rate for each county compared to the disease rate for the state.  A p-value of less than 0.05 indicates a statistically significant difference between the annual disease rate for the county compared to the state rate. 

As the table shows, four counties have p-values less than 0.05.  County 1, County 3, and County 5 have rates that are significantly higher than the state rate.  County 4 has a rate that is significantly lower than the state rate.  County 2 and County 6 do not have statistically significant p-values, p = 0.28 and p = 0.15 respectively.  Therefore, the rates for County 2 and County 6 are not statistically significantly different than the state rate.   

The p-value does have limitations.  A non-significant p-value does not necessarily mean the outcome is not clinically important.  Similarly, there is a possibility that an outcome can be statistically significant and not be practical. The “common sense” component of public health practice is an essential part of decision-making and should not be limited by a p-value. To further explain this limitation, consider two studies.  The first study looks at a low-fat diet and breast cancer in women with prior breast cancer.  The second study looks at a low-fat diet and breast cancer in women without prior breast cancer.  The p-value in the first study was significant (p = 0.03) and the p-value in the second study was not significant (p = 0.07).  However, in both studies, the low-fat diet groups developed fewer new breast cancers.  Although the second study was not significant, the result of fewer new breast cancers was clinically significant. 

Another limitation of the p-value is that the value can be influenced by the number of subjects.  If there are too few people in the study groups, the p-value may be unstable.  There is no way to distinguish when no difference exists between groups (no significance) and when there are not enough people in the study groups. 

The p-value is important but should not be the sole determination of significance.  A better assessment would also include appropriate confidence intervals (along with an odds ratio or risk ratio), which will be discussed in the next issue of the Indiana Epidemiology Newsletter

References

  1. Kuzma, Jan.  Basic Statistics for the Health Sciences. 3rd Edition. 1998.
  2. Prentice RL, Caan B, Chlebowski RT, et al. Low-Fat Dietary Pattern and Risk of Invasive Breast Cancer, JAMA. 2006; 295:626-642.
  3. Chlebowski RT, Blackburn GL, Thomson CA, et al. Dietary fat reduction and breast cancer outcome: interim efficacy results from the Women's Intervention Nutrition Study (WINS). J Natl Cancer Inst. 2006; Vol. 98, No. 24, pp 1767-1776.