When measuring differences between groups, 95% confidence intervals which do not include 0 have p values of < 0.05, the traditional cutoff for statistical significance.
In this study, four separate diets were investigated for their ability to cause weight loss at a time point two years after baseline. As such, a mean weight loss can be calculated for each group of dieters and a 95% confidence interval about that mean. For example, the average change in weight on the Atkins diet was -2 lbs, with 95%CI (-3, -1 lbs). The change in weight on the Atkins diet would be considered significant at p < 0.05 level because its 95%CI of (-1, -3 lbs) does not cross the zero value. Likewise the low fat diet is also significant, albeit in the opposite direction. Both the Atkins and low fat diet groups have statistically significant changes in weight, although the Atkins dieters lost weight while the low fat dieters gained weight on average.
Rosner delineates the factors which affect the size of the confidence interval. 1) As the sample size increases, the length of the confidence interval shortens. 2) As the standard deviation increases, the confidence interval widens. and 3) As the confidence level increases, e.g. from 95% to 99%, the confidence interval widens.
Guyott et al. expand on the clinical importance of confidence intervals. CIs convey to the reader within which range, given the study data, the true effect of the intervention or exposure is most likely to lie. Moreover, the reporting of confidence intervals, as opposed to p values alone, has increasingly become common practice in the literature as the confidence interval conveys not only whether the result is statistically significant, but also the direction, magnitude, and precision of the finding.
Answers 1 & 2: Both the Atkins and low fat diets would have p values < 0.05, but each of these answers includes only one or the other.
Answer 3: The low carbohydrate diet's 95%CI of (-3, 1) includes 0 and thus would not have a p value < 0.05.
Answer 4: The Zone diet's 95%CI of (0, 5) includes 0 and thus would not have a p value < 0.05.
Rosner, B. Fundamentals of Biostatistics; 7 ed. Cengage Learning, 2011.
Guyatt G, Walter S, Cook DJ, Wyer P, Jaeschke R. Chapter 8. Confidence intervals. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2nd ed. New York, NY: McGraw-Hill; 2008.