Help Center

What is a Confidence Interval?

A confidence interval can be thought of as an estimate plus-or-minus a certain amount. It’s a way to show the uncertainty around a survey result. For example, if you see a bar that shows a black vertical line (the “point estimate”) at 50%, and the confidence interval is plus-or-minus 5%, that means we’re reasonably sure (95% confident) that the ‘true’ population value lies between 45 and 55. “50%±5” or “[45, 55]” can be hard to read in a table, but works well in a graph. The richly colored bar around the black vertical line (the “confidence interval") represents this range:

Why are Confidence Intervals Important?

• Judge the Precision: The plus-or-minus value helps you grasp how precise the survey result is. A smaller plus-or-minus number means the result is more precise. Note that precision is not the same as being correct. A poorly worded question or a biased sample can still give misleading results, no matter how precise the estimate.
• Informed Decision Making: Knowing the range within which the true value probably falls (the confidence interval) can guide better decisions based on your survey data.

How to Interpret Confidence Intervals in Crunch

• Look at the Range: Each bar will have a black vertical line (the point estimate), which sits at the center of the confidence interval. This range is where the actual value is likely to be (95% confidence). Note that the whole range is equally likely, given the data. The interval is constructed around the point estimate, but the center cannot be considered any more likely than either extreme.
• Check for Overlap: If the confidence intervals of two bars overlap, it approximately* means there’s not a statistically significant difference between the two groups or values you are comparing. You can see this overlap happening between Brand C and Brand D in the example above. Note, though, that statistical best practice is to not treat this significance distinction as binary. The size of the intervals and the degree of overlap are relevant (don’t hold a ruler up to the screen; that defeats the purpose of showing the whole interval).
• Consider the Width: The width of the confidence interval (the total range of the plus-or-minus values) can tell you a lot. A narrower interval (±2%, for example) means you can be more confident in the accuracy of the survey result than if the interval is wider (±10%).

Exporting graphs to PowerPoint

• When exporting a dashboard, graphs with confidence intervals enabled will use PowerPoint's native "Error bars" functionality to represent the confidence interval range. They don't look as nice as the Crunch ones but we want to ensure that exported visualizations remain editable, native objects, rather than static images.
• Exports from decks, for now, will not include the error bars. This will be addressed in a future product update.

Summary

• Confidence intervals are essentially your “best guess plus-or-minus a certain amount.”
• They provide a handy way to understand the accuracy and reliability of your data.
• If two intervals overlap, it suggests that the difference between the two values may not be significant at 95% confidence.
• The size of the plus-or-minus range offers additional clues about how much confidence you can have in the results.

Technical note: The overlap between independent confidence intervals is not exactly the same as the margin of error of the difference between the two. In general, the margin of error of the difference is larger because it considers the variance of both together rather than each on their own. If they look like they overlap, there is more than a 5% chance they are not actually different.