For many data analytics studies, summarizing the “bottom line” with only a few statistics is very tempting. This is often done to simplify communication and maximize the impact of the outcome. On the other hand, failing to consider the efforts behind the calculations of these statistics, or to fully understand the specifics in data collection and the assumptions used to make the inferences can lead to erroneous conclusions and missed improvement opportunities. The prevalent use of capability indices offers a prime example. These unitless measures of the ratio of the range of the engineering specifications to the natural variation in the process offer a very valuable summary of the capability of the processes yet they are all calculated based on a sample of observations and a set of assumptions. Ignoring all these and just focusing on the point estimates can be misleading. In this Quality Quandaries, we discuss some of the conditions and assumptions in the calculation of these indices, and for the violation of the normality assumption, we present two case studies highlighting potentially problematic issues.