When analyzing any repeatable random process (portfolio returns, equipment uptime, cohort conversion rates—anything with an element of randomness), it's easy to limit yourself to a single number: the average expected result. But the average says nothing about how far individual observations may deviate from it—and that's a separate and equally important parameter.

Imagine two commutes to work with the same average travel time—30 minutes. One takes 28–32 minutes every day, with no surprises whatsoever. The second, on average, takes the same 30 minutes, but in reality, three days in a row, it takes 20 minutes, and on the fourth, an hour in traffic. The final weekly average for both will be the same, but in terms of feel and planning reliability, these are two different routes.

Formally, this is described by dispersion/standard deviation (σ)—a measure of how much individual values ​​diverge from the mean. The wider the dispersion, the less predictable the process's behavior in the moment, even if the long-term average is exactly the same.

A separate practical pitfall: if you calculate σ "head-on" across the entire sample, a single rare extreme outlier can significantly distort the final dispersion estimate. Therefore, it is often more reasonable to calculate σ separately for each data segment (by day, by session, by cohort), and then take the median of these σ values ​​rather than the mean. This filters out the influence of isolated anomalies and provides a more robust picture of the typical dispersion.

By the way, someone in another thread posted a similar analysis with numbers based on this distribution: https://slot-stat.com/volatility-and-variance/. It's convenient for getting straight to the point, rather than explaining it in layman's terms.

How do people in their fields handle the task of filtering outliers when estimating dispersion—do you use the segment median, the trimmed mean, or something else?