Settings

The detection, and what it is pointed at

All four as decimals: 0.95 is ninety-five per cent, 0.000002 is two in a million. The prevalence is the one nobody writes down and the one the answer turns on, so guess it honestly rather than leaving it flattering; the page is most useful when that number is uncomfortable.

What the queue actually is

What it costs, both ways

Precision and recall move in opposite directions, and the only honest way to choose between them is to price both: the hours the queue costs, and the genuine events that never reach anybody. A detection tuned until the queue is bearable is a detection that is missing things, and how many is computable from the same four numbers.

If a second detection had to agree

This assumes the two are independent, which they usually are not. Two rules keyed off the same parent process, the same host or the same log source are not two opinions. Where they are correlated the real false positive rate is higher than the product, sometimes by orders of magnitude, and the precision above is the best case rather than the expected one.

The working


    

Questions people ask about BASERATE

Why is the queue mostly noise when the detection is good?

Because precision depends on how common the thing is, and sensitivity and false positive rate do not. Take a detection that catches ninety-five per cent of what it looks for and fires on one benign event in a thousand (both respectable) and point it at ten million events a day where genuine intrusions run at two in a million. It finds nineteen real ones and ten thousand benign ones, so one alert in five hundred is worth opening. Nothing about the detection is wrong. The base rate did that, and no amount of arguing about sensitivity will move it.

What is the number I should actually be asking a vendor for?

The false positive rate, expressed as one in how many, and then you supply the prevalence yourself and compute the precision here. A vendor cannot tell you the precision because it depends on your estate, not their product, which is convenient for them and is why the conversation is always held in the two terms that cannot settle it. If a rate is quoted as a percentage with one significant figure, that is not precise enough to be useful: 0.1% and 0.05% differ by a factor of two in the queue.

Is requiring two detections to agree really that powerful?

Arithmetically, yes: the false positive rates multiply, so two rules at one in a thousand and one in fifty become one in fifty thousand, and precision can rise by a factor of thirty or more. The catch rates multiply too, so recall falls and you miss more. But the assumption underneath is independence, and that is usually false. Two rules that key off the same parent process, the same host or the same log source will be wrong together, and then the real rate is nowhere near the product. Treat the improvement on this page as the ceiling, not the forecast.

Why does the page insist on showing what is missed?

Because tuning a rule until the queue is bearable is the same operation as tuning it until it misses things, and only one half of that usually gets discussed. A detection quietly dropped from ninety-five per cent to seventy to make the volume manageable is a decision to miss one real event in four, and it should be made on purpose with the number in front of you rather than arrived at by a fortnight of complaints about alert fatigue.

Where does the hundred-square picture come from?

It is a hundred alerts drawn from the queue at the precision computed above, with the real ones marked. At one per cent, one square in a hundred is dark and the rest are not, which is the experience the arithmetic describes and the reason people stop reading a queue. It is the same information as the percentage; it is there because the percentage does not land and the picture does.

Can I use this outside security?

Yes, and the shape is identical. It is the same calculation behind screening for a rare disease, fraud scoring, content moderation and every other place a good test is pointed at a rare thing and produces a pile of false alarms. The labels here are a detection and an analyst, but the arithmetic knows nothing about security: only that being right about a common thing and being right about a rare one are different problems.

Doing this often? OBSCURA Gnosis runs the statistics properly: descriptives, tests and regression, with tables saved to Word. On its own, US$2 a month; with everything else, US$5 a month or US$48 a year.