Part 2 of the AI Detection Library Hygiene Series

Mike Wade, VP of Customer Success

The last post in this series explored detections that quietly stop working. But broken detections are only part of the problem. What about detections that work perfectly but serve no purpose? Or valuable telemetry that never gets used? Gravwell Logbot can help security teams identify both.

A detection library isn't just a pile of queries; it's a portfolio. Every detection in it costs something: compute when it runs, analysts’ attention when it fires, and engineering time when it needs maintenance. And every data source feeding your SIEM costs something too: storage, pipeline upkeep, and the ongoing effort of keeping the ingest healthy. A portfolio deserves scrutiny. Is the spend going where the risk is? Is anything in here redundant? Is anything we're paying for producing nothing?

Almost nobody asks these questions on a schedule, for the same reason nobody audits for dead detections: the work is real, the payoff is diffuse, and there's always an incident that matters more today. So we gave the job to Logbot, Gravwell's AI agent framework, which can do the tedious detection maintenance for Gravwell users without ever getting bored.

For your portfolio review, use Logbot to cover these four core functions:

1. Fire-rate and disposition analysis

Every unnecessary alert consumes analyst attention that could be spent investigating meaningful threats. Identifying these detections helps teams reduce noise without sacrificing coverage. So start with the simplest question: which detections actually earn their alerts?

Logbot pulls the alert history for every detection in the library and computes fire rate over a trailing quarter. If you record dispositions (closed benign, escalated, converted to incident), it goes even further and computes the alert-to-action ratio per detection. The extremes are where the findings live. A detection that fired 900 times with zero escalations is a candidate for tuning or retirement. A detection that fired twice in a year and both were true positives is quietly the most valuable thing in your library, and probably deserves sibling detections covering adjacent behavior.

Logbot analyzes detection fire rates and alert dispositions to identify noisy, low-value alerts and highlight detections delivering meaningful security outcomes.

Humans usually build this report, roughly once, perhaps for a QBR, and then never again because it took days. The point of handing it to an agent is so this analysis happens continuously.

2. Overlap and duplication

Detection libraries accumulate near-duplicates the way codebases accumulate copied-and-pasted code. Two engineers, three years apart, each write a detection for password spraying. Different names, slightly different filters, same technique, and overlapping fires. Nobody notices because nobody has the entire library committed to memory at once. Once a library grows beyond a hundred or so detections, nobody can—except an agent.

Logbot reads the query logic across the full library and clusters detections that are semantically close: same tags, same extracted fields, and same core comparison, differing only in threshold or scope. For each cluster, it proposes a resolution, which is usually to consolidate into a single parameterized detection, keep both because they cover genuinely different edge cases, or retire the older one outright. That last judgment is the important one, because the clusters that look like duplicates but aren't (one covers service accounts, one covers human users, and the difference is a single filter) are exactly where naive deduplication breaks coverage.

Logbot identifies overlapping detection logic and recommends opportunities to consolidate redundant rules without sacrificing coverage.

The human reviews the clusters and makes the call. Logbot's job is surfacing the clusters, which is the part that was never going to happen manually.

3. Coverage mapping against reality

Every detection engineering methodology says your library should be mapped to MITRE ATT&CK. Almost no library actually is, because the mapping is manual annotation work that loses to urgent operational work every single day. Detections get written during incidents, onboarding pushes, and "we should really catch this" moments. Meanwhile, the technique ID was never anyone's priority. As a result, the honest state of most libraries is a few detections with MITRE ATT&CK T-numbers in the name from that one quarter someone cared, and everything else unmapped.

This is annotation work, and annotation work makes for great agent work. Logbot reads each scheduled search in the library, both the query logic and the description, and infers the MITRE technique. A query watching for repeated failed logins across many accounts from one source is password spraying; that's T1110.003, whether or not anyone wrote it down. Logbot then applies the T-number as a label on the scheduled search, so the mapping lives on the detection itself as metadata, not in a spreadsheet that's stale the week after it's built. Where the query and the description disagree, or the logic is ambiguous between techniques, Logbot doesn’t guess; it flags the detection for human review.

Logbot maps existing detections to MITRE ATT&CK techniques, helping security teams understand their current coverage and identify potential gaps.

4. Untouched Tags

Now flip the coverage mapping question. Mapping detections to MITRE ATT&CK reveals which techniques your library covers. But what about the telemetry you're collecting that isn't contributing to your detection program? That's where untouched tag analysis comes in.

Logbot cross-references ingest statistics against search history: scheduled searches, alerts, dashboards, kits, and ad-hoc queries. Any tag receiving entries but not referenced by a scheduled search, dashboard, kit, alert, or ad-hoc query in 90 days gets flagged for review. That doesn't automatically make the data unnecessary. It may represent a detection coverage gap, serve a compliance or retention requirement, or simply be data the organization no longer needs to collect.

Each orphan warrants a closer look. Some represent coverage gaps: telemetry collected for a reason but never incorporated into detections. Others may be retained for compliance or retrospective investigations, even if they aren't actively queried. And some may genuinely be unnecessary. Logbot helps distinguish between these scenarios and suggests potential detections for data sources that could strengthen coverage.

Logbot compares ingested telemetry against search and detection activity to surface underutilized data sources and uncover potential coverage gaps.

The dashboards-and-kits part of the cross-reference matters. A tag consumed only by a dashboard isn't orphaned; it may not be serving detections, but it’s serving humans. The naive version of this audit flags it anyway and erodes trust in the whole report. Checking every consumer is exactly the kind of thoroughness that's cheap for an agent and expensive for a person.

The review you can actually run

None of the four functions above is a product feature you wait for or a project you staff. Each one is simply a prompt. The portfolio review that every detection engineering methodology says you should do quarterly, and that almost no team ever does, turns out to be four questions asked of an agent:

• Which detections never fire?
• Which detections duplicate each other?
• What does coverage look like by technique?
• What data is nobody watching?

The reason those questions went unasked was never that the analysis was particularly difficult; it was that answering any one of those questions meant a tedious day of pulling histories and cross-referencing by hand. Doing it once was a project and doing it repeatedly was out of the question.

Like the tasks we covered in part one in this series, this is agent-shaped work. It’s bounded because the library and tag list are finite. It’s verifiable because every finding comes with a query you can rerun yourself. And it's low-risk: the analysis itself is read-only. The only modification, applying ATT&CK labels, is limited to additive metadata that humans can review and remove without changing detection behavior. Nothing about a detection's behavior changes without a person deciding it should.

AI-powered triage assistants get the headlines, but an agent that continuously evaluates detection effectiveness, identifies wasted effort, and exposes coverage gaps can have an equally meaningful impact. It doesn't just help analysts respond to threats. It helps security teams build and maintain a better detection program.

Next in the series: the maintainability problems nobody gets paged for. Naming schemes that lie, documentation that drifted from the query it describes, and the search schedule pileup happening at the top of every hour.

See how Gravwell and Logbot can help you uncover noisy detections, redundant logic, coverage gaps, and data nobody is watching, without turning the audit into a manual project.