query

Check for "Bouncing" Ingesters

Apr 19, 2021 5:56:13 PM / By Gravwell

This query checks the logs in the Gravwell tag for ingester connect/disconnect events and charts the frequency of these events per ingester. If you see an ingester disconnecting and reconnecting frequently, it means one of two things: either there's something wrong with your connection, or the ingester is getting periodically kicked by the indexer(s) due to inactivity. Either situation warrants investigation! Normally, ingesters should only reconnect after a software upgrade.

tag=gravwell regex "INFO (?P<ingester>\S+).*\s+(?P<action>\S*connected)\s+(to|from)" | stats count by ingester | chart count by ingester

Below is an example of a table showing the results for this query:

QOTW_Bouncing-Ingesters

 

Gravwell
Written by Gravwell