Blog

John Floren

John's been writing Go since before it was cool and developing distributed systems for almost as long.
Find me on:

Recent Posts

New Release with Netflow v9 Support for Gravwell

Jan 8, 2020 9:30:00 AM / by John Floren posted in Network Analytics, Integrations

0 Comments

Gravwell has officially supported Netflow v5 and IPFIX for some time. As of Gravwell 3.3.3, we're happy to announce that we now support Netflow v9 as well! This post will talk about the essential differences between Netflow v9 and IPFIX, how we implemented support, and how to get up and running with Netflow v9 ingest. We'll also talk about some pretty serious efficiency improvements we made in our IPFIX/Netflow v9 parsing module.

Read More

Announcing Gravwell's Office 365 Management Log Ingester

Dec 17, 2019 11:45:00 AM / by John Floren posted in Events, EventLog, Microsoft, Windows, Integrations

0 Comments

If your enterprise is using Office 365, your users are generating log entries every time they log in, upload files to OneDrive, send an email--the logging is pretty extensive! You can analyze these log events in the O365 console, but wouldn't it be nice to pull them into Gravwell and correlate with other data sources? Thanks to the new Office 365 ingester, you can.

Read More

Introducing Gravwell Macros

Nov 27, 2019 9:24:55 AM / by John Floren posted in Software Updates

0 Comments

One of the exciting new features in Gravwell 3.3.0 is search macros. Anyone who's experimented much with Gravwell knows you may often end up crafting a long and complex regular expression which you'll want to use over and over, but such a long regex makes the query hard to work with. Macros let you turn that long regular expression (or any other part of a search query) into a much shorter name you can use again and again.

Read More

Fellow Go Devs, Here's Our Experience Moving to Go Modules From Dep

Oct 17, 2019 2:14:57 PM / by John Floren posted in developer, golang

0 Comments

When we started Gravwell years ago, we knew it was going to be a significant undertaking requiring some serious tooling under the hood. Building a custom data lake and analytics platform from scratch that can scale to hundreds of TB/day ain't easy. We chose Go for a lot of reasons and that choice has paid dividends in terms of what we've been able to accomplish in so short a time.

This post is about our tooling, and some of the lessons we have learned along the way in managing a large Go codebase. A few weeks ago Gravwell made the switch to Go modules on both our open source github repositories and our internal repo. Let's talk about about our planned workflow going forward and a few caveats we've run into.

 

Read More

Introducing the Key-Value Search Module

Oct 1, 2019 2:35:15 PM / by John Floren posted in Gravwell Story, Software Updates, Logging

0 Comments

With Gravwell 3.2.4 we've introduced a new search module: kv, short for 'key-value'. This module is designed to help you extract key-value data from text entries without having to hand-craft regular expressions. It also interfaces with the fulltext indexer automatically, so you can analyze your indexed data more quickly.

Read More

Announcing Gravwell Version 3.2

Jul 31, 2019 9:26:42 AM / by John Floren posted in Software Updates, DevOps Analytics

0 Comments

We are happy to announce the immediate availability of Gravwell version 3.2.0!

Read More

Fighting social media propaganda

Feb 19, 2019 9:55:00 AM / by John Floren posted in Software Updates, ingester, reddit

0 Comments

We're continuing to work with investigative reporters to research unscrupulous activity on social media. Most recently, Engadget published a piece on nefarious political influencers on Reddit. We’ve written in the past about analyzing social media comments, but didn’t make the ingesters publicly available. With an increasing need for research in this area, we decided that releasing our Reddit and Hacker News ingesters could help new users get started with Gravwell even faster, so we open-sourced them. Read on to learn how to get the ingesters, how to run them, and how to get started with the data.

Read More

Announcing the new Gravwell HTTP Ingester

Feb 7, 2019 11:22:19 AM / by John Floren posted in Gravwell Story, ingester, Community Edition

0 Comments

Gravwell recently introduced a new ingester which accepts entries via HTTP POST requests. Now it's easy to send arbitrary data to Gravwell via scripts using only the curl command. In this blog post, we'll use the HTTP ingester to build a weather-monitoring dashboard!

Read More

Gravwell 2.2.1 Released!

Aug 1, 2018 3:48:22 PM / by John Floren posted in Software Updates, ingester, docker, Community Edition

0 Comments

We’re pleased to announce the release of Gravwell 2.2.1! For a point release, it’s got some very cool new features; read on to learn what we’ve added.

Read More

Monitoring Netflow with Gravwell Community Edition

Jul 18, 2018 1:21:59 PM / by John Floren posted in Network Analytics, Community Edition, Home Operations Center

0 Comments

Gravwell Community Edition is perfect for monitoring your home network. With a generous 2GB/day ingest quota, you can capture netflow records, DNS requests, WiFi hotspot associations, and more. In this blog post, we’ll show how to ingest and analyze netflow records. We’ll assume you’ve already set up a Gravwell instance as described in the quickstart (https://dev.gravwell.io/docs/#!quickstart/community-edition.md); for this post, we’ll assume the Gravwell instance is at 192.168.1.52. Your instance will almost certainly be different, so be sure to substitute your own information.

Read More