Amazon’s Kinesis Streams service provides a powerful way to aggregate data (logs, etc.) from a large number of sources and feed that data into multiple data consumers. For instance, a large enterprise might use one Kinesis stream to gather log data from their cloud infrastructure and another stream to aggregate sales data from the web services running on that infrastructure. Once the data is in the stream, it remains available for up to a day (or optionally longer) for any number of applications to read it back for processing and analysis. This is particularly useful to customers that want to deploy and destroy virtual machines on a whim; data is stored in the stream, rather than the ephemeral VMs.
We’re extremely excited to announce a new major release of the Gravwell analytics platform. It’s been a long road full of interesting (and sometimes annoying) challenges.
It’s Thanksgiving Weekend in America and that means most people have acknowledged the blessings in their lives and are gearing up for something America does better than anyone: consumerism. I had a bit of down time and thought I’d do something else America is good at: Freedom Fighting.
With Thanksgiving on Thursday, the start of the winter holidays is here in the states. In addition to seasonal celebrations spanning the weeks, shopping often increases around this time. Two such days, Black Friday and Cyber Monday, are some of the biggest shopping days of the year and people often wait to see what deals can be found. Products are launched on or around Black Friday/Cyber Monday in the hopes of garnering more sales and to drive up excitement. Often, this is a great idea. Sometimes, though, a product drops in such a way that could only be dubbed failure.
In this post, we take a look at analyzing Industrial Control System data to detect unauthorized manipulation of relays in a process.
You never forget the first time… and we’ll always remember getting together with hundreds of leading security experts at the first ever Wild West Hacking Fest in Deadwood, South Dakota. We got a lot of praise before the first guest arrived at our table, but that’s probably because we sponsored the coffee! Still, when people came over to look at Gravwell’s products, we got a lot of positive feedback and eager experts wanting to test what we can do.
For this post, the Gravwell analytics team ingested all 22 million+ comments submitted to the FCC over the net neutrality issue. Using Gravwell we were able to rapidly conduct a variety of analysis against the data to pull out some pretty interesting findings. We scraped the entirety of the FCC comments over the course of a night and ingested them into Gravwell afterward. It took about an hour of poking around to get a handle on what the data was and the following research was conducted over about a 12 hour period. So we went from zero knowledge to interesting insights in half a day. We’re kinda nerding out about it.
In this post we'll walk through a case study with a customer trying to identify an infrastructure capacity issue in which the system becomes unresponsive during a swell in page visits. We'll follow Alice and Bob (names changed, obviously) as they work through the issue.
We are happy to announce the release of version 0.2.6. This release has your standard array of bug fixes and quality of life improvements but the major change comes in the form of relational analytics enhancements. We have added support for force directed graphs which allow for some advanced relationship analytics and data correlation.