One of the many improvements in our recently re-deployed ELK stack is the use of CloudWatch to store performance metrics of our Logstash pipeline. Event rates for our shippers and indexers, buffer lengths, and host information are now all in a common platform without having to maintain a separate time series system. To track how the system is performing over time, we have a CloudWatch Dashboard that displays many of these key metrics. This is a terrific single pane of glass, but it does have some limitations. Graph types in CloudWatch are limited to line charts with a minimal amount of static text, and all graphs displayed on a common dashboard must use a single time window. This last one is particularly limiting as metrics that are measured on a daily basis (such as the number of objects in a S3 bucket) come across as either a single dot or, if midnight UTC is not in the displayed time window, fail to display at all. Getting this data out of CloudWatch and into something more versatile has long been on my wish list.
|Sample CloudWatch Graphs|
The connector works well, though there are a number limitations in the WDC, mostly surrounding the inability to edit parameters given to the AWS SDK apart from the initial connection. At the moment, once a time frame is passed to AWS for query data, that period is locked in and cannot be changed without establishing a whole new data connection.
So, give this new WDC a try and see if you don’t find your BI analysts and AWS admins having conversations along the lines of whether they have “Your Tableau in my CloudWatch!” or “Your CloudWatch in my Tableau!”