I have wanted to set up a real-time dashboard for some time now. The goal has always been to mount a display to the wall and have graphs showing pretty lines that have meaning. That should not be hard, right? I always seemed to run into some problem that would put the project on hold… until now!
While attending Zendcon, one of that talks I went to was on application monitoring. I walked away with a laundry list of things to look at and excitement about getting my idea finally off the ground. The promises of better insight alway sparkle in the light of being able to see things while they happen.
Below are some of the tools and software I found and used to build the dashboard. All of this was done using a low powered Ubuntu 14.04 linux server.
StatsD is one of the items on my list to research. I really liked the concept of StatsD focusing only on counting and timing. The other draw for me was not needing to define any structure ahead of time. Just pass in names with values and let StatsD do the rest. Finally, StatsD uses UDP connections for receiving data which allows the client application to continue functioning even if the StatsD service is unavailable.
Graphite is another item to research and is used with StatsD. StatsD supports several backends; I chose Grafana and InfluxDB as alternatives. InfluxDB is a Round Robin Database with an SQL like query interface which was part of the reason why I choose it. Grafana is the graphing web interface and is based on Graphite.
Setting up all three projects was very straight forward. Following the basic installation instructions, I started setting up InfluxDB, Grafana, and StatsD. With StatsD you have to take the extra steps of installing the InfluxDB driver, updating the default config for Influx, and setting up a way to run StatsD as a service that starts on boot. I found some material on setting up a Monit script for managing the StatsD process as a service.
I used Ubuntu’s apt-get dependency manager to pull in a StatsD client and added the couple lines of code needed to send the data I wanted to collect. Afterward, I opened Grafana and with a couple of mouse clicks had a graph created. No fuss about schemas or configs for adding new metrics. Just send data, graph data.
Using StatsD and Grafana (or Graphite) there are no more excuses for not collecting data for monitoring your systems. If I had to do it again, I would have stuck with the standard Graphite install to avoid the need for extra drivers. I only wish I had known about StatsD two years ago.