Posted Thursday, October 9, 2008 by
PJ Hinton
If I have learned anything in my current and previous jobs, it is that so much of our world is event driven. That is, most of what we do and choose is based heavily on our analysis of observations and sensations.
At a personal level, the human nervous system generates a multitude of signals, supplying the brain with a snapshot of our physical and chemical environment. Through our five senses, we interpret this information and respond, consciously based on past experience, and unconsciously through reflexes.
Timely and credible information is essential. If you need an example of this on a dramatic scale, think about the recent turmoil in the finance sector. Those with money to lend are refusing to do so because of uncertainty, rooted ultimately in large numbers of institutions holding complex securities. No one has a good idea of the securities' true value.
For as much as we feel overloaded by information, there is so much that transpires without our immediate awareness. If those events are of little consequence to our lives, that's not such a bad thing. But in the world of software engineering, ignorance is seldom bliss.
There are a good number of tools, both free and commercial, that
will help you monitor your hardware and it's network environment. For
several months, we have been using such tools to keep an eye not only
on our servers, but external services upon which we rely.
Web applications run atop on server software, like the Apache web server, which provides a lot of raw information about what it's up to, such as who is requesting what at which time, but that's hardly the full picture of what's going on. Between the request and response , there is considerable activity which is not visible from the webserver's logging facilities..
Once you get into the body of a web application, especially one that you've developed from the ground up, the ability to detect and understand what's going on within falls upon the developers. Unfortunately, many application developers treat this as an afterthought, even though many web application frameworks give you some basic tools for collecting data.
I remember hearing some horror stories from a couple years ago, where another startup went live with a lot of publicity, and their servers got hammered by traffic. Most visitors were disappointed because the application was barely functional from their standpoint. On the server side, little effort had been put into providing facilities to diagnose problems with each application component. Even if they knew something was wrong, they didn't know where to start looking.
Because we're software-as-a-service, we're responsible for providing reliable, hosted blogging software to a large number of paying customers. Uptime and responsiveness are critical, and we've been putting a lot of design and development effort toward tools that will help us keep an eye on that.
Over the past couple of weeks, we have been adding to our application the infrastructure that enables event tracking at a very fine level. This week, we completed an extensive instrumentation of our code base to take advantage of that foundation.
This project has been exciting because it gives us a very high level of flexibility. Each consumer of this rich data stream can pull the level of granularity that is needed. The tracking exerts minimal overhead on the system and allows us to get a detailed view when we need it.
While this enhancement may not be immediately visible to our customers and visitors, they will certainly appreciate the rewards of an rock solid web application.