Logging has been used since the most primitive software days as a form of event storage abstraction. Obviously, system and application level logging have matured over the years. Application logs now usually store records of application events like request details, response codes, and triage constructs like unstructured errors messages or stack traces. Logging frameworks like log4j in the Java world and utilities like syslog have made it easier to control logging levels, history (rollovers) and make application behavior human readable (short for text). Logs are still core to debugging or analytics regardless of the automation and tooling built over them. The same applies with Node.js applications too.
The problem with Node.js logging
Error stack traces in Node have still have some distance to cover to become really useful. The absence of a thread local type context in the asynchronous world of Node, makes error correlation a nightmare. This is complicated by anonymous/non-strict function constructors. Zones and Promises solve the problem to an extent for development time but still face overhead challenges in production time. Logging frameworks like Bunyan, Winston or Strong-Logger have provided control over verbosity using logging levels – Debug, Info, Warn, Error and Node specific structure by prepending timestamps, cluster – worker processes IDs, app-names or hostnames and rollover capabilities.
But the big fact remains that these logs have to be humanly monitored almost impossibly and quite a few interesting or abnormal events preluding a fatal condition can be missed. I obviously don’t need to preach to you how important this logging is for analytics – which is another weak spot in the Node application space. We discuss a simple but effective solution using StrongLoop Controller and Splunk.
What is Splunk?
For those of you not familiar with Splunk, it’s software that searches, monitors, analyzes and visualizes machine-generated big data from websites, applications, servers, networks, sensors and mobile devices. More than 7,900 organizations use Splunk software.
What is StrongLoop Controller?
StrongLoop’s Controller provides an integrated development, runtime and operations tool-chain for deploying scalable, high quality Node code in production. Features include:
- On-premise – take control of your Node DevOps in your datacenter or on your own cloud infrastructure
- Open Source – developed in Node, built for Node apps
- Clustering – horizontally scale across multiple processes with built in dynamic resizing, load-balancing, hot deploys, rolling restarts and state sharing
- Debug – Node Inspector in browser with unit test integration, inspection of deep call stacks, pre-emptive breakpoints, source map loading, display & edit of expressions and variables.
- Log Management – log aggregation across processes in cluster, structured logging, log levels and integration with third party tools
- Build – build apps without registry dependencies, support for Git merge, Output tarball packages usable by npm pack and npm install, publishable archives with dependencies
- Deploy – publish app to local or remote servers, service lifecycle management (boot, start, restart, status, destroy), redeploy without downtime, scaling
- Private Registry - multiple npm registry, on-premise or cloud, promote package versions, switch between registries.