In the movie This is Spinal Tap, one of the members of the fictional band proudly states that their band is the “loudest.” “Why is that?” the interviewer asks. “Because our amplifier’s volume goes all the way up to 11.” was the answer. How many times has the business customer heard similar meaningless proclamations from IT staff or the management about the performance or availability of a critical IT Service? Spinal Tap’s answer was intended to spoof rock band documentary films. What is behind IT’s answer?
Historically IT organizations have not done a very good job of providing meaningful reporting either to their customers, or internally. With the emerging trend toward the adoption of various frameworks and methods to improve the planning, design, delivery and operation of IT services, there is not a whole lot of practical guidance on how to actually go about planning, designing, implementing and operating a comprehensive framework for the measurement of IT services. In fact the Continual Service Improvement volume even goes so far as to say, “Setting up a [measurement] framework is as much an art as a science,” and then goes on to talk about what you should do to set a measurement framework. So much for process.
Although setting up a service management measurement framework is not necessarily easy, it does not belong in the “arts & crafts” category either. The straight-forward application of four fundamental steps will enable IT organizations to establish a comprehensive IT service measurement framework.
Guiding Principles – Before jumping in the deep end of this, let us look at some basic principles that will help put together a meaningful and usable measurement framework. The first thing we need to understand is that a measurement framework is a “means to an end,” not the “end” in itself. Its intent is to enable the continual improvement of both services and supporting processes. The framework should not be considered a “work of art,” thus immutable. As organizations mature, the business changes and other factors impinge on the “what and how” of the framework, it must be flexible enough to evolve along with its environment.
The structured approach below follows the fundamental Plan, Design, Implement and Improve model.
Step One – Establish the Context
The first step in establishing a service measurement framework is to clearly define the context in which it will be used. In other words, you must clearly define the “world view” that will be the environment of the framework. This creates the “anchor points” for the measurement framework.
A good starting point for such a definition is the work done by the Managed Services Association in 2000-2001 in conjunction with its Service Management Quality Initiative. The intent of the initiative was to define a framework that service providers could use, and that their customers could use, to evaluate the viability of the organization and their service offerings.
In short, it established a three-dimensional model that described the service provider’s Quality of Service (QoS), its Capability of [supporting] Process (CoP), and it’s Quality of [supporting] Process (QoP).
The concept behind this model was that the quality of a service directly relates to the service provider’s capability to provide the processes supporting the planning, design, delivery and operation of their services. This dealt with resources applied (capability and assets) to supporting processes.
The second part was quality of process, which dealt with the service provider’s demonstrated efficiency, effectiveness and compliance to supporting processes. Quality of Service measured the performance of the service in terms of its availability, capacity, continuity and security (in ITIL v3 terms, its warranty).
While all of this is a good starting point, each organization must determine how this model matches its world and tweak it accordingly.
Step Two – Architect the Framework
The term “architect” in this context uses the definition relating to the “systemizing of knowledge.” In other words we need to formulate, or reduce into a system, how data and information is gathered and used in a regularly structured manner (not a very “artsy” approach I’m afraid). In this step, each dimension of the framework is evaluated and key factors are identified and described. For example;
Capability of Process (resources – capability & assets)
- Capabilities
- Management
- Organization
- Processes
- Knowledge
- People
- Assets
- Financial Capital
- Infrastructure
- Application Information
- People
Quality of Process
- Compliance
- Efficiency
- Effectiveness
- Value
Quality of Service (a.k.a. Warranty)
- Capacity
- Availability
- Continuity
- Security
Step Three – Engineer & Build the Framework
This step could have easily been called, “Plan & Build …,” but the choice of the term “engineer” is intended to convey a specific meaning, and that is a highly structured or rigorous approach to identifying and documenting metrics used within the framework to properly measure the performance of each of the factors in each of the dimensions of the framework.
An elegantly simple, but powerful, method that can be used here is the Goal-Question-Metric (GQM) Method developed at the University of Maryland. It is publicly available at ftp://ftp.cs.umd.edu/pub/sel/papers/gqm.pdf. It uses a simple three-step approach of identifying a goal to achieve, asking questions about how that goal can be achieved, and then identifying what metrics can be used to measure its achievement. It is well worth the effort to take a look at it.
Once the engineering part is done, the build part deals mostly with figuring out if the data is or can be made available and making it so. A caveat at this point is that it is best to understand and engineer the desired end-state, but build and implement incrementally. This is very similar in nature to a spiral method in software development. This becomes apparent in the next step.
Step Four – Mature the Framework
It should not come as a surprise that things change, thus one of my opening comments about building flexibility into the framework. Not only will the business and technological environment that existed at the beginning of this effort have changed, but hopefully so will the IT organization (for the better). As organizations mature (as the result of the deployment and utilization of a service measurement framework), the actual metrics needed will change. This is because the factors within the framework’s dimensions will have changed and the required metrics and their usage will become more sophisticated.
In this step the Plan-Do-Check-Act approach to incremental (or in ITIL terms “continual” improvement) is suitable to iteratively validate the context of the framework, confirm its architecture and identify any modifications to the framework itself and how its implemented in the supporting processes and tools.
Summary
The famous military strategist Carl von Clausewitz once said that “… just because something is simple doesn’t mean it’s easy.” Developing and deploying a comprehensive service measurement system seems to be relatively straight forward and easily understood. However, actually building and operating it is a bit more complicated. While many consider measurement and reporting an art, it can be reduced to an engineering effort and enable a service provider to achieve a significant level of control over the capability and quality of IT service processes and the resultant quality of the services provided.
This leads to a final thought from Spinal Tap’s fictional David St. Hubbins, “Well, I don't really think that the end can be assessed as of itself as being the end because what does the end feel like? It's like saying when you try to extrapolate the end of the universe, you say, if the universe is indeed infinite, then how - what does that mean? How far is all the way, and then if it stops, what's stopping it, and what's behind what's stopping it? So, what's the end, you know, is my question to you.”