Enterprise Architecture: From Incite comes Insight...

Metrics are defined processes for measuring things. When creating a metric, be aware that people tend to modify their behavior so that measurements improve, even when there is no direct reward...



You will find many folks are not believers in the collection of metrics as they believe the overhead of collection outweighs the benefits they may bring. I am of the belief that metrics help in the following situations:

• clarify objectives


• evaluate final outcomes


• input into incentive schemes


• enable informed choice by consumers/clients


• indicate performance standards in licencing/contracts/monitoring


• evaluate effectiveness of different activities on dimensions of achievement


• trigger for investigation/action for improving quality


• determine most cost effective service level agreements for given target


• identify cost savings in attaining intermediate outputs

You may have noted that many enterprise architects within the blogosphere tend to blog on nomenclature of enterprise architecture with a strong focus on process while rarely ever talking about solutions in terms of changing behavior. Enterprise architecture needs to move well beyond talking about frameworks such as TOGAF, Zachman and others towards figuring out ways to measure the value of itself.

Gene Laganza at Forrester will be doing an upcoming briefing on collection of metrics that I encourage others to pay attention to. Wouldn't it be interesting if the smaller analyst firms such as Redmonk, The 451 Group, ZapThink and others provided guidance to large enterprises on metrics we should be collecting to uncover opportunities to embrace open source, SOA and other beneficial things.

In any undertaking, we usually only get to hear about successes and resulting best practices while ignoring any unintended consequences. Hopefully bloggers such as Robert McIlree, Scott Mark, Todd Biske and others would be willing to share not only successes within their own enterprise when it comes to metrics but also any unintended consequences in terms of collecting them.

In terms of ways to avoid unintended consequences, I recommend folks consider the following:

• involve folks at all levels in the definition of metrics relevant to them


• retain flexibility of use (don't make the job just meeting targets)


• review metrics for relevance frequently


• try to quantify all objectives even those who on the surface seem subjective


• measure satisfaction of all involved with metrics - remember reality isn't reality, perception is reality


• seek third-party expert interpretation of metric scheme


• use only a few indicators


• develop benchmarks independent of past activity

Coming up with a good metric is difficult. Here's a few things folks should noodle when considering creating a metric:

• What problem am I trying to solve?


• How will this metric help?


• What impact will this metric have on folks behavior?


• How will they subvert the metric to make themselves look good?