mezhermnt Handy Hints archive

#87 How to Bid a Performance Measure Farewell?

2008-06-18T16:47:00.001+10:00

There are a few reasons why you may have performance measures you just don't need anymore:

your strategy or goals have changed, and old measures are no longer relevant to what you're trying to achieve
you've spent some time designing more meaningful measures, better than the old ones
you've sharpened your focus and only need a subset of the measures you currently have

In any case, you may feel uncertain about what to do with the measures you no longer need. The idea of just throwing them out entirely might feel liberating, or it might feel a bit rash. But throwing them away is just one option.

Option 1: Throw them away.

Stop collecting the data you no longer need. Stop capturing the data, and stop reporting the measures that are based on that data. Totally eliminate the measures and all the effort behind them from you business or organisation. We do have a tendency to hold onto things we don't need, and they waste time and effort that is much better spent elsewhere.

For example, many contact centres are moving away from measures of average call handling time (AHT) in preference to first-call resolution (FCR), because the former drives behaviour to rush each customer, while the latter is much more customer focused. Why measure something at all if it drives the wrong behaviours?

Option 2: Give them to someone else.

Delegate the measures you no longer want to focus on, to someone who is better positioned to focus on those measures and take responsibility for tracking, interpreting and responding to them. Of course, they still need to be measures that are relevant to the goals of the business!

We also have a tendency to hold onto things that are better done by others. It will free up your time to give to more important performance results by letting go. Just like a CEO that delegates the measurement of operational results to her operational team so she can stay focused on the strategic measures, what measures should you be delegating?

Option 3: Put them on the backburner.

If you have clear definitions for your performance measures, where their calculation and source data are documented, you can safely stop actively reporting a measure knowing that if you ever need it again in the future, you just need to turn it back on again.

It's all still there in the background - just not distracting you as you focus on more important measures. The data is still there, the process for producing the measure is still there, you just have to compute some historic values and then start reporting again should you ever need to.

Finally, mark the occasion.

When you have decided which measures you're saying farewell to, and you've sent them off to someone else, to the backburner or to oblivion, mark the occasion by removing them from your business plans and reports too. Announce the change, and then get everyone refocused on the measures that now matter most.

#86 Are You Too Busy NOT To Measure

2008-06-05T17:08:00.003+10:00

"We think performance measures are important, yes, but we just don't have the time for it!"

You'll never hear these words uttered in a business or organisation that is truly successful. And that's because success depends on being able to make measurable progress toward your goals. It's not an accident.

Just look at Google, Marriott, 3M or even Fremantle Ports (winner of the Australian Business Excellence Medal in 2007). They have time to measure not because they're successful. They are successful because they made time to measure!

Beware the downward spiral of being too busy to measure...

If you don't measure what matters, you won't be managing and improving what matters. You'll be chasing what's urgent. And there's a big difference between what matters and what's urgent.

The more you delay getting meaningful measures that focus you on what matters, the more you're wasting time and wasting effort on things that don't matter most. The more time you give to what's urgent, the more you have to delay getting meaningful measures that help you focus on what's important. That's the downward spiral.

How do you make time to measure, when you don't have any time left?

When you're too busy to measure performance, you're actually too busy NOT to measure performance. You need to improve performance, or you'll stay too busy (and performance will just keep sliding).

So finding the time to measure performance is not about finding some spare time hiding somewhere. It's about deciding what you're going to *stop doing* to free up some time!

Aim to liberate a couple of hours a week that you can devote to selecting just a few meaningful measures and implementing them so they can highlight what matters most and help you improve what matters most.

Where can you find a couple of hours a week? Here are some ideas:

Get very clear about the results that you simply must achieve - the essence of your role or team or department. You can't prioritise anything unless you know what the results are that truly matter.

Curb distractions by planning your week and each day so that you've scheduled first the tasks that will make real progress toward your priority results. Schedule your 2 hours on performance measurement and performance improvement before you schedule anything less important.

Delegate or delete administrative tasks that don't directly help you make real progress toward your priority results. Who else could do your data entry, preliminary research or meeting organisation?

Stop reading emails that don't help you make real progress toward your priority results. It will feel wrong, but it's not wrong. Give it a try. You could easily find an hour a week with this tip alone.

Cancel or defer projects or initiatives (however exciting they may be) which threaten the time you can give to making real progress toward your priority results. If you're running several races at once, you can't win any of them.

Build a small buffer into each week, to handle those unexpected but essential tasks that pop up, or to handle the overrun of planned tasks that took longer than you expected. That way, they won't affect other important tasks you've planned (like performance measurement!).

The bottom line is that performance won't ever get better unless you make the time to measure and improve performance. And to make the time, you have to stop doing things that are less important.

#85 Three Measures of Personal Productivity

2008-05-20T13:36:00.002+10:00

Most of the talk about performance measures is how they are applied to monitor the performance of an organisation, a business, staff, a project, or a process.

But have you ever used performance measures for your own personal performance? One area that really lends itself to being measured is personal productivity. This is about how well we use our time to achieve whatever goals we've set. Of course, it virtually goes without saying (but let's say it anyway), if you have no goals then you have no way of assessing your productivity!

Personal productivity is something we all have to deal with, particularly when there are personal goals we're striving for, and no end of obstacles and distractions getting in the way of our striving!

So here's a handful of measures, over which you might like to ponder with the question "would it be useful for me to know this?"

Return on time invested (nickname is ROTI)

You have to define what the "return" is for you, and how you'll quantify it. Then you just divide that quantity by the number of hours (or days or whatever) that you've invested in generating that return.

If you're a consultant or solo business owner, for example, your return might be profit. So you could measure the profit your business earns for every hour you invest in your business.

It's a measure that easily leads you to the question "which activities give me the highest return, and how can I do more of those activities?"

Percentage of time spent on priorities

Do you know what your priorities are? Can you recognise when you're working on them, as opposed to things that aren't a priority?

If you can, then by keeping simple records in your Outlook Calendar or diary, you can easily tally up the proportion of your time each week that you gave to your highest priority tasks.

It's a great measure to really appreciate the extent to which we can let distractions and other people's priorities invade our time.

Task cycle time

If there are tasks that you perform time and again, such as preparing for meetings or writing a specific type of report, or recruiting staff, then a little careful analysis might highlight where you can save time that's currently being wasted.

It happens to the best of us - we get carried away with how things are done, and forget to check for better ways of doing them. Measuring the cycle time of your regular tasks can encourage you to ask this question (and hopefully find ways to improve your personal productivity).

These aren't the only measures of personal productivity, of course. But if you're currently measuring nothing about how well you use your time to achieve whatever goals you've set, could one or more of these be a good place to start?

#84 Thirteen Things You Can't Do Without Measures

2008-05-07T11:11:00.002+10:00

Measuring performance carries a stigma of being boring, threatening, tedious and difficult to do in a meaningful way. Often these perceptions can be enough to stop people from measuring, and even the clichés like "you can't manage what you don't measure" just don't have enough bite to get people to do it.

If you, or any of your colleagues, need some convincing about the benefits of measuring performance, that it is indeed worth the effort, consider these 13 compelling reasons as you think about one of the goals you've struggled to achieve up to now.

Reason #1: make your goals tangible and more real

Too many people have goals that are vague, motherhood statements that mean 19 different things to 7 different people. How can anyone possibly feel compelled by, or passionate about working toward a goal they cannot see in detail in their mind's eye?

Reason #2: achieve your goals faster

Feedback is the absolute essential process that systems of all kinds - airplanes, oak trees and the human body - use to keep functioning properly. Without feedback that is regular and relevant to your goals, you have no reason to expect to achieve the goal. The more regular and relevant feedback your measures give you, the faster you can act to correct your course toward your goals.

Reason #3: achieve your goals with less effort and waste

You could do the things to pursue your goal that you want to do, that you feel you should do, that others have done, or that seem like a good idea to do. But you will very likely waste effort because you don't know what is working and how much it is working without measures to pinpoint the real results of your actions.

Reason #4: keep your success far from the hands of chance, luck and superstition

Measures put you in a state of knowing what otherwise you would have guessed, assumed or simply never even considered about the progress toward your goals. Without measures, your goals are at the mercy of fear, hope, arrogance, ignorance, the latest fad, impractical theory, unchallenged assumptions and limiting beliefs.

Reason #5: where attention goes, energy flows

Earl Nightingale said decades ago, "you become what you think about". You cannot make real anything you are not giving your attention to through a very frequent routine. Having measures that you regularly monitor helps to keep your attention on the goals you want to make real (and off the things you don't want to make real).

Reason #6: find the points of highest leverage to achieve your goals

When you measure your goals, and see patterns in those measures like shifts or trends, you naturally ask why? And asking why is how you find the root causes of the results you're getting right now. Fixing the root causes (rather than treating the symptoms) gives you faster and bigger influence over reaching your goals.

Reason #7: know without question when your goals are truly achieved

Licking your finger and holding it up to the wind is not going to tell you if your goals have been achieved. Neither will the opinions of other people or your own, conveniently collected anecdotes or even other tenuously related existing measures. Measure your goal directly and deliberately, and never be in doubt as to whether or not (or how much so far) you've achieved it.

Reason #8: forecast future success objectively

Good measures will reveal to you patterns that have occurred in the past and in what ways these patterns are associated with various events, actions or circumstances. So what happens in the future becomes easier to estimate, when you plan how to influence future events, actions or circumstances (or not influence them).

Reason #9: prioritise your strengths and weaknesses

Who's got time to do it all? There will always be more opportunities, more problems, more ideas than we ever have time or energy to give to. Measures help us objectively sift out the strengths most valuable to exploit, and the weaknesses most worthwhile to improve on.

Reason #10: fail less often, and learn more when you do

Each goal your strive for, you will either achieve or not. When you don't achieve a goal, measures help you learn and understand why, so you have more wisdom in striving for future goals. And of course, measures also help you understand why you did achieve the goals you succeeded at.

Reason #11: take calculated risks, not reckless decisions

You can't really make a business successful without investing resources like time and money, and therefore you face the risk of losing these investments or not getting the return that would make the investment worthwhile. Measures help you choose the investments that will support your goals, by quantifying the return that would make the investment worthwhile.

Reason #12: learn faster through systematic testing

Because good measures of your goals give you regular and concrete feedback, you have the opportunity to test various solutions to achieve your goals. It helps you learn quickly what works and what doesn't, so you can do more of what does work and less of what doesn't.

Reason #13: find your core capability, your niche strength

Since measures can tell you what your business is good at, they will also tell you what you're exceptionally good at too. They can help you find what your biggest strength is, that possibly sets your business apart from competitors. Measured results are very convincing, so it's easier to market that niche strength to your advantage.

#83 De-Weasel Your Strategy

2008-04-27T20:41:00.001+10:00

I went to a seminar a few years back given by Don Watson, political speech writer and author of two rather confronting and very humorous books:

Death Sentence: The Decay of Public Language
Watson's Dictionary of Weasel Words

Both books are a strike at how the language we use in our society is communicating less and less meaning, despite that around 20,000 words are added to the English language every year! Watson, in both books and very lucidly at the seminar I attended, described how business, politics, and even schools and religious organisations are more and more choosing to use "weasel words" to communicate their organisation's purpose, intentions, goals, products, services and performance. Productive, process efficiency, effective, decruited, high value targets, engage, key stakeholders, best value, product offering, strategy elicitation, structural adjustment, sustainable future outcomes, endeavour, and on and on and on.

The implication for performance measurement? They say you can't manage what you don't measure, but before this becomes a relevant cliché to quip, ask yourself how you can even measure what you don't truly understand.

How many words in your strategy are weasel words?

Weasel words are inert, they are incapable of producing a reaction from those that hear (or even utter) them. Especially infected with this vocabulary virus is business, where inert language has taken over almost every corporate document and meeting room like a cancer. It's too easy to find examples to illustrate this point. The first example I noticed at a public display at the airport, and the others from five minutes exploring the internet:

"This unique partnership is a joint effort to leverage the research strength and depth of [the University] to assist [our airport] in better understanding its development objectives and impacts on the surrounding environment as we seek sustainable future outcomes."

"Instilling community confidence is an ongoing objective that aims to ensure we have a sustainable revenue collection system."

"The community housing system is proficient in delivering flexible, quality, cost effective and accountable housing services."

Now, for each of these examples, try to create in your mind a clear, vivid, colourful, life like, detailed movie that tells the story each example is trying to describe. You may have to read them a few times, you may have to guess at what some of the expressions mean, but you probably can't easily get this detailed movie. It's because the language used has left you cold, failed to inspire you, put you to sleep.

Have a look at those things you want to measure - your business goals or objectives or critical success factors (more weasel words!) - and hunt for words that aren't really saying anything, aren't telling the story about what you really what to achieve. Why are those words there? They aren't communicating meaning, which is the purpose of words, so what purpose (or person) are they really serving? How can you make the improvements or differences implied by the words, if people in your organisation aren't going to understand, feel inspired by and thus respond to those words?

What are you really trying to say?

So having caught the weasel words in your crosshairs, what do you do next? Basically, replace them. Use richer, simpler, more descriptive words instead to express your goals and objectives (and so on).

Try to involve your physical senses in the act of understanding the goal better. It's through our senses of sight, hearing, touching/feeling (and smelling and tasting, too) that we take in information about the world around us, and our senses are also a great basis for describing our experiences or ideas to others in a way that they can easily understand. And understanding your goal in sensory terms makes it easier to measure!

#82 When It's Impossible To Measure It

2008-04-08T23:04:00.002+10:00

When you're trying to measure a result and just can't seem to find the perfect measure, you're probably in need of a proxy!

What is a proxy measure?

Imagine for a moment that you're the Director for a government agency responsible for building and maintaining the roads in your state or country. And the politicians are asking you to prove that what you're doing is improving the quality of roads. How do you measure the quality of roads, exactly?

And now imagine that you're the passionate creator of an education program designed to scare the rev-headedness out of teenage drivers so they kill and mame themselves and each other far less often. What would convince you the program was working?

Imagine once more, that you own a business whose values include being environmentally friendly. Are you really going to put the effort into understanding, counting and calculating all your employee's carbon footprints?

Good proxy measures are stand-ins that can help you understand the results that matter to you, when you don't yet have the perfect way to measure them. Usually they are measures of correlated results, measures that you'd expect to behave reasonably in sync with a good measure of the real thing.

Why would you use a proxy measure?

Measuring results like those mentioned above poses several challenges that are big enough to drain your commendable intentions to objectively know, and reduce you to a feeble lump that utters "it isn't possible to measure this!"

One challenge is that the result is so broad it's almost a motherhood statement and thus near impossible to make it tangible enough to measure. What is road quality anyway? Is it how long the infrastructure lasts? Is it how comfortable and kind the road surface is for cars and their drivers? Is it a combination of these things?

Another challenge is how to get the data in a way that costs less (hopefully far less) than the value having the data would bring. Imagine the cost of following all those teenagers that attended your program, to track their incidence of road accidents. It probably costs more money they you have just to run the program!

And a third challenge is measuring a result in a way that the right people understand what it means and how they can do something about it. If you told each employee what their carbon footprint was and that they must reduce it, would they really be inspired to action?

Good proxy measures side step these challenges.

How do you find good proxy measures?

An ideal measure of the quality of roads would encompass all the dimensions of road quality - surface roughness, corner camber, corner sharpness, road shoulder condition, traction (there are loads of them). It takes a very long time to construct a meaningful measure like this. In the meantime, a proxy measure might be something like the mean number of kilometres between suspension repairs for cars registered in the locality. A well designed survey of local car owners and/or suspensions shops could inexpensively estimate this.

To get an ideal measure for the impact that a driving accident awareness program has on teenagers' road accidents would involve a study that followed participants in the program and compared their incident rate and severity with a control group of teenagers that didn't participate in the program. Costly. Instead, a proxy measure might be to track the trend in road accidents involving drivers aged between 17 and 19 years. Data readily available from police records.

And even though measuring your carbon footprint is all the rage for the green crowd, it's not nearly as practical and easily understandable as proxy measures like percentage of lunches that are take-away (think of all the packaging) or average number of days to fill the waste basket (if only full waste baskets are emptied).

Good proxy measures aren't perfect, but their trends help you know something about the result you're interested in, quickly and easily.

What are the risks of using proxy measures?

There is a price to pay for proxy measures. Often they only tell you part of the story, and often they are influenced by other forces that can bias or distort your understanding. Often you have little or no control over the data or calculation of the proxy measures.

But when you appreciate their limitations, they can still speak volumes to you about those hard-to-measure results.

Choose them consciously, from a starting point that involves really understanding the different angles or sides of the result you want to measure. Describe your result as tangibly as you can, even if it's not a complete description. Leave your worries about data availability until after you've listed and considered several possible proxy measures. And open your mind as much as possible, by involving out-of-the-box thinkers and giving yourself a few short sessions (instead of one meeting) to consider options.

#81 Five Dangerous Assumptions In Using Measures

2008-03-19T08:19:00.003+10:00

The most common methods of analysing data in business these days, and for quite some time into the past, has included moving averages (or rolling averages), trend lines, annual trends (using as few as 3 to 5 years), stacked line charts (with each line representing data for one particular financial year), tables of "% difference" comparisons between this month and this month last year, budget and year to date.

These analyses dangerously influence the interpretation made by their users - and in fact, if each of these analyses is applied to the same set of data, the conclusions drawn are almost always different for each of the analyses!

This seeming-anomaly is due to the fact that each of these analyses are based on assumptions that aren't all that sensible when you consciously examine them.

Dangerous Assumption 1: In "same month" comparisons, last year was normal.

This is when you take a look at performance this month, and compare it to performance in the same month last year, and say, "well things seem to have gotten worse (or better)!"

But honestly, it makes little sense to expect that every November should perform the same! To expect November's variation should be within ±5% (or 3% or 10%) isn't based on any sound reasoning. Just convenience. And how do you know that last November was normal, and a sensible benchmark for this November?

Dangerous Assumption 2: If Excel can calculate a trend line for a set of data, then there must be a trend.

Trend lines are calculated mathematically in a way that places a straight line as close to the middle as possible of a smattering of points in a time series. How far away the points are from the trend line is an indication of how reliable the trend line is in explaining the variation of the points over time.

Large variation means little reliability. And little reliability means no real trend.

Dangerous Assumption 3: The world starts anew on the 1st July every year.

Many analyses of performance focus only on the data from the current financial year, often because it is the overall end result of the financial year that people set targets for and are trying to therefore manage.

But usually this year's performance is a product of last year's performance, and the year before that, and so on. And it's this big picture that we need to understand if we are going to validly interpret what's really going on with performance this year.

Dangerous Assumption 4: The only probable cause for a difference is that something changed.

If you focus on point to point comparisons and your eye picks up a difference, that difference is not necessarily real. It doesn't mean that something has changed.

It might mean that the level of data integrity is less than 100% (it's rare to find any data that has 100% integrity). It might mean that the process or system producing the performance result has less than 100% control over that result (variation is a fact of life - no two things are ever exactly the same, mostly because we can't control every single factor that affects it). Most differences are not the result of a change - most differences are the result of natural variation in stable systems and processes.

If you use time series of 4 or 5 points of data, then you are really only making point to point comparisons, as opposed to truly understanding trends over time (for which you need at least 20 points in the time series).

Dangerous Assumption 5: All changes happen gradually - at least for the data we choose to put moving averages through.

The purpose to moving averages is smoothing out variation (particularly seasonal or cyclical variation) with a view to picking up long term trends.

These analyses are only capable of picking up gradual trends - not sudden shifts. They are not capable of identifying when a shift or trend began or finished. In fact, any analysis that smoothes out or removes variation from your data is a danger to valid interpretation. You have to see all the data in order to assess if there is a real change, and what kind of change it is.

What should you do instead?

The common thread with all of these 5 dangerous assumptions is that none of them help you sort out the two types of variation in your performance values:

normal variation, which occurs even when there is no real change in performance (it's just random noise)
abnormal variation, which occurs when a real change has happened to make performance better or worse (such as a process improvement you've made).

So there are two things you can start doing now to make sure you can see and distinguish both forms of variation in your performance measures:

look at a larger window of values - say a time series of at least 20 values in a row (e.g. about 2 years of monthly values)
graph the performance values as they are, don't use trend lines or rolling averages or cumulative calculations

If you want to learn more, Donald Wheeler's book "Understanding Variation: The Key to Managing Chaos" is a tremendous resource.

#80 Should You Use A Performance Index?

2008-03-04T22:39:00.003+10:00

How cool would it be to have just one, single, solitary performance measure to tell you everything that's going on with your business or organisational performance? You just have to look at this one performance index, and immediately know where you're heading, and what to do about it.

Hmmm. Sounds a bit too much like a crystal ball, the kind that a dark-haired, hoop-earinged russian gypsy would gaze into and see your future. Yet so many people love the idea of the performance measurement equivalent to the crystal ball - the performance index.

What is a performance index?

When you take a raft of performance measures and mathematically combine them into a single figure, you've got yourself a performance index.

This raft of performance measures might be a collection of customer survey questions that get rolled up into a pseudo "overall satisfaction" index. Or perhaps a department's or business group's entire suite of performance measures, cleverly combined into a "current status" indicator.

Sometimes some extra magic is woven into the performance index, in the form of weightings that are given to each performance measure involved, with the intent of allowing the more important measures (or sometimes the better-performing measures) to have more influence in how the overall index looks.

Why do you want to have a performance index?

Too many performance measures to review and analyse is one of the most popular reasons behind the pleas for a performance index. "I don't have time for all these measures! But I don't want to miss out on anything either!"

The performance index is often just some ointment applied to treat symptoms of a deeper problem that many don't realise can be solved directly.

If you have too many measures, then perhaps you're not sharing them around enough among your colleagues, or perhaps your measures aren't displayed in a way that makes for fast and easy and accurate interpretation at a glance. Perhaps you're measuring things that you just don't need to give your attention to?

What price will you pay for a performance index?

The trends of each of the underlying performance measures in your performance index cancel each other out. For example, if cycle time is improving and customer satisfaction is declining, the net effect of these two measures on your performance index is highly diluted. But these are two very important signals for you to see and respond to, that the performance index is hiding from you!

And even if most of the underlying performance measures did lean the same way, and your performance index thus showed some kind of signal itself, what would you do next? You'd probably ask "why?" and then go searching for the answer. And how do you find the answer? You have to detangle the performance measures from the index to find causes. The performance index is just giving you extra work, not saving you any.

Should you use a performance index?

In a word, no. Certainly avoid using a performance index as your first solution to a performance measure overload problem. Much better for the performance of your business or organisation will be your taking time to ensure that you're only measuring the things that matter most, that you're able to quickly and easily and accurately detect the signals in your performance measures, and that you've got readily available information to make cause analysis a natural part of using your performance measures.

Remember that performance measurement is about valuable feedback to inform your future choices and actions toward improving the performance of your business or organisation. Most performance indexes don't have the power to do this.

#79 Five Tips For Using KPI Master Lists

2008-02-20T17:28:00.003+10:00

When you start trawling through all the lists of KPIs (or performance measures, which is the term I prefer to use) that are lurking on the internet, you'll see just how many performance measures there are. Thousands. Even within just one industry.

And many of these so-called performance measures aren't in fact measures at all. A performance measure is evidence of the degree to which you are achieving an important result, over time. And that brings us to the first tip for making the best use of KPI master lists:

Tip 1: Know how to recognise a real performance measure from a fake.

"Existence of Business Plans" is not a performance measure. It's the result of an action. "Employee Survey" is not a performance measure. It's a data collection instrument. "Monthly Average Delivery Cycle Time" is a performance measure, because it gives feedback of a result, over time. You'll need to be wise enough to pick out the real measures from the fakes.

Tip 2: Before you even go looking for KPIs, clearly define your priority business results.

To avoid swamping yourself with way too many trivial and irrelevant measures, you need to fish these KPI master lists with a net designed to catch just the types of measures you need. And that net is made from the business results that are your highest priority to achieve. After all, that's what KPIs are for: to give you feedback about the results that matter.

Tip 3: List potential measures first, then evaluate to select the best.

It will be faster and easier if you list a dozen or so potential measures for each important business result that is worth measuring. So trawl through the KPI master lists quickly, just to pull out this first list of potential measures. After that, evaluate the potentials, and decide which measure (or two) will give the best evidence of your business results.

Tip 4: Be prepared to tailor the measures to your business.

The vast majority of KPI master lists will offer you nothing more than a measure name, and if you're lucky, a short sentence describing what it means. That's just not enough for you to successfully implement the measure in your business. So be prepared to flesh out each measure in more detail, right down to the data you'll use to compute it.

Tip 5: Don't fit a square peg to a round hole.

It's entirely likely that your business or organisation is unique is some way. And that means that the complete set of performance measures it needs won't be out there in someone else's KPI master list. You'll have some results for which you'll just have to design your own measures, from scratch. It will take you an hour or so to do this, but that's much better than wasting weeks, months or years measuring the wrong things, or not measuring the right things at all.

#78 Three Tips To Reach Your Priority Performance Targets

2008-02-07T08:46:00.001+10:00

Don't think you're being a hero by having dozens of performance measures and targets, and driving hard to achieve them all. You might say "well all these measures are really important and I just have to achieve them all". You're fooling yourself, because all the evidence says that those who prioritise ruthlessly, achieve far more of their goals, and far bigger goals.

Tip 1: Make only a few measures the priority at any one time.

Usually you're measuring a goal or result because you're NOT currently achieving it to the level you really want or need. And that means that your business or organisation doesn't yet have the innate capability to do it well. When you can't yet do something well, trying to focus on dozens of things is like trying to learn to swim in a giant washing machine.

The more priorities you have, the fewer of them (if any) you will do justice. Each month or quarter, choose just your 3 to 5 highest performance priorities and give your attention to them first.

Tip 2: Display your priority measures everywhere.

You know what they say: out of sight, out of mind. Successful athletes, business people, celebrities - they all know that you have to keep reminding yourself what matters most. Every single day. At least once, every single day.

So carry your priority performance measure graphs and targets around with you, put them before anything else on management meeting agendas, hang them up on the office walls. One CEO I worked with hung laminated poster-sized graphs of his 3 most important performance measures right outside his office door, updated daily by his personal assistant.

Tip 3: Practice focusing on your priority measures.

While keeping your performance priorities in clear site all the time can actually make some real progress toward achieving them (a phenomenon of the reticular activating system [1] in your brain), if you want to truly achieve those results, you have to take deliberate action.

Practice every day focusing on your priority measures. Read your priority goals and visualise them being achieved. Plan each day to ensure you are dedicating at least one task to close in on priority targets. Ask colleagues and staff what they are doing to improve the priority measures' performance. Check and test if actions you and others are taking are having some real and objective impact.

Performance improvement - no matter how big - starts with you.
Changes - like business performance improvement - just won't happen without clarity of thought, focus, and the discipline to act. Which changes are currently most worthy of your daily discipline and effort?

[1] see http://www.make-your-goals-happen.com/reticular-activating-system.html

#77 Stop Measuring What You CAN Control

2008-01-16T17:15:00.001+10:00

Yes, you did read that title correctly. No, it isn't a typing error. Yes, it does go against everything you've been told about performance measures.

But, if you truly want performance measures (or metrics, if that's what you call them) to really make a difference to your business or organisation, you have to STOP measuring what you can control!

You're going to get far more improvement, far more learning, much faster progress toward your goals if you start measuring the results outside your complete control, the results you only have some INFLUENCE over. Here are 3 reasons why.

Reason 1: What you can completely control is trivial.

What do you really have complete control over? To a great degree, you can control what you think about it. To a lesser degree, you can control what you do and how you do it.

About these things, no-one really cares. Changing these things makes little difference to results that are bigger than yourself. What you can completely control is inside the boundaries of your comfort zone and what you already know, so any change is only incremental and small.

The important stuff is important BECAUSE it's outside our circle of control.

Reason 2: Performance is more than just personal.

Improving what you have complete control over reduces down to improving your own skills and knowledge and competence. In business, performance is about much more than that. Everyone having more skill, more knowledge, and more competence does not guarantee business success.

Performance that matters is mostly about the business results (like profit, stakeholder value, customer loyalty) and the processes that produce these results for the business (like sales, customer support, recruitment). And no one person has complete control over any of these. It's a team effort.

Reason 3: You are creative enough to stretch beyond control.

If you never stand face to face with a result you are passionate about but feel you can't control it, and ask yourself the question "how can I make this possible?" then you are denying your innate creativity.

Why not at least ask this question of one important result outside your control? How can you make it possible? How have others made similar results possible? What ideas do others have for making it possible?

Another implication: reward people for influencing, not just for achieving.
One of the main reasons so many people feel compelled to measure what is in their control is the fear of personal loss if they don't achieve their measures' targets. So they choose easy measures that have very easily attainable targets. This is just jumping through performance hoops, and not about the true purpose of performance measures (to improve important performance results).

So what if you could start rewarding people for how much they try to influence results that aren't inside their control, even if they don't get all the way to the target?

#76 Five Steps To An Achievable New Year's Goal

2008-01-03T07:40:00.001+10:00

The time of year when goals are most discussed is at the turn of the New Year. We get fired up and motivated to achieve new personal New Year's Resolutions for our health, our hobbies, our spirituality and our businesses.

However, the motivation can easily wane if we lose focus on our goals, if we feel overwhelmed by the effort it really takes to achieve them, or if we don't see immediate results on our first attempt.

So it's important to realise that these New Year's resolutions are just the same as any other goal we set. If we treat them like real goals, they'll more likely become real!

Step 1: Have no more than 3 really important New Year's goals.

Too many goals means too much scattering of your attention. If you've struggled to achieve all your goals in the past, then ask yourself: "Is it better to have a lot of goals and achieve none, or have a couple of goals and achieve one?"

Step 2: Make each goal super-specific.

No pie-in-the-sky wishes (Santa should have taken care of them already). Make your New Year's goals so detailed that you can create in your mind the exact experience of them already being achieved. "I want to be fit and healthy" is a pie-in-the-sky wish. "I will attain and keep my weight at 55kg" and "I will comfortably run 5km in under 25 minutes" are super-specific goals.

Oh, and write them down in this super-specific language, somewhere you can see and read them every day.

Step 3: Check there are no problems associated with having each goal.

It's called systemic thinking, when you ask yourself questions like "what if?" or "what else?", and these are important questions to ask about any goal you set for yourself. Will achieving the goal be all good, or are there some nasty side effects that could mean it's better to rethink the goal, or abort it altogether?

Step 4: Measure each goal!

Having a goal without measuring it is like watching a movie with your eyes closed. You will need regular and ongoing feedback about how fast you're dropping the pounds or what your average 5km run time is week on week.

And because there will always be ups and downs, measuring makes certain you stay objectively focused on the overall trend.

Step 5: Know what you're going to do differently to achieve your goals.

"Action! Nothing Happens Until Something Moves" is the title of a book by Robert Ringer, and they are wise words. You will not achieve your goals without making some distinct (and usually uncomfortable and challenging) changes in what you do each day. It's not fun saying no to a chocolate cupcake, but you must. And yes, especially if you don't feel like it, you have to put on your running shoes and start moving.

Each of these steps requires discipline. Are you prepared to give the discipline in return for achieving your goals? If not, perhaps you have the wrong goals. But if so, there's no time like right now to get started!

#75 Eleven Powerful Measurement Insights

2008-01-03T07:35:00.001+10:00

The greatest management thought-leaders in the world insist that measuring the performance of your business or organisation is essential to its succeeding. There are no qualms about that. If you want to improve the performance of your business (or anything), you must measure performance.

Fewer and fewer managers are struggling with this premise, that you have to measure performance to manage it. But what they do struggle with is how to do measurement properly. These eleven insights will guide you to improve how you do go about measuring performance.

Insight 1: Only measure what you're going to do something about.
Don't measure just because you can, just because you always have, just because you've got the data, just because someone says to. Measure only the results you are going to give your time to improving, by "working on the business, not just in it".

Insight 2: Measure drivers, not just outcomes.

It's great to know how profitable your business is, or how well you've kept to budget, or how happy your customers are. It's at least as important to know also what operational results have the most influence over these outcomes. It's those drivers that you can do something about, to get the outcomes you want. You can't influence outcomes directly. So find the drivers, and measure them too.

Insight 3: Measure not what you can control, but what you can influence.

No-one really has control over anything other than their thoughts. We do have a lot of control over what we do, but even extraneous factors can limit that control too. If we only measured what we could control, we'd be measuring useless things. So expand your thinking to what you can influence, and you'll find yourself measuring much more meaningful results. Remember, your target doesn't have to be 100%.

Insight 4: Measures impervious to change are useless.

Why do call centres continue to measure the number of calls received? It's not a performance measure - it doesn't measure how well the call centre is performing. It just tells them how many calls they're getting. And even if they could change this number in some way, what kind of change would reflect an improvement anyway? Measure only the results you know you can (and should) change for the better.

Insight 5: It is essential that your measures conflict with one another.

Everything is about balance. Cycle time versus quality, profit versus customer retention, employee satisfaction versus productivity. These things are in conflict with one another, and that's how it should be. Management is about balancing the conflicts that are important, so if your measures aren't in conflict with one another, then you're missing essential information to manage.

Insight 6: Think like a marketer to engage people in measuring.

What do great marketers do? They capture people's attention, they get the right message across, and they influence people to act in a way consistent with that message. Traditionally, we think of marketers as selling products or services. But why can't we use the same process to sell performance measurement? Consider collaborating with your marketing department, or learning more about marketing yourself, to increase the engagement your colleagues have in performance measurement.

Insight 7: There is no "set it and forget it" with measuring performance.

There is no set of industry standard performance measures you can buy off the shelf, bolt onto your organisation or business, and then sigh in relief that you've "done performance measurement". Performance measures are a reflection of the things that matter most, and the things that matter most are a reflection of what's going on in your business and it's environment. And in case you haven't noticed, this is constantly changing!

Insight 8: Reward people for local results AND organisational results.

Reward people just for local results, and you'll be encouraging them to compete internally with other departments and teams and individuals, or cause unintended consequences for them. Reward people just for organisational results and you'll be frustrating them with the expectation to influence results they can't directly influence. Reward people for both local results and organisational results and collaboration across traditional organisational boundaries toward common goals is what you'll get.

Insight 9: Use data and not opinion to determine causality.

Sitting around the meeting room table to discuss an increase in error rates (or cycle time or costs or whatever), everyone's got an opinion about why. We're so quick to find solutions that we often forget to define the problem properly. A proper cause-effect analysis has to involve scoping potential causes and using data to determine which are the most influential causes.

Insight 10: Measuring performance is not a tool, it's a way of life.

If you've been interested in performance measurement for more than a few months, you've probably already discovered that it's not all about numbers and data. Mostly it's about culture, a culture of results-orientation, feedback, learning and continuous improvement. It's not enough to learn the tools and steps of performance measurement, you need to live the philosophy.

Insight 11: Performance measurement requires humility and transparency to work.

Ego, fear, arrogance, carelessness and sloppy thinking lead to performance measurement attempts that fail because bad results are swept under the rug, data is manipulated, only good results are measured, and any kind of objective evidence is ignored in favour of intuition and experience. Those who are humble will learn from the valuable feedback measures offer, and those who aren't afraid of transparent feedback will turn their performance measures into performance improvement.

#74 Five Time-Saving Tips To Manage Performance Data

2007-11-21T12:34:00.001+10:00

If you knew in advance how much effort you'd have to put into collecting, collating and managing the data that provides the foundation for your performance measures, you may be put off entirely. Particularly for smaller businesses, departments or teams, you don't have big corporate business intelligence systems offering you the data you need at the press of a button.

No, you're often forced to collect and organise the data you need, all by yourself, and on top of your real job! But don't give up on measuring performance just for this reason. If you're still a firm believer that meaningful performance measures make your day job better, then these 5 tips for saving time in managing your performance data may just make the difference.

Tip 1: Collect data that is useful, not just interesting.

Market researchers are used to hearing their clients say things like "It would be interesting to also know [blah blah] - can we add a question about this to the survey too?" And good market researchers will say in response, "Not if it's just interesting. It has to be useful." They know the time and cost that gets wasted collecting data that doesn't serve the purpose of the research.

If you collect performance data that strays from the purpose of your performance measures, you'll collect much more than you need, much more than you'll use, and probably enough to paralyse your performance measurement process. Be ruthless: collect only the data you know is useful to calculate and analyse the performance measures that really matter.

Tip 2: Build the data collection into work processes.

Imagine you operate a health club and you're interested in measuring how committed your clients are to their gym programs. The time-wasting way to get the data for such a measure is to devise a survey and get your staff to call as many clients as possible to go through the questionnaire. And this is over and above their everyday work, too!

A time-sensible way is to see where the data might already exist or be easier to get through existing work processes. Personal trainers already fill out gym cards for their clients, to track the sessions their clients complete. Making sure that this data is collected consistently avoids the need for a separate data collection process.

Tip 3: Use a relational database to manage data (not spreadsheets!).

A freight business has their delivery turnaround times not in one spreadsheet, but in dozens of them: one for each month. And they do little better with other performance data such as revenue, delivery misdirections and service costs. They waste hours and hours every month manually organising the data in an attempt to produce trend graphs to report performance.

Even small businesses, departments and teams can recoup the time wasted in manually managing data with an investment in a simple database application, like Microsoft Access. All the data goes into well designed tables, via easy to use data entry forms. Not only is the data all in one place, it's fast and easy to get access to for both regular performance reporting and ad hoc queries to analyse the data more closely.

Tip 4: Don't freak out over imperfect data.

A management team in a government agency routinely spends well over half of their decision time debating the quality of the performance measures' data. What is the real cost to the organisation of losing time in delaying a decision until data is perfect, compared with taking a decision with imperfect data now?

Data will never be 100% accurate, and it doesn't have to be. Imperfect data can still give you rather reliable feedback about trends in performance. Take a quick check for any systemic data quality problems, to estimate their real impact on the decisions you are taking. After correcting any vital data quality problems, your time is better spent in cause analysis and performance improvement of your business results, not perfecting your business data.

Tip 5: Use samples to estimate, instead of populations to calculate.

To measure the accuracy of their stock management process, an inventory management team were visiting all store locations and counting all stock items held. This took them many months, and almost a full time team to chew through all the counting.

When they sought some advice from a statistician, they learned (with great relief) that they could get a very reliable estimate of their stock management accuracy through stocktaking a sample of stock items at a sample of store locations. In fact, with the smaller task, the counting process had fewer fatigue related errors, and the estimate was not only cheaper and faster, it was more reliable.

Spend your time where it matters most

Take a little time to think about what data you really need, how much integrity it honestly needs to have to be useful, the easiest and fastest way to collect it, and an accessible way to store it. It will be time well spent, which will create more time down the road for the real work of performance measurement: improving your business performance.

#73 Why Balanced Scorecard IS NOT A Measurement Tool

2007-11-08T09:20:00.001+10:00

If you haven't yet heard of the Balanced Scorecard, it's a world-renowned model developed by Robert Kaplan and David Norton in the mid 1990's to encourage corporations to realise that relying on just the financial metrics is relying on too little, too late. They wrote the landmark book, "The Balanced Scorecard: Translating Strategy Into Action" to demonstrate how to measure corporate performance in a more balanced way, that aligns everyone in the corporation to its strategic direction.

But does it?

The Balanced Scorecard has well-known implementation problems.

There has been much criticism of the Balanced Scorecard by organisations that have attempted to implement it. They hoped to quickly manifest a meaningful suite of performance measures that has everyone focusing on what matters most in fulfilling the mission, vision and goals. But didn't.

Implementers of the Balanced Scorecard find they end up with too many measures in the tangible parts of their scorecard, not enough measures in less tangible parts of their scorecard, and altogether not measuring meaningful results. Why does this happen?

The Balanced Scorecard doesn't put enough emphasis on results versus strategies.

The primary building blocks of the Balanced Scorecard are 1) it's four perspectives of financial, customer, internal business process, and learning and growth; 2) the strategies that populate and link through each perspective; and 3) the measures that link to each strategy.

Most people express their strategies as vague, jargon rich actions, and struggle to tease out the specific and tangible results implied by these strategies. So they measure what is easy to measure: progress against planned activity, the reaching of milestones and whatever else they have data for. The measures focus on activities like "develop target markets" and "upgrade staff competencies", not outcomes.

The Balanced Scorecard doesn't offer the steps to measure design.

Bordering on prescriptive, the Balanced Scorecard literature offers ideas for measures to use, for specific strategies typical of each of the four perspectives. Functionality. Brand Image. Relationship.

But these terms mean such different things to different organisations and people, so to measure them meaningfully takes more thought than brainstorming or copying from a book. It takes a thinking process that extends from an intimate understanding of the result that needs measuring, and this often requires climbing out of mental ruts like "this isn't measurable" and "we don't have the data".

The Balanced Scorecard doesn't show the process of measure implementation.

Populating a balanced scorecard with measures assumes that the populating process is obvious and straightforward. Select some measures, then give the IT department the brief to report them. "Add customer retention to the scorecard, thanks!" is just too brief.

You can't ignore the many details that can (and do) make the difference between measuring what's easiest versus measuring what was meant to be measured. There are many subtleties to measuring something like customer retention, which need to be drawn out into a full definition, from which the measure can successfully be brought to life.

So is the Balanced Scorecard all bad?

No, the Balanced Scorecard is not all bad - it will go down in history as the serious beginning of non-financial performance measurement in the corporate world. And its heart is really the roadmap for linking what happens from day to day in a corporation to its strategic, longer term direction, through deliberate conversation about strategy design. But it's not about performance measure design and implementation.

It's just not specific enough to stand alone as a performance measurement tool.

To select truly meaningful measures for your scorecard - beyond the traditional profit, customer satisfaction and employee turnover - you need to learn a way of thinking about measurement that Kaplan and Norton didn't address. To make your scorecard a success, couple the strategic design and translation power of the Balanced Scorecard, with a deliberate performance measurement process like PuMP.

#72 Four Performance Improvement Set-Backs To Avoid

2007-10-16T20:22:00.001+10:00

You know the ultimate reason why we measure performance - why we have metrics and KPIs - is to improve performance, don't you? But has just the thought of the time, money and effort you'll need to improve performance, over and above your day job, almost got you running for the hills?

This is why pilot testing is such an invaluable practice to adopt, each time you discover a performance result that you believe is important enough to improve. Pilot testing is a small scale implementation of an improvement initiative, to learn how to make the full scale improvement get the best return on the time and money and effort you invest.

The power of pilot testing lies in minimising the effect of "Gumption Traps", what Robert Pirsig, author of Zen and the Art of Motorcycle Maintenance calls the set-backs that throw us off our focus on Quality, on producing meaningful results.

So here are some tips for how pilot testing can help you prevent the set-backs of performance improvement:

Set-back 1: the wrong hypothesis about what really improves performance

If you have picked just one improvement idea, you've put all your eggs in one basket. You run the very high risk of that idea not in fact being the best idea to improve performance.

Instead, consider a range of ideas and test them as alternatives on a small scale. Then, after you pilot test them, look objectively at the impact each idea actually had on your results. You can then more confidently choose the best idea to implement on a full scale.

Set-back 2: not enough support to agree to a performance improvement

A decision maker is going to support improvement initiatives that are much more than just good ideas. Before she's going to allow resources to be taken away from something else and given to your performance improvement idea, she'll want to know it's going to work.

Rather than expecting decision makers to agree to an improvement idea because you think they should, demonstrate the impact of the idea with the results of a successful pilot test. Don't forget, a measure of return on investment (ROI) is a convincing way to measure success.

Set-back 3: loss of momentum when improvement takes too long

Some performance initiatives can take months or years to implement, and almost always the time people have to give to improvement is over and above their full time job. It's no wonder the momentum can wane.

Using the pilot test approach of keeping the timeframe and the scope small, you can make performance improvement initiatives happen as fast and focused as possible. When people see results happening quickly, their energy and motivation grows and adds to the momentum for continued improvement.

Set-back 4: when implementation goes awry

Any change you make to a process is going to be fraught with implementation bugs: incompatible systems, people misunderstanding, a step taking too long, hidden costs, and so on.

Pilot testing is tremendous for finding and ironing out implementation bugs in a controlled environment - where cleaning up the mess they create is no big deal. When you find the bugs, and decide how to fix them, then you can unleash your improvement initiative to its full scale.

#71 Are Your Graphs Wasting Everyone's Time?

2007-10-02T16:34:00.001+10:00

Software these days can do heaps of exciting and creative things with your data, including summarising it, graphing it and making it flash and ding to get your attention.

Because all this can happen at the touch of a button, it's often hard to remember that software can't reason and therefore can't decide what is a meaningful and useful thing to do with your data.

These principles of useful graphs will guide you so you can decide what is a meaningful and useful thing to do with your data.

Principle 1: answer your question

When you choose a graph to convey information for a specific purpose, be clear about what the purpose is and choose measures that are relevant to that purpose. For example, make sure your graph title explains the measure presented, and make sure the measure presented really is the relevant measure for your decision.

Principle 2: highlight comparisons

Graphs are a visual representation of objective comparisons. When you graph measures, it is important to use techniques that help the user make comparisons in the information you are graphing. For example, use line charts for time series measures, use bar charts to compare departments or segments or other categories with one another, avoid pie charts altogether.

Principle 3: optimise simplicity

Keep graphs simple and easy to interpret by avoiding superfluous symbols, laborious legends, dispensable decorations and volumes of information that take the audiences' attention away from comparisons. For example, remove gridlines, don't bother with data labels and forget the artwork (like little dump trucks instead of data points).

Principle 4: maintain integrity

Help the data tell its story. If this story is unclear, then say so (as opposed to putting the data through contortions and manipulations for the sake of having an answer to your question). For example, be careful how you scale the axes, don't use 3-dimensional effects and don't assume every trend is a nice straight sloping line.

Principle 5: be visual

Pattern, colour and space are the fundamental characteristics of graphs. These characteristics are processed and given meaning through a visual process, so use them in ways that human eyes (and often photocopiers) best respond to. For example, avoid patterns that don't start to move the more you stare at them, use colour to highlight signals in the data and size your graph only just large enough to convey its message.

Tips to transform your graphs, easily and quickly

How easy is it to transform your graphs? Very easy! You just need to know the practical tips that show you how to change parts of your graph's anatomy to make your graph more relevant, informative, simple, trustworthy and easy to interpret for decision making!

#70 How Many Measures Do You Need?

2007-09-19T21:59:00.001+10:00

If you're starved of any decent performance measures at all, or you're drowning in the overwhelm of too many, then you're also likely not making any decent progress on improving your business performance, or achieving your goals.

As mere mortals, we seem to get the best results when we can focus on just a small number of priorities. Remember this as you make that important decision about exactly how many - and which - performance measures will focus you to achieve your priorities, as properly and promptly as possible.

TIP #1: Prioritise your goals, and just measure the priorities.

Each quarter, I review my business goals and Results Map to set the 1 to 3 most important themes to focus on for the coming few months. I don't monitor everything in my business plan all the time. The data is being collected for these measures, yes, but I'm not reviewing those measures. Not until and unless they become a priority.

So it can help to think of your entire business plan as a rolling evolution. Your focus is always just on the most important goals, and as those goals are achieved, your focus moves to the next most important.

TIP #2: Expect that for each goal, you may need 1 to 3 measures.

Rarely does a single performance measure give you a complete enough picture of a goal or result you're chasing. For example, I use two measures to monitor a goal I have, that my customers spread the word about the value of my services: Net Promoter Score and Number of Customers Referred by existing customers. It means I can measure my customers intentions to refer and how this manifests into reality.

So make allowance for this when you prioritise which goals are the priorities to measure: each goal could have a few measures attached to it.

TIP #3: For any one person, 3 to 5 measures is enough.

Reaching performance targets, as with any form of change, takes effort over and above the doing of our everyday tasks. It takes thought, learning, analysis and experimentation, and time. If you try to improve everything that matters all at once, the effort is exponentially greater.

So at any one time, having just 3 to 5 measures with targets on your radar is enough. Get some real improvement in even one area, and you're streets ahead of making insignificant improvements in many areas.

TIP #4: Be disciplined to focus on the priority measures first.

Especially if you are reporting more than just the priority measures, the priority measures should be at the top of your performance review agenda, if not the entire agenda.

For example, my weekly reviews focus just on the priority measures for a single performance theme until I've achieved my targeted improvement. The theme comes from my quarterly review of priorities, such as growing the number of subscribers, growing revenue from a new product, or improving my customers' service experience.

TIP #5: Use monthly and quarterly reviews to check for new priorities.

With a strong focus on a few measures, you can make some great progress quite quickly. And as this happens, those few measures can become a lower priority relative to others. So a regular review of your complete business plan or set of goals is useful to choose new priorities.

But remember to let go of old priorities, take them off your radar. Your monthly review will pick up if they ever need your attention again.

A tool to prioritise your measures

The first step of measuring the right things is to define the results that are worthy of being measured. We use the technique called Results Mapping to do this with our clients. And I use it personally too, for my own business goals and measures.

Results Mapping lays out all the results that are implied by your business plan or goals (what's important to your business or organisation) and the cause-effect relationships among them. Each result can have 1 to 3 measures to monitor it.

It's all on a page, and this makes it easier to choose and visually highlight which results are the priorities right now. And it's these priority results we measure and monitor, until improvement targets are achieved. After that, the priorities are reassessed, and the highlighting changed accordingly.

Here's how to get the Results Mapping tool for yourself, go to: http://www.staceybarr.com/pumpshop/howtokitP1001.htm

#69 What Does 'KPI' Really Mean?

2007-09-10T21:58:00.001+10:00

Do you need KPIs? What about targets? It goes without saying, you need goals and objectives, doesn't it? You'll need some measures and metrics as well, won't you? Or at least some indicators.

Good grief! It sounds like a performance management shopping list! But even if there were performance management shopping malls around, I'll bet a lot of people wouldn't find what they were looking for. And that's simply because, they don't really know what all these things actually are!

Let's explore these terms, and see if we can't get some more clarity about how they are different from one another, and why.

A measure is a quantification of something you can observe, such as the proportion of customers that have purchased from you more than once or the average time it took you to ship customer orders last week.

Tip: If a measure is not measurable, then it can't really be a measure. You've probably not made the "something" specific enough to measure.

A performance measure is a type of measure that quantifies an important result, that impacts a business' or organisation's success. Performance measures are usually taken regularly over time, so that changes in performance can be determined. For example, as time goes by, does it take us less time or more time to ship customer orders?

Metrics and indicators are measures too There really isn't a strong enough body of knowledge out there to give them their own unique meaning. Measures are indicators too, in that they indicate the extent to which something is happening. They are rarely precise or exact, due to constraints with data collection, measure definition and analysis methods. But precision in a measure is not as important as the measure being able to give us useful knowledge we otherwise did not have.

KPIs, or key performance indicators, are measures too. But generally, KPIs are the most important measures for a business or organisation, usually having the highest leverage to impact its success. Due to their catchy name, the term KPI has almost become synonymous with the term performance measure.

Tip: I prefer to use the term 'performance measure' as the generic label for any quantification of a performance result.

A target is not a goal or a performance measure but rather a value that represents a level at which you want performance to be. If you have a performance measure that tracks the average number of errors in your invoices, then a target for that measure is also an average number of errors. But while your measure shows that currently you have an average of 0.14 errors per invoice, your target would represent an improvement on that, which could be 0.05 errors per invoice.

Objectives and goals are statements that link performance measures to targets and timeframes. "Reduce the average number of errors per invoice to 0.05 by December 31 2007" is an example of an objective, or a SMART goal. So it's ingredients are a performance measure (average number of errors per invoice), a target (0.05) and a timeframe (December 31 2007).

Tip: There is no one true definition of terms like these, so you might find it useful to make your own definitions clear in your organisation, to guide how they are understood in practice. But make sure you do some research first, to scope how others use these terms, to widen your awareness.

#68 The Power Of The 80/20 Rule

2007-09-10T21:55:00.001+10:00

In 1897, over a century ago, an Italian economist called Vilfredo Pareto made the discovery that 80% of the wealth of a population was owned by 20% of that population. The Pareto Principle, or 80/20 rule, is a pattern of predictable imbalance that keeps popping up in all kinds of contexts ever since. For example, the Quality Movement certainly grabbed onto it, coining phrases like "80% of the problem is caused by 20% of the causes."

Why the 80/20 rule is so important to us today.

The 80/20 rule is a guide for how to think about improving things. If you want something to improve, you have to change something. And with even a basic 80/20 analysis, you can find out what those "somethings" are, which will have the greatest impact.

And in an age where time is scarce, and to-do lists are long, the sharper our focus can be on what matters most, the better our results will be.

What the 80/20 rule can apply to.

If you're not completely happy with how your work or life is going right now, or you know that there is scope to improve it, then practice asking questions like these:

* Which are the 20% of tasks I perform that generate 80% of my output?
* Who are the 20% of customers that generate 80% of our profitability?
* Who are the 20% of customers that have 80% of the need for our services?
* What are the 20% of products that generate 80% of our profitability?
* What are the 20% of investments I make that generate 80% of the return?
* What are the 20% of interruptions that cause 80% of my productivity problems?
* What is the 20% of literature I read that gives me 80% of the knowledge I need?
* Who are the 20% of suppliers that give me 80% of the goods and services I need?
* What are the 20% of complaints that take up 80% of my complaint handling time?
* Who are the 20% of friends & family that get 80% of my attention?

Asking questions like this can reframe what is going on in your life or work, in a way that shows you how much more influence you can have in changing things. The 80/20 rule helps focus your attention on the things that have biggest impact on the results you want in your life. And by focusing on those things, you can more easily examine how you can influence them.

How to do a simple 80/20 analysis.

When you have framed your result and its potential causes in 80/20 questions like those above, you've set the scope for what kind of data to collect. If you're going to know which are the 20% of causes that produce 80% of your result, you'll need to measure both your result, and the degree of impact of each cause.

If you're uncertain exactly what to collect data on, try using a cause-effect diagram to map out the possibilities (fishbone diagram - click here for examples). Then you can measure or estimate the relative impact of each cause on your end result.

A simple bar chart or Pareto chart is a great way to display the 80/20 analysis, once you have the data. For each cause you list, against it you will have a number that represents the size of its impact on your result. It might be dollars or hours or incidents, depending on your result. Chart the data and look for the tallest 20% of bars in the chart that visually account for about 80% of your measure (click here for examples).

A useful note: it won't always be 80/20.

#67 Measurable Goals For Performance Measurement

2007-09-10T21:53:00.001+10:00

In a recent email to me, mezhermnt subscriber Corina from Hong Kong asks:

"This will be the first year my company uses the SMART approach to do performance management. As a manager, I am supposed to set up measurable objectives for my subordinates. It's not a problem for me to set up measurable objectives for my assistant managers as they have deadlines to meet. But when it comes to my secretary and the clerical staff, I am not sure how to set measurable goals for them as their duties are very routine and tedious. Could you give me some examples?"

Are deadlines the only thing worth measuring?

Corina mentions that setting up measurable goals for her assistant managers is relatively easy because they have deadlines to meet. Does this mean that the only results worth measuring for the assistant managers is whether they complete things by their deadlines?

What about completing the right things, instead of wasting time and effort and money on doing things that really don't need to be done at all?

And what about completing things well, instead of rushing to get it done on time but producing an end result that falls below the standards required?

Spend the time to think about results, before thinking about goals and measures.
So this is the first key to setting measurable goals for performance management: first spend some time to define the most important results that the person, in their role, is responsible for achieving. And check that you've got the right balance among those results using a checklist something like this one:

* timeliness (finishing the work on time, or with as short as feasible cycle time or total effort)
* quality (the goodness of the output produced, perhaps in terms of customer expectations or standards)
* quantity (the total amount of work performed, or output produced)
* cost (the total amount spent to perform the tasks)
* efficiency or productivity (the best use of time and resources)

So what are some examples of measurable goals for a secretary or personal assistant?
First we need to talk about the results that are important for a secretary or personal assistant to achieve, rather than get hung up on the duties they perform. For example, rather than focusing on the duties of "send agendas for meetings" and "schedule appointments", one key result might be "their boss is always able to focus on the priorities and not distracted by administrative tasks".

Next, what could be some goals for a secretary to strive for over the coming year? One goal, in line with the above result, could be to "Reduce the proportion of administrative items that go into the boss's in tray or diary."

And lastly, how could you measure this goal? One way could be to add the total hours the boss spends on administrative tasks (or tasks that are not in their list of priorities) and divide it by the total time the boss works, to give a proportion of time spent in administrative tasks. Clearly the goal is to reduce this amount.

See how this thought process gets you monitoring important results, instead of just measuring activity, like how many agendas were produced or how many appointments were scheduled?

So next time you're setting goals and measures for your staff, make sure the conversation starts with a clear statement of the results they are responsible for producing, as opposed to the tasks they perform. Measure the results, not the tasks.

#66 Celebrate Your Performance Successes

2007-09-10T21:52:00.001+10:00

If you've been monitoring your business or team or personal performance for some time, you should celebrate! Even if you still haven't achieved your targets, or even if you still haven't got any significant performance improvements, celebration should be a regular part of managing your business performance.

3 important reasons to celebrate performance successes

Firstly, celebration injects positive feelings into the typically dry experience performance management. Performance measurement most certainly is not just about the numbers. It's about the kind of change we want in the things that matter to us, the things we care about or have strong feelings about. Celebrating performance successes firmly funnels our attention on great things we have achieved, however small. It helps dilute the sting of negative results or negative feedback, which human nature funnels our attention toward with far greater bias than the positive results.

Secondly, celebration builds momentum toward your performance targets. Particularly when you set staged targets along the timeframe to your ultimate targets, celebrating the progress along the way consolidates what's working, and reminds you that you do have influence and control over your performance results.

Thirdly, celebration reminds you what matters most. The spotlight of celebration keeps your attention on what you want more of, doesn't distract you with over emphasis on what you want less of. It's too easy for performance management to be all about fixing problems. Where attention goes, energy flows. Where do you want the energy in your business to flow?

But what kinds of successes can you celebrate?

Decide on some small goals along the way to having that complete performance dashboard that tells you all you need to know about your business results. Celebrating these goals will remind you of the positives of measuring performance, build on your momentum toward your targets, and . Consider celebrating goals like these:

* Celebrate when you have finally chosen a set of performance measures that your team feel excited about, or at the very least feel committed to reporting and using.
* Celebrate when you have brought a measure to life - even just one measure - so that you are now getting the data and using the measure to make your decisions that little bit wiser!
* Celebrate when you find out why performance isn't where you want it to be, and you now have more choices - better choices - for how to influence that performance.
* Celebrate when you see your performance measure respond to the improvements you are making.
* Celebrate when you see your performance measure not responding at all to the improvement you are making (you've learned what doesn't work - very important).
* Celebrate achieving staged targets along the way to your ultimate targets. It helps if you make the earlier targets easier, gradually building the challenge in proportion to the momentum each achievement adds.
* And of course, celebrate when you achieve your ultimate targets!

How do you celebrate performance successes?

There are lots of ways you can celebrate progress along your journey of business performance management. But do choose those that mean something to your staff, that, given their values, they will truly appreciate.

* Consider an awards ceremony (serious or light-hearted) to reward and recognise people that have played special roles in achieving the current performance successes.
* Have a party! Just make sure the reason for the party is clear - perhaps theme it along the lines of the performance win it celebrates (e.g. a birthday party for your new performance measures).
* Reinvest some of the gains from performance improvement, like saved time, saved money or additional revenue, into the workplace for everyone to enjoy - a coffee machine, a massage chair, weekly fresh flower deliveries, bowls of fresh fruit every day. Whatever your staff would most appreciate.
* And many times, a simple heartfelt "thank you" is all that's needed.

What you and your team recently achieved in your management of business performance? Have you celebrated it yet? Drop me an email and let me know how you celebrate it!

#65 Building Your Hierarchy Of Measures

2007-09-10T21:50:00.001+10:00

It doesn’t matter what type of business or organisation yours is, it will still have a few layers or levels of performance results that, through cause-effect or relationship mapping, interdependently sum up what the business must improve as it moves into the future. The most common four layers are depicted in the following model as four concentric circles.

Level 1: success & sustainability measures

The highest level in your performance measurement hierarchy monitors the success & sustainability results for your business. These are the results that are implied by your vision, mission and ultimate outcomes for your stakeholders (i.e. customers, shareholders/owners, partners, communities, employees). These results are the ultimate evidence of the success of the business, and it’s likelihood to sustain that success into the long term future.

Examples of success & sustainability measures might include profit, market value of your business and customer loyalty.

Level 2: strategic measures

The measures that monitor your whole-business strategic objectives or goals are the next level in the measure hierarchy. These measures track the results implied by your business’s current strategic direction. They basically describe what the organisation is going to be like in the next 2 to 5 years.

Examples of strategic measures might include return on investments, market share, revenue and customer churn.

Level 3: tactical (or process output) measures

Tactical objectives or goals are the next level in your performance measure hierarchy, and they are derived from your core, end-to-end processes. It is these processes that have the significant impact on the business’s ability to achieve its success & sustainability results, and its strategic results. Your strategic objectives or goals provide the focus for what results matter most for these end-to-end processes now.

If you're not yet a process-oriented business, then another way to determine your tactical measures is by exploring the impact each department or division in your business has on the strategic results.

Examples of tactical measures might include product development cycle time, new leads, product sales, customer satisfaction (with specific products or services), lost time injuries, on-time delivery to customers.

Level 4: operational measures

The results implied by operational objectives or goals or specific activities are monitored by operational performance measures. They usually track the root causes of tactical performance results. They are the drivers of whole-process results and are where resources are allocated to improve process performance and ultimately improve organisational success and sustainability.

Examples of operational measures might include sales conversion rate, rework, near-miss safety incidents, inventory turn.

Mapping your measure hierarchy

A little (but not much) like the strategy mapping discussed by Balanced Scorecard authors, Kaplan and Norton, mapping relationships among the levels of your performance measure hierarchy can be surprisingly valuable. Each measure has a relationship of some kind with at least one other measure, such as cause-effect, companion, or conflict.

Seeing these relationships mapped visually across the hierarchy of your measures makes it easy to see gaps in your strategy, easy to see how each part of the business contributes to ultimate success and sustainability, easy to pinpoint improvement actions when higher level measures aren't achieving targets.

Why not give it a go?

Take a few of your business performance measures and just doodle a relationship map to get a taste for the impact it can have.

#64 Five Basic Performance Measures

2007-09-10T21:48:00.001+10:00

If you're starting out with measurement but don't have a clearly articulated strategy - or any strategy at all - you're probably feeling stuck about what to measure to manage performance. With no goals, no objectives, and no clear priorities, everything seems important. And it's too overwhelming to measure everything.

So if you are at a loss for what to start measuring, then try these 5 basic performance measures. Spend as little time as possible tailoring them to your business; spend more time pilot testing them. You'll learn tonnes more about the best measures for your business through getting started with something, rather than waiting until you've designed the best way to measure.

measure #1 is customer satisfaction

This first is probably the most important of the 5 basic measures. It's the only measure that will connect you with the relevance of the work you're doing. If customers aren't happy, then everyone is wasting at least a portion of their time. Measure how your customer judges the outcome of your product or service, through surveys or at the end of each transaction with the customer. You can ask them directly, give them a survey form, or send them to a website form.

If you also collect data about what aspects of your product or service are most important to customers, it will give you clues about more specific things that might be important to measure also e.g. easy access to support staff or accuracy of bills.

measure #2 is product/service defects

Defects is a measure of quality, and a translation of what the customer expects your product or service to do, into something you can count to assess how often the product or service actually does what is expected.

Your customer satisfaction measure is a companion to this one. And the extra data collected about what is most important to customers about your product or service will help you define what constitutes a defect (e.g. something breaks, something doesn't operate correctly, a delivery deadline was missed, an invoice has errors).

measure #3 is cycle time

The time it takes to produce or deliver your product or service for your customer is a surprisingly useful thing to measure. It's not just about meeting the time commitments you made to your customer. It's just as importantly about focusing everyone on the things that make the cycle time what it is. And this is usually dead time between hand-offs in the process, waste and rework due to errors or lax standards, and even things that didn't need to be done at all.

An alternative or companion measure to cycle time might be on time delivery, which links it more to the customer's experience. Just remember the value of measuring cycle time for internal benefit too.

measure #4 is productivity

Productivity is a measure of your process efficiency, and is essentially the rate at which you can produce outputs, relative to the input it takes to do so. A great measure to focus you on eliminating waste and rework in delivering your products and services to your customers.

For example, one way to think about productivity is to compare how much you're producing relative the time it takes, such as number of work hours. Another way to think about productivity is about quantity versus cost - how much are you producing, relative to what it costs in resources and labour.

measure #5 is innovation (or improvement) ideas

Even if you're not ready to call it innovation (call it improvement instead), this fifth basic measure is about stimulating one of the behaviours that support a performance culture, namely making active suggestions about how to improve performance.

Particularly when the first 4 basic measures are shared and discussed among the team, actively measuring something as simple as the number of improvement ideas suggested, or the number of targeted improvements tested or implemented, encourages everyone to deepen their understanding about performance, and how they can influence it.

The 5 basic measures are a springboard, not a solution

Remember, don't try to get it perfect before you begin measuring anything. It's not until you start using measures that you discover new questions and clearer information needs. Use these five basic measures as a springboard to get used to measuring and through their use, get closer to understanding what you really do need to measure.

#63 Got A Performance Measure Dictionary?

2007-09-10T21:45:00.000+10:00

Many organisations have hundreds, even thousands, of performance measures. And some of the problems associated with having so many performance measures are:

* lots of measures can be unnecessarily duplicated, and dozens of people are taking dozens of hours each month independently reporting the same things
* measures that should be calculated the same way often aren't, and therefore don't have the power of consistency (aka "apples with apples" comparability)
* measures that aren't brought to life yet have no clear implementation plan or blueprint
* it's difficult to formally flag unneeded or unimportant measures for deletion or modification

One very important strategy to manage your suite of measures is to have a performance measure dictionary, a structured, single system where details about every measure you monitor is kept up to date.

The data *about* your measures

A well structured Performance Measure Dictionary contains fields where you can detail exactly how each measure is to be named, the correct way to calculate it, the appropriate data to use (and where to find it), how to report it, what signals to interpret and who is responsible for it.

This is the data about your measures, often referred to as metadata, and what types of metadata you choose to define your performance measures is important. It can't be vague or incomplete - it has to be sufficient for people to know and understand how to report each measure, so the measure actually tells you what you think it's telling you, and that it is consistent over time.

How to set up your Performance Measure Dictionary

First step in capturing and organising your performance measures is to decide what metadata you're going to use. At the very least, you'll need the measure's name, a brief description, a statement or formula for how it is calculated, where the source data comes from, and who is responsible for the measure.

Second step in setting up your Performance Measure Dictionary is to create a single system to capture these details about all your measures. You could start out using a simple Excel Spreadsheet or Word document, but it will quickly become cumbersome, particularly because it's difficult to sort and report summary measure information like their current 'bring to life' status or all the measures owned by Bob. A Microsoft Access database is a better starting point. And as you get more advanced, some dashboard and scorecard software - like SAS's SPM - enable you to record your measure definitions.

Third step is to stocktake your measures. This is time consuming, but get it done now and you'll save much time down the track that would otherwise be wasted trying to determine exactly how a measure was calculated, or whether the wrong data was used, or why two different reports show the same measure with different trends. You can get someone to go around and collect all the performance reports and details, then enter all the measures into your Performance Measure Dictionary. Or you can send copies of a measure definition form throughout your organisation or business for people to fill in and send back to a data entry person.

Fourth step is to develop some standards or policy around how the Performance Measure Dictionary should be used, and how all new measures in the organisation should be treated. Unless the measure has been documented in the Dictionary, it will be ignored by decision makers. Like all new behaviours, it's not going to be habit straight away, so give it time and encouragement to become the normal way of managing performance measures.

#62 Why Measures Don't Have To Be Precise

2007-09-10T21:43:00.000+10:00

STACEY: Doug Hubbard, of Hubbard Decision Research, is an internationally recognized expert in the field of IT value, with over 18 years experience in IT management consulting including 10 years experience specifically in teaching organizations to use his AIE (Applied Information Economics) method. But today I'm talking to him about the challenge of getting practical measures when people are overly concerned about precision.

Doug, for years now you’ve been coaxing people in the IT field to change their views from “IT is too intangible to measure” to “everything is measurable.” And I can attest that it’s not just in IT that needs to happen. Why has this been a hard transformation to them to make?

DOUG: IT often sees measurement as a choice between perfect 100% certain precision and nothing. Since they see perfect certainty as unachievable, they opt for no measurement at all. They’ve overlooked the usefulness of a third option: the “good enough” measurement. There may be three reasons why IT seeks illusory precision over something less precise but still useful.

STACEY: What have you found are the reasons this third option – and in my view often the only option – has been overlooked?

DOUG: Well, measurement doesn’t mean what they think it means. In my book, I explain that the practical scientific understanding of measurement is quite different from how IT management often uses the term. When I ask CIO’s and other IT managers at my seminars what measurement means, I often get an answer like “Assigning a specific value”. This is wrong in at least two ways. First, in science values aren’t simply “assigned”, they are based on observation. This is rarely the case in IT (when’s the last time you saw a random survey or controlled experiment used to measure something in IT?). Second, a measurement is seen as a reduction in uncertainty, almost never the elimination of uncertainty. In effect, science uses the term measurement to mean “observations that reduce uncertainty about a quantity”.

IT, on the other hand, thinks of measurement more like accountants think of it: absolutely precise, but often arbitrary and not always based on an observation. The book value of an asset, for example, is not based on any observation; just the “accepted procedure” and it may be very different from market value. IT should deal with uncertain reality, not a false sense of precision. When we make a real measurement, it is expressed as a range like a “90% Confidence Interval” that shows our uncertainty about that measurement. Further measurements should make this range narrower, but will rarely shrink it to a point value.

STACEY: So that’s one thing you can do – reframe measurement as a process that helps to reduce uncertainty, not eliminate it. Are there any more hurdles to jump over before they get comfortable to start measuring imperfectly?

DOUG: Yes, and that’s accepting the idea that the presence of noise does not mean a lack of signal. Many managers, IT and otherwise, start anticipating potential errors in any measurement and assume that any existence of any kind of error undermines the value of the measurement. A client of mine was considering a way to measure how much time employees spend in some activity the company wishes to automate. The solution I proposed was to conduct some sort of random survey of the staff over the course of a few weeks where the employee was asked to describe how much time they spent on that activity in that particular day. As soon as the idea was proposed, the mid-level IT managers were almost in a contest with each other to think of all the errors such a survey could have in it. Would the survey be truly random if people who spent more time in the activity were more likely to respond to the survey? Would some people be dishonest in their responses? And so on. Yet when the survey was conducted, they found the potential errors they identified could not possibly account for the findings. The people who had no stake in the outcome gave about the same answers as people who had a stake in the outcome (addressing potential bias in the responses). The response rate was 95% so it is unlikely that the remaining 5% could have changed the findings by much. And simple statistics showed how unlikely it would be that by chance alone, we happened to pick employees that spent more time in this activity.

The presumption the managers originally made is that merely identifying the potential error is sufficient to make the determination about whether any survey would be useless. But if we remember the definition of measurement above, we see that as long as the error is less than the previous state of uncertainty, it counts as a measurement. They were presuming that these errors must be too great to allow for uncertainty reduction but, without conducting the survey and doing the math, there is no basis for such a claim. The fact is that they have to make assumptions about how common these errors are or the effect they would have on the outcomes without having any idea of the relative frequencies of these problems. To put it another way, they have more error in their “error identification” method than the measurement is likely to have.

STACEY: Ha! I like that! It would be a really interesting experiment to find out how big the error in the measurement could get before it drowned out signals in the measure. Wouldn’t that help convince the skeptics?

DOUG: Actually, a formula for the value of information has been around since shortly after WWII. It is used to compute the monetary value of information in a wide variety of industries and government agencies. Ironically, the fact that there even is such a formula is mostly unknown to IT management. In my consulting practice I compute something the decision sciences call “Expected Value of Perfect Information” (EVPI) and “Expected Value of Sample (Partial) Information” (ESI). Of course, the cost of perfect information would almost always be greater than the value of perfect information. In fact, it is almost always the case that the “biggest bang for the buck” is the initial, small amount of uncertainty reduction in a measurement. I tell my clients to start taking a few observations, a random sample, a controlled experiment, etc. and see if the results are surprising in some way. Sometimes the initial observations are surprising enough that it can reduce the initial range for a value significantly and further observations may not be justified.

STACEY: In my previous life as a survey statistician, we’d use pilot samples in the same way, to estimate how much error there would be in the measurement. And this would help us choose a sample size only just large enough to make the measure useful without costing more than the value it would give. Doug, what’s the take-home point you’d like to leave mezhermnt readers with, regarding the imperfect nature of measurement?

DOUG: These three concepts should help IT – and anyone hung up on precision - start to make usefully imperfect measurements. Of course, they require thinking about measurement more like a statistician, scientist, or actuary would and less like an accountant normally would. Adopting these ideas not only encourage people to settle for “good enough” measurements, but they will probably cause them to focus on very different measurements in the first place. In the last 12 years, I’ve completed 55 major IT valuation studies and I’ve seen that most things IT measures have little or no information value and that the measurements with the highest information value tend to be those things IT almost never measures. This means that the measurements IT seeks, even if they achieve infinite precision, probably have no bearing on pragmatic business decisions. You should always choose imperfect but relevant measurements over arbitrarily precise and irrelevant measurements.

STACEY: Measures just need to have enough precision to give you the signals you need to make meaningful improvements to the business. Makes sense. Thanks Doug for sharing your ideas.

#61 A Performance Measure Self-Assessment Checklist

2007-09-10T21:40:00.000+10:00

Where do you start improving your organisation's performance measures? What kinds of things could you improve about how your organisation measures its performance? In what ways are you already good at performance measurement?

These questions may not be exactly keeping you awake at night, but certainly by answering them, you could both save a lot of time and effort in the process of getting better measures to manage your organisation's performance. That's because with the answers to these questions you can decide where to focus your efforts to make performance measurement work better.

So to give you a start in answering these questions, here is a rough-and-ready checklist of the key criteria to assess where your current performance measurement system is working well, well enough, or needs more work. Tick it if you do it well. Tally up your ticks for each section and start exploring how you might improve those sections where your tick count is lowest.

1) Selecting performance measures that are meaningful

[ ] The organisation's strategy is the guideline for what should be measured.

[ ] Each performance measure provides objective evidence of the degree to which a specific result is occurring over time.

[ ] Ownership of the measures happens.

[ ] No one is responsible for more than 7 (or so) performance measures.

[ ] All performance measures are defined using a consistent definition framework that specifies exactly how each measure will be constructed, reported and used.

[ ] Performance measures are driving the right behaviour (which has been defined).

[ ] The linkages or relationships between all performance measures are understood.

2) Collecting performance measure data that is reliable and relevant

[ ] Only relevant and useful data is collected.

[ ] There is a policy that makes explicit the degree of integrity required of data for each measure.

[ ] The data collection tools that used throughout the business are designed to collect data with the degree of trustworthiness required.

[ ] Each data item collected is defined consistently as part of a 'data dictionary' for the organisation.

[ ] Data collection processes dovetail into work processes seamlessly with minimum, if any, disruption to operational effectiveness or efficiency.

3) Storing and managing performance measure data for easy and quick access

[ ] Data capture is simple, effective and maintains data integrity.

[ ] Data can be easily accessed by those who need it, when it is needed.

[ ] Historical data is readily available when required (historical data means data that is more than a couple of years old).

4) Analysing performance measure data to reveal the data's story

[ ] All data analyses performed, whether internal or external to our organisation, are focused on answering pre-defined driving questions.

[ ] Statistical techniques are used validly and appropriately.

[ ] Variation in the performance of business processes is measured (not % differences, true statistical variation).

5) Presenting performance measures to make interpretation easy and valid

[ ] All performance reports produced have a clearly defined and understood purpose and a clear target audience (or audiences).

[ ] The physical layout of reports is simple to follow and makes finding information easy and quick.

[ ] Graphs are the preferred method of presenting performance measures (and the correct graph type is used to answer the driving question).

6) Interpreting performance measures to draw the right conclusions

[ ] The owners of performance measures are the people that interpret those performance measures and communicate their conclusions to others.

[ ] Statistical methods (such as statistical process control charts) are used to flag signals in the data (e.g. levels of stability and change in process performance).

[ ] There are consistent and well defined guidelines for interpretation of performance measures (such as a definition of the evidence of a true trend or change in performance levels).

[ ] People involved in using data and information have the appropriate level of skill in interpreting it effectively, efficiently and validly.

7) Applying performance measures to improve performance

[ ] The results of performance improvement decisions are tracked using the same measures of performance that these decisions aimed to improve.

[ ] The results of interpretation of performance measures are an input into our planning review process, taking a visible role in the formulation and evaluation of our business goals & targets.

[ ] The root causes of performance results are identified through further analysis of lead indicators and/or other data.

[ ] Performance improvements are decided upon through application of systemic thinking and are prioritised before they are taken on and implemented.

[ ] Intuition, emotion and gut feel are used to guide further collection and analysis of objective data (both quantitative and qualitative) rather than to drive decision making alone.

#60 When NOT To Measure Something

2007-09-10T21:37:00.000+10:00

We're always talking about why it is so critical to have performance measures, what is most important to measure, how to design meaningful measures, how to measure the intangible things. But there is a lot of value to knowing when measuring something just isn't the best idea.

Don't measure it if you have no intention of managing it.

Is it really responsible to be measuring something you have no intention of doing anything about? If it's because you were told to measure it, then you have at least two choices.

You can talk to whoever has demanded to measure it, and in the spirit of authentic curiosity, explore their and your points of view and negotiate a more meaningful measure, or drop it entirely if your existing measures sufficiently cover the most important results you are responsible for managing.

Or you can talk to whoever has demanded to measure it, also in the spirit of authentic curiosity, ask for their help to determine what kind of response you should be taking to the measure, it's priority over your other measures, and the guidelines for how much of your resources to throw at it when it goes south.

Don't measure it if the cost of measuring outweighs the value of knowing.

Many data collection systems, like surveys, cost lots of money. Especially when you have to consider factors like measuring over wide geographic areas or measuring to high levels of accuracy or measuring very rare phenomena.

Get in the habit of checking if the likely gain of using the measure (like improvement in sales or reduction of waste) has a smaller net impact on the bottom line than does the cost of measuring it. If you can't show a decent positive return on investment for measuring something, don't bother. How else could you have used the time and money to impact your business' or organisation's success?

Don't measure it for old times' sake.

Do you take on more new measures that you let go? Letting go of measures seems to be such a psychological battle - we hang on to them because they're already being tracked and we might need them again someday. Fine. Keep collecting the data (if it doesn't cost too much), but stop reporting the measure!

It's time to test if there is something more important to put that time and resource into. Perhaps to focus on other higher priority measures, or to spend some time designing more meaningful measures for your current strategy. Unimportant measures will slow you down and waste your time and energy.

Don't measure it if it will be a big stick.

Measures have the worst reputation of being used as big sticks to beat people's performance into shape (or to at least attempt this). Measures can be very indispensable in managing people's performance, but the big stick approach means using measures to point blame or CYA.

You'll see a ripple of fear and defensiveness in every direction around the one who yields such a measure, and it will build into a wave of destruction. If you don't have a performance improvement culture, if there is a real risk that the measure will be used as a big stick, then avoid measuring it. Put the time and effort into some open and candid dialogue to explore the results that matter and how to improve them.

Don't measure it if you're already measuring too much.

Drowning in measures? Do you have more measures than you have time and resources to review and improve? Then don't take on any more! Too many measures is often worse than not enough measures. Overwhelm is so much more debilitating than scarcity when it comes to measures. At least with scarcity you have the time and energy to move in the direction of a concise set of meaningful measures. But with overwhelm, you usually feel stuck and unable to move in any direction at all.

Instead of working on more measures, take stock of the measures you have already, cull those that really aren't that important, and put your focus on just 3 or 4 measures at a time, improving performance in all results systematically.

#59 You Can't Measure An Ill-Defined Strategy

2007-09-10T21:35:00.000+10:00

If you're trying to measure things without first having your strategy - your goals, objectives or priorities - firmly and crisply articulated, then you're probably frustrated by where to start, overwhelmed with too many possible measures, or disappointed because you haven't found the measures that really matter.
That's what many of my consulting clients are experiencing when we first start working together: they have no strategy, or they have a strategy that is too vague, or they have a strategy that hasn't been properly cascaded. The first obstacle to overcoming their measurement problems is to make their strategy measurable. This is one of the powers that great measurement has: to make your goals so tangible, so vivid, that you can't not achieve them!

Here is a checklist of symptoms for a quick and dirty diagnosis as to whether you need a little more work on your strategy, before it will be worthwhile to start on your measures.

Symptom 1: no written record of your goals, objectives or priorities

Do you have a document - like a business plan or plan on a page - that includes a summary of the 3 to 10 priorities for your business or organisation right now? If not, then you probably haven't designed a strategy and are instead just managing day to day. Or maybe you do have a loose strategy but it's either in the CEO's head or keeps changing from week to week. Or perhaps your business plan reads more like an essay that fails to make the highest 3 to 10 priorities clear.

In any case, without a formal announcement of and spotlight on the current goals for the business, no-one has a way of prioritising what to measure, because they have no direction as to which results matter most. They're paralysed by the overwhelm of all the possible things they could measure. Not everything matters enough to be measured. A single, agreed strategic direction narrows down the choices (and streamlines the measurement effort).

So, write down your strategy, let everyone know about it, and keep reminding everyone to pursue it, above all else.

Symptom 2: the strategy is a collection of motherhood statements

If you do have a written strategy, is it expressed clearly, succinctly, tangibly, in language everyone understands and uses comfortably? Or is it written in management-speak, jargon, inert language that could be interpreted 7 different ways by 3 different people? Are you using words like efficient, productive, effective, quality, sustainable and best-practice?

If you could copy and paste your goals into another organisation's strategic plan and no-one would blink an eyelid, then you don't really have goals. Without specific, tangibly expressed goals that make it clear what efficient means, what quality looks like and how you'd recognise sustainable, your measures will fail to provide the right evidence of the right results. Remember, what you measure is what you get, irrespective of what your goals say.

So, make sure your strategy is worded clearly and very specifically so that people can easily understand and relate to the words, and all have a consistent understanding of what it looks, sounds and feels like when it becomes a reality.

Symptom 3: everyone's goals are a carbon copy of the organisation's goals

Does your business cascade its strategy in the way that if cost reduction is a goal for the organisation, then cost reduction is a goal for each and every department and team, function and process? Is every team's plan a "Mini Me"(1) version of the organisation's plan? If so, people throughout the business are likely frustrated, being asked to measure things that have no meaning to them, that don't help them manage their real and most significant contribution to the whole business' performance.

Why distract the Design Department with measures about lost time injuries just because it's a strategic priority? Manufacturing probably have the primary influence on that result. The cause-effect model of cascading will lead to more meaningful measures than the Mini Me model. The Design team's contribution to safety is not in their own injury rate (repetitive strain injury or paper cuts) but rather through the features and manufacturing steps they design into the products that Manufacturing will construct.

So, cascade your strategy not by giving every department a copy of it to scale down, but by asking the question "how does this department most impact our strategic goals?"

The process of measurement can make your strategy better!

No strategy is ever perfect, if only for the fact that the business environment changes continuously and often unpredictably. Don't strive to get your strategy perfect before you start measuring, or you'll never end up executing the strategy successfully. The starting point to measuring is a strategy that doesn't show the above symptoms in the extreme. And when you get started with designing measures, you'll find it seduces you to reflect on and refine your strategy even further!

(1) Mini Me is the tiny clone of Dr Evil, both characters in the Austin Powers movies.

#58 The Engine Under The Hood Of Performance Measurement

2007-09-10T21:31:00.000+10:00

Donald Rumsfeld said, “…we don’t know what we don’t know.”

In a climate of increased competition and diminishing resources, organisations are seeking ways of creating a distinctive ‘edge’ over their competitors and identifying operational efficiencies. Organisations are adopting analytics to meet these needs.

So what does ‘analytics’ actually mean?

Despite centuries of use we all have our own interpretation or definition. Even the Roman Empire used analytics to understand their regional dominance. For example early analytics underpinned the first noted census in 443 BC which they then used as a decision making tool to understand the services required for the efficient running of their economy.

Today mathematics and enumeration are commonly used forms of analytics. They measure with accuracy activities within the economy and drive effective resource planning. Mathematics has also led to the formation of statistical analysis. This involves the science of approximation and explanation, not only how things fit together, but what will happen in the future.

Analytics is a practice of sifting through and analysing whatever data you have to reveal trends, correlations, and other patterns in the data that help you answer questions like:

* Why are we getting these performance results (as indicated by our performance measures)?
* What changes could we make to get better performance results?
* What performance results are we likely to get in the future?

What does 'analytics' do?

As businesses and governments have more data available and more complex issues to solve, the convergence of traditional enumeration and statistical analysis has occurred. Analytics underpins both qualitative and quantitative output, so organisations can describe what has happened and what will happen consistently, reliably and with validity.

These analytics techniques are applied in today’s world to determine such things as the identity of fraudulent activities or whether a specific demographic will take up a government service. Commercial organisations use analytics to market their industry presence, or to competitively determine the drivers of the market and how to acquire that market share. It helps them understand their distinction, how to drive that distinction and to maintain the edge over their competition.

Analytics is the union of rigourous statistical analysis techniques with technology that makes data more easily accessible and automates the analysis. So you don't have to have statistical or IT expertise to do it.

Why go down the 'analytics' path?

Analytics allows managers and decision makers to:

* optimise key business processes by pinpointing the factors that most influence overall process results;
* identify the best customers by understanding customer preferences and buying behaviours;
* choose the right policy by exploring the impacts of different actions on possible future outcomes;
* minimise inventory and maximise the availability in supply chains; and
* allocate costs accurately and understand how financial performance is driven.

Analytics is the engine behind real performance measurement. It gives you the information that helps you isolate the causes of poor performance, and the points of leverage to accelerate performance improvement.

#57 The Ingredients Of Accountability

2007-09-10T21:28:00.000+10:00

In a recent article in the Performance Improvement journal, William Liccione presents a model that estimates a person's motivation to accomplish their assigned goals based on the relationship between factors like goal commitment, instrumentality to attain the goal, expectancy of receiving compensation, the value of the compensation and the fairness relative to compensation given to others.

This article inspired me to think about the relationship of motivation to accountability, a word that is bandied around a lot in workplaces today, without much practical discussion about exactly what it means nor exactly how to get it. In essence, it's the pointy end of responsibility - that when you are responsible for getting something done, you will need to show account (show a count?) of having gotten it done, or bear the consequences of not.

So if we want more accountability, what are the things we need to pay attention to?

Ingredient #1: Clear, clear, clear goals

What exactly is the person accountable for? How similar is the vision in your head to the vision in their head about what it looks like when their goals are attained? Get rid of ambiguity, inert language, all-encompassing broadness and keep the number of goals manageable (e.g. 3 to 7 priority goals per person).

Ingredient #2: Personal interest in pursuing those goals

Does the person care about the goals? What's in it for them? You know how hard it is to apply yourself to a task you just don't want to do - like vacuuming under the rug, cleaning out leaves from the roof gutters, giving high pressure sales presentations to disinterested audiences. Craft goals with the person, not for them. And help them explore the most ways they can contribute the best of themselves to their work.

Ingredient #3: Belief that the goals are attainable

Has the person got (or will they get) all the resources and time and knowledge and skill to pursue and attain their goals? Do they believe they will have these things? How much fear do they have about not attaining the goals? When you are choosing and designing goals, it helps to draft the action plan at the same time to both check that the goals are attainable and to pace the person through their objections and concerns about their ability to achieve them.

Ingredient #4: Belief that attaining the goal is worth their effort

Do they want to attain the goal enough that they are willing to pay the price to achieve it? Is the size of the reward for them large enough to compensate the extra mile they'll need to go to attain the goal? Discuss and agree the rewards up front, and contrast the having of those rewards against the time, effort, frustration, fear and whatever other price the person must pay to take on the goals.

Ingredient #5: Buy-in to objective measures that will track the goal

Are they confident that the measures chosen to monitor progress toward the goal - and evidence its attainment - are fair and useful measures? Do they believe that the measures are a true reflection of their progress and that, through their constructive goal-pursuing actions, they can affect the trends of these measures? Design the measures collaboratively at same time you design the goals.

I've written here about accountability of individual people, but I wonder how different the ingredients list would be for organisational accountability?

#56 Criteria Checklist For Excellent Measures

2007-09-10T21:26:00.000+10:00

Not everyone wants to (or indeed needs to) throw out their existing measures just because they aren't perfect. To make sure you don't throw the baby out with the bathwater, evaluate your measures using the following simple checklist and you might be surprised at the ideas you get to improve your not-quite-right measures, and cast a keener spotlight on those measures that really should be thrown away.

Oh, and a word of warning: No list of criteria is ever complete or ever totally correct. So as you read these and apply them, tune into the purpose of the criteria if the prescription just doesn't fit.

1. Does it have clear link to strategy?

For your measure to be excellent, it must have a clear line of sight to your business direction - to your strategic goals or priorities. If you are measuring something and improving it won't make any significant contribution to achieving your strategy or goals, do you really need to measure it? (Idea: design your measures at the same time you formulate your goals.)

2. Is it owned by someone?

Unless your measure is officially owned by someone - not a department, not a team, but a person - it's super likely that it isn't being used to make performance improve. It may not even be properly reported. Measures need an owner to make sure it is reported and used for the benefit of the business. (Idea: find a person who has enough authority to respond to the measure.)

3. Can it be brought to life?

Vague ideas, surveys and kooky acronyms are not measures. A measure needs to be spelled out in enough detail that you can know exactly how to calculate it, how often and from which data. (Idea: define the details of bringing your measure to life to test its viability.)

4. Are you able to track it regularly over time?

Before and after measures (which is what annual measures usually end up being) don't give enough feedback to manage improvement efforts. Your measure must be tracked regularly enough (such as monthly or weekly) to give you clues about whether your improvement efforts are working, before it's too late. (Idea: measure more frequently or find a lead indicator that you can measure more frequently.)

5. Does it give you more value than it costs?

Measurement doesn't happen for free. But you need to be confident that the costs associated with its design, data collection and reporting are less than the benefits it brings to your business, which is usually through decision making and the resulting improvements you get for your bottom line or your stakeholder value. (Idea: remove waste and duplication from data collection and reporting, e.g. use sampling instead of measuring it all.)

6. Do users understand it?

If your measure is a convoluted index of other measures, or it's calculation is difficult to explain in everyday language, it can make using it too daunting a job. (Idea: borrow from Ockam's Razor and find the simplest measure that can convey the needed information.)

7. Does it inspire the right behaviour?

If your measure is not encouraging people to choose performance-improving behaviour over sweep-it-under-the-rug behaviour, it's a candidate for throwing out. The measures that are the most potent in improving business performance are those that make it obvious - even unconscious - the right actions to take to get better performance results. (Idea: involve staff in designing the measures so they have more understanding and buy-in.)

These seven criteria are a good basis to judge your measures. You might like to make up a grid with your measures listed down the rows and these criteria listed across the columns. You can then do a quick evaluation of your existing measures, and see at a glance which to keep, which need some work, and which should probably been thrown away!

#55 Three Risks Of Off The Shelf Measures

2007-09-10T21:23:00.000+10:00

Do you go looking for measures that your organisation or team can adopt? Do you trawl through the annual reports of organisations in your industry? Do you survey your colleagues at industry conferences, do you search for KPIs in your industry on the internet? Do you ask the so-called experts what measures you should be using? Are these your only strategies for finding the most meaningful measures for your goals? Yes? Oops! You're probably about to take a tumble into one of the following traps!

The "how-do-I-actually-measure-this" trap.

You've got your list of measures that you hunted and gathered from various sources, and now you want to start implementing them. Only the information you have about them is too thin and you can't figure out how it should be calculated and what kind of data to use and how frequently to measure it! In fact, the more you think about those measures, the more confused you are about what they actually mean.

The "why-aren't-my-staff-buying-in-to-these-measures" trap.

It's true that measurement of performance is not everyone's favourite task to turn to when some free time comes up. No. Most organisations that are starting out with measurement really don't have a healthy measurement culture, where people are all about using measures as feedback for ongoing business improvement. Handing them some measures-we-prepared-earlier is like handing them the stick you're about to bop them on the head with. And you want them to happily and cheerfully own these measures!

The "hang-on-these-measures-don't-match-our-goals" trap.

Adopt someone else's measures and you're adopting their strategy. Did you know that some organisations treat their performance measures as part of their intellectual property and competitive advantage, and refuse to let anyone outside the organisation know what they are? There's a good reason for this. Measures (when they are well designed) tell the complete story of an organisation's direction and journey in that direction. They give knowledge to the leaders and decision makers that is priceless, and this is possible because those measures were designed very specifically to match the organisation's unique goals and strategic direction.

Know and understand your goals, and design the right measures.
Searching around for ideas about what measures to use is not the problem. In fact, it's a good idea to help you build your list of potential measures. But not until you first understand the results you are most needing to measure, so you have a sensible filter to discard potential measures that really aren't right for you and to keep the measures that make the most sense to your strategy, and to your team.

#54 What 'Balanced' Really Means For Measures

2007-09-10T21:17:00.000+10:00

When most of us hear the term 'balanced measures' we see the Balanced Scorecard flash before our eyes. The success of this decade-and-a-half old framework has been both a windfall and a worry. Yes, our mid-1990's fever for good measures that actually measured what mattered was somewhat tempered by Kaplan and Norton's medicine. But it's unprecedented success brought on a new fever: the expectation that a balanced suite of measures is a simple plug-and-play bolt-on to your business' performance scorecard. No thinking required, just grab some KPIs and stick 'em in the right perspective (financial, customers, internal processes or learning and growth).

Many of the organisations I work with share with me this rationale for seeking help from a performance measure specialist: the Balanced Scorecard hasn't made measurement any easier for us. They aren't using the four original Balanced Scorecard perspectives, and if they are, they aren't really comfortable with the fit to their unique organisation. The natural remedy is to turn our brains back on, and think more deeply about what balance really means to our unique organisation, BEFORE we worry about balanced measures. Here are some prompts that will help you improve the balance in your organisation's overall performance.

Stakeholder Groups

How does each stakeholder group with some investment in your organisation define success for your organisation? What's important to your shareholders, regulators, owners, customers, partners, employees, and community, important enough that doing it better would mean they feel your organisation is more successful, and they ramp up their support for it's future success?

Business Processes

How you do business is more important that what business you do. Your end-to-end processes impact very significantly on the results your organisation gets through their very design. What are the impacts of service delivery, research and development, marketing and sales, procurement, and other essential business processes on your overall organisational success? Ignoring any of the important processes is balance-heresy, and makes it too easy for parts to succeed at the expense of the whole organisation.

Strategic, Tactical and Operational

You don't have one scorecard, a single basket of measures. If any one person in the organisation wants to know all the measures throughout the organisation, they are micro-managing and thus mis-managing. Which results are strategic, which are tactical, which are operational? Better balance comes hence when there is a logical cause-effect relationship between these layers of results (and thus their measures). This balance does wonders for throwing resources at the right initiatives for success.

Lead Versus Lag

If you're like me, you are completely over hearing the "driving a car by looking in the rear vision mirror" metaphor for managing organisational performance with lag measures. But it's still a good point. Which results are the end products or ultimate outcomes for your organisation? Measures of these are your lag measures, and it's still important to track them. Which results offer clues about what those end products or ultimate outcomes might look like in the future, based on how things are now? Measures of these are your lead measures, and you need to track these too!

Short Term, Medium Term, Long Term

The Japanese apparently plan with future generations in mind, decades into the future. The Western world considers long term planning to be 5 or maybe 10 years from now, but really only manages within a 12 month timeframe. Better balance will come from sorting which results are day to day, which are month to month, which are year to year, and which are truly longer term. This balance helps put the right measures into the right decision processes.

Test your strategy for balance before you design measures.

One of the most frequent reasons (if not the supreme reason) for poorly balanced measures is starting with a poorly designed strategy. If your strategic plan reads like a case study of motherhood statements and management-speak, you're in trouble. Tease out the results you understand this strategy to be implying (dig deeper behind the jargon) and examine those results for balance using the above prompts to get you started.

#53 Measure Your New Year's Resolutions!

2007-09-10T12:30:00.000+10:00

I love this time of year, when we symbolise through our New Year's resolutions the fact that we get another chance to renew ourselves in some way. It might be our health, or our career, or our devotion to something bigger than ourselves - we lift out one or more specific goals and pursue them with all our heart. As you now solemnly resolve to yourself to achieve these personally important things this year, why not take it seriously enough to actually measure? You know you are far more likely to achieve something if you get regular feedback about it, so here are some simple steps to help you keep your eye on this year's most important goals for you:

STEP 1: Write down each of your resolutions as a goal.

As the clock struck midnight on December 31, you may have closed your eyes and muttered under your breath "This year I resolve to..." Taking your New Year's resolution seriously means going a few steps further, and the first is to write down each resolution as a goal statement - preferably on a colourful piece of card or paper in big clear words that you can easily read.

STEP 2: Make your goal sensory rich.

"This year I promise to get healthy". Not good enough, I'm afraid. If you seriously want to achieve your goal, you have to make it so intensely real in your mind that a part of you almost believes you've already achieved it. What do you mean by healthy? Do you mean lose enough kilograms to slip easily into a size 10 pair of Levis? Or do you mean be able to run 5 kilometres in less than 25 minutes without your heart rate going above 150 beats per minute and feeling elated and comfortable the whole distance? Make your goal real in your mind so you can feel, touch, hear, see, smell and taste it - direct the movie of arriving at your goal for your mind to experience in full.

STEP 3: Choose the measures that will keep you on the path.

Select between 1 and 3 measures for your goal. Enough to keep you focused but not too many that measuring becomes a chore. You might like to have 1 or 2 quantitative measures (such as weight or % body fat or waistline measurement) and balance this with 1 or 2 qualitative measures (such as your self-confidence or how loose your clothes feel or how well you feel you're sticking to the path toward your goal). You may find it useful to measure not just your end goal (e.g. your weight), but the little changes you need to work on in order to achieve it (e.g. calories you eat and exercise frequency).

STEP 4: Set up your measurement regime.

Discipline is called for here. Quickly establish the habit of collecting the data that tracks your progress toward your goal, calculating your measures' values and charting them so you can see the entire journey toward your goal. How often should you collect the data (e.g. daily or weekly)? When exactly will you collect the data (e.g. every Monday and Thursday mornings)? What method will you use to collect the data (e.g. scales or a tape measure)? Where will you collect the data (e.g. in your bathroom or at the gym)? How will you remind yourself to collect the data (e.g. a sticky note on the bathroom mirror or a recurring reminder in your PDA)?

STEP 5: Start right now, this instant. Don't dilly-dally!

Grab some coloured paper or card. Write down your New Year's resolution goal in the centre. Write down sensory rich words around the goal (better still, draw pictures or paste photos) and create the movie of achieving it in your head right now (close your eyes, take your time). Select 1, 2 or 3 measures to track your goal. Create an Excel spreadsheet or wall chart or journal to start collecting the data and charting it. Go and collect your first set of data for your measures now.

STEP 6: Review your goal every day, at least until you achieve it.

When you wake up in the morning, one of the first things you should see is your goal. When you go to bed at night, one of the last things you should see is your goal. Look at your goal, read it, and play that movie over in your mind (you'll find the movie gets more detailed, clearer and more intense the more you play it). In addition to the actions you choose to take to achieve your goal, reviewing it daily makes you believe it, makes you unconsciously work toward it. It's very powerful, this habit!

Is reaching your goal worth achieving it?

Setting a goal takes very little in comparison to the effort you'll need to put in to reach it. This applies to personal goals like New Year's resolutions and it applies to business or organisational goals too. If my Grandad was still alive, I'm sure he'd say that one of the main reasons we don't achieve our goals is lack of discipline. Becoming more disciplined is one of my important goals for 2007, because there is much I want to achieve and time waits for no-one!

#52 Measurement Tips From Table Tennis

2007-09-10T12:26:00.000+10:00

Recently I have been spending some time with my husband playing table tennis in our garage after work. I'm new to table tennis, so it's a steep learning curve. And even though a lot of my attention was on hitting the ball back and landing it on the table instead of skewing it off toward the tool rack or up into the fluorescent lights, I couldn't help reflecting on how similar the experience was to any kind of performance improvement in business. In fact, here are the six tips that learning table tennis (or trying anything new) can teach us about improving the performance of anything:

TIP #1: Be very clear what result you want.

When you start out with something new, aiming to be the best at it straight away is not what I call realistic goal. So rather than set my immediate sights on thrashing my husband by the end of our first table tennis match, my focus was more modestly on just hitting the ball back and having it land where I intended it to go. With such a clear goal in front of you, it's so much easier to reach it, one logical step at a time. (Your eyes know exactly what to look at and what to look for.)

TIP #2: If you're not good at it yet, expect high variability in your performance.

With little skill or knowledge about table tennis, I could only expect to have little control over where I put the ball, and with little control I could only expect to have little predictability in my results - the distance between where I intended the ball to land and where the ball actually landed fluctuated randomly and wildly. Understanding (and measuring) your variability is your baseline - understand this natural variability before you attempt to improve anything.

TIP #3: To really improve, change only one thing at a time.

As simple as table tennis is, there were many things that I could have changed to try for a better result. How I held the paddle, how I positioned my feet, how I moved my wrist, how hard I hit the ball, how accurately I read the spin that my clever husband put on the ball (in his wicked attempts to make my returns even more unpredictable). I found I improved best (very satisfying) when I thought about just one thing to do better, like holding the paddle consistently and at the right angle. Improvement happens so much faster when you bed down one improvement at a time. Trying to figure out the complex interactions among several changes at once is confusing, exhausting and takes many times longer to get results.

TIP #4: Performance will probably grow worse straight after you start improving something.

The moment I became more conscious of how I was holding the table tennis paddle, things got worse. The ball seemed to grow a mind of its own for the next 10 to 15 hits. Yes it did more often land where I intended it to, but it would also unpredictably ping off at the most obscure angles. However, it didn't take long to get a feel for the new grip on the paddle and - lo and behold - the ball began doing mostly what I wanted it to. I had more control! Any kind of performance improvement can have a 'bedding in' period, but then things can grow better almost in an instant.

TIP #5: Keep focused - if you take your mind off it, you lose control again.

Pumped by my quick success at bossing the ping pong ball around, I thought I could let go and relax into the game a little. Big mistake. A few fast and furious returns from my loving husband's paddle made me instantly aware that holding a table tennis paddle wasn't yet second nature. The ping pong ball 'pertwanged' out my control and was at the sole mercy of my husband. So remember, if you take your mind off an improved change before it becomes second nature, you risk losing control again and the variability widens once more.

TIP #6: Get feedback regularly, and don't misinterpret it.

"You're hitting down again!" My husband was sounding like a broken record (now there's a metaphor that's losing relevance!). So again I lifted up my swing to correct for the mistake. "You're hitting down again!" (He's a very patient man.) What?! Then I asked him what he meant and it turned out that his idea of hitting down meant my paddle was at the wrong angle, but I interpreted it to mean my swing was at the wrong angle. Assumptions! So make sure you track the changes made by your improvement frequently enough that you can correct things if they go askew - but make sure you know what the feedback is telling you.

#51 How To Find Time To Measure Performance

2007-09-10T12:22:00.000+10:00

"We're just so busy and have too much on our plates, but we know we have to find time to measure performance - it's too important not to."

Sound familiar? I've been hearing complaints like this more and more frequently over the last year or two. And you don't have to look too far to see the nasty consequences of trying to do too many things: half-baked strategic direction, most projects under-resourced, staff accumulating too much annual leave, flurries of activities and no-one knows which are working and which are a waste.

Performance measures are even more important when things are busy and chaotic. Well designed measures make priorities clear, give specific and definite direction to activity, and provide feedback so you can avoid wasting time.

The first tip for finding time to measure performance is about reducing the rest of your workload

What is one thing you are doing now, that is less important than getting more control over your workload and your performance?

* Is it a project that you've lost passion for, that just isn't getting the results you need or that you feel compelled to finish just because you started it?

* Are you still doing administrative work that you can easily delegate to an assistant, like typing and formatting documents, basic internet research, sorting and sending emails, organising meetings and workshops, conducting simple surveys or consultations?

* How many hours a day do you give to distractions like chatting over the photocopier, answering the phone any time it rings, checking your email every 15 minutes, starting new tasks that you didn't even plan to do?

* Are you driven by your priorities, or the priorities of other people? Which tasks are you doing that really are not your responsibility, that are dragging you off the path to your most important goals?

Once you identify just one thing less important than having meaningful performance measures, stop doing it (yes, that can be hard and will take a real serve of discipline).

Then, allocate the freed up time to designing and bringing to life performance measures for the results that really do matter most. It will help if you treat this as a project of its own, and plan it properly and resource it sensibly and schedule it in your diary with at least as much importance as anything other appointment you have made.

The second strategy for finding time to measure performance is about reducing the measurement workload itself

How can you save time in designing and bringing to life your measures right now, so you get some runs on the board, so you can start making it a natural part of your work?

* Of all the results you want to measure, which are the 3 most important results to measure now? Just measure these. Leave the rest on the back-burner until you have the first 3 up and running smoothly.

* Can you use a smaller team to draft the measures, and then consult more widely afterwards? Who are the 2 or 3 other people that can help you most in measuring the 3 most important results to you? How can you make it easy for them to help you now?

* Does the data need to be 100% accurate, or is a reliable indication of trends really all you need? What data can you already access or very easily collect to provide your most important measures?

* Where are you collecting data from entire populations when random samples could work well enough?

* Do you really need to put all that effort into an electronic reporting dashboard when some Excel charts in a Word document report will do the job for now?

Focus more on building your momentum for measuring performance, and worry about perfection on your second or third iteration through the performance measurement process. Starting small and deliberately will lay a solid foundation to build more and better measures upon, as you get faster and more skilful at doing performance measurement.

#50 Six Performance Measure Facilitator Attributes

2007-07-11T08:23:00.002+10:00

Over the last 5 or so years, there seems to be an ever-increasing number of organisations that are creating a new role in the corporate office: the Performance Measurement Officer. Actually, the title of this role varies from organisation to organisation, and where exactly in the organisation structure that role is placed also varies.

Titles for performance measure facilitator positions have included Performance Measurement Officer, Performance Measurement Director, Manager Performance Measurement, Corporate Planning and Performance Reporting Officer, Corporate Performance Management Coordinator and Manager Planning and Performance.

Most often the person in this role of performance measure facilitator will be associated with the corporate planning team, but they are also associated sometimes with the information services team or even somewhere in the human resources department.

The one thing that is consistent, however, is the thing this person is responsible for: to facilitate the design, reporting and use of performance information in decision making about organisational results and improvement, usually across the entire organisation. This calls for some very specific attributes, and the following six should be considered the bare minimum.

attribute #1: intimate understanding of the organisational planning process

Without a very detailed understanding of how the organisation does its strategic planning, and cascades this strategic direction down into tactical and operational plans, the performance measure facilitator will struggle to assist managers and teams to focus on measuring what matters most. Knowing how to integrate performance measurement with the planning process ensures everyone is measuring the results that will most likely lead to the organisation fulfilling its strategic direction and achieving its vision.

attribute #2: a working knowledge of several performance measurement frameworks

If a performance measure facilitator can only claim knowledge of the Balanced Scorecard, then the organisation faces the risk of having its strategy too quickly packaged into a model that may not be the most appropriate. They need know how to apply a range of frameworks (e.g. the Performance Prism, Triple or Quadruple Bottom Line, Six Sigma Business Scorecard, EFQM or ABEF or Baldrige models) to assist managers and teams to decide what types of things to design measures for.

attribute #3: experience with at least one performance measure implementation process

There are far more performance measurement frameworks out there than performance measure implementation methodologies (e.g. six sigma and PuMP)! A performance measure facilitator that is worth their salt will have experienced at least one step-by-step process for designing and implementing measures, and will be on the lookout continuously to find emerging methodologies or to continue to develop and fine tune one that works for the organisation.

attribute #4: basic quantitative skills for creating and reporting performance measures

While they certainly don't have to be a statistician, the performance measure facilitator does need to be comfortable and capable to design simple data collection processes, manipulate and prepare data for analysis, perform simple analysis calculations (such as percentages, averages, ratios, standard deviations), choose and format charts that clearly announce the true signals in the data, and validly interpret those signals.

attribute #5: change management skills that are second-nature

Performance measurement is not usually fun and enticing, and it is usually threatening and hard work. The successful performance measure facilitator will know this, and will be so well equipped with at least some basic change management techniques that they find it almost second nature to establish the support of leaders, encourage ownership and buy-in, make the reason for change clear and communicate very well to all kinds of people.

attribute #6: intermediate project management skills

Often the performance measure facilitator is running several areas of the organisation through the performance measurement process at any one time. And especially when they don't have a large enough team to meet the demand for performance measures throughout their organisation, very strong project management skills can keep them focused on the priorities and keep everything else as organised as possible.

#49 The First Performance Conversation

2007-07-11T08:23:00.001+10:00

Are you so busy that you battle to find time to have the kind of conversation with people that absorbs your full attention? The kind of conversation where you're listening to them with your eyes and ears and speaking to them from your heart? Do you instead write them emails, speak in bullet points and hope that when you call their phone you'll go straight to message bank so you can leave a concise message without getting caught up in small talk?

Are you writing your business goals and "communicating" them to everyone through email and presentations? Is "consultation" when you run some brainstorming workshops so people feel that have participated (irrespective of what you do with their ideas)? Then you are very likely still having trouble getting people to understand and buy-in to your strategy, performance measures and performance improvement.

Emails, brochures, PowerPoint presentations, strategy documents and vision/mission posters fail to get people excited about organisational performance. They consist of words and maybe a few images that are usually too vague and too bland to paint colourful and animated visions in the minds of those that read them. These artefacts of modern organisational strategy are always political: designed more to not provoke those that would oppose it, designed less to evoke those that would bring it to life.

When people read things that are written in typical management-speak, what happens in their minds, honestly? They can jump to their own conclusions about what "efficient, effective and productive best practice processes" look like, or they can slide deeper into cynicism or learned helplessness, or they can keep on keeping on, oblivious and unresponsive to any change in organisational direction. Not buying in, not owning it, not seeing their own aspirations and values in it.

Remember: staff usually have no knowledge whatsoever of the conversations that were had before the goals were written (and polished and rewritten and polished some more). The seven strategic objectives or the five critical success factors are just the sanitized remains of what probably started out as a very rich, emotive and inspirational dialogue about the things that really matter right now for the organisation. And here lies the secret to getting staff to buy-in to strategy: giving them that same chance to engage in rich, emotive and inspirational dialogue about what matters most right now for the organisation.

When you take the time - and it need only be one hour each month - to facilitate a conversation among staff about what the organisation should look, sound and feel like, then you'll have started the transformation. Stimulate this conversation with prompts like these:

What results are implied by our goals?
If we were already achieving our goals, what would we notice was different to how things are now?
What are some of the things that our team does that directly influence how the organisation's goals are achieved?
Are there some things our team does that impacts on other teams' performance?
What are the most important results that we should be trying to produce or improve?

This is the kind of conversation that should precede any other conversation about performance with your staff (especially the individual performance management or appraisal conversations). A leader has no right to expect staff to perform in a way that improves organisational performance if that leader has failed to make space and time for everyone to clearly and colourfully paint in their minds a picture of that place in the future they are collectively trying to create.

#48 Five Tips for Naming Measures

2007-07-11T08:22:00.002+10:00

What's in a name? Well for performance measures, there's a lot in how they are named. Different organisations, in their performance measure experiences, have helped me see that what we call each of our performance measures can have a big impact on how useful those measures are. Here are five of the tips I'd recommend you consider when you want to formalise a particular measure in your organisation (you don't have to use them all, though):

unique name
accompany with a description
motivating language
adopting industry standards
5 words or less
leave the target out

tip #1: give each measure a unique and specific name

A transport company I have worked with measures hundreds of things. One of them is the number of orders for deliveries. A pretty straightforward measure, you might think. Except that depending on who reports it, it is called different things, so users of the reports never know exactly what they are looking at.

Make sure the adopted name is the one that is used where ever and when ever that measure is reported.

tip #2: accompany every measure name with a description

Have you ever been frustrated by a report where a name like "Customer Loyalty Index" sits above a chart, and you have no idea what the numbers mean?

Use a sentence that describes what your measure is, giving more information than any name can. You might like to include things like the type of statistic (e.g. average or percentage), for what population (e.g. all employees versus non-managerial employees), and what the construct of the measure means (e.g. have attained all competencies associated with their current roles).

tip #3: use engaging and motivating language

I've recently worked with an organisation whose people are very creative, and they inspired me with their approach to naming measures: they used very emotive exclamations as measure names. For example, "You can't keep me away!" as the name for a measure of customers coming back for more.

Play with using affirmations, catch cries, headlines or other sensory rich statements to name measures.

tip #4: adopting industry naming standards

In the procurement industry, how fast inventory is turned over is a commonly used measure, and most often, it is referred to as 'Inventory Turn'.

If you're using measures that are accepted more widely in your sector or industry, adopt the naming conventions that are already accepted.

tip #5: use five words or so in the name

Too few words in a measure name can be as bad as too many. "Customer Index" says virtually nothing, whereas "The percentage of customers that either strongly agreed or agreed that our service is better than any of our competitors" is too long. A balance might be struck half way between the two: the measure name of "Compared to Our Competitors" with a description matching the longer statement above.

Aim for writing you measure names in around 5 words, and fine-tune it from that starting point.

tip #6: leave out the target

"Reduce waste going to landfill by 20% next year" is not a measure, but a goal (or objective if you prefer). The measure is actually the amount of waste going to landfill. The rest of it is really the target and timeframe.

Because measures often outlive their targets (that is, a single measure may have several targets throughout its lifetime, each subsequent target encouraging further improvement), name your measure before you frame it in a goal or objective statement.

the advantages of well named measures

Irrespective of whether you take on these ideas for naming measures or not, you'd have to acknowledge that when measures are named well, they get higher recognition, greater ownership, and far less confusion. So thoughtful naming of your measures is one little thing you can do toward simplifying an activity that probably already causes you more rework than you dare to think about!

#47 Measure Your Measurement

2007-07-11T08:22:00.001+10:00

How do you know if all your efforts to do with measuring organisational performance are efforts worthwhile? Do you know what impact your measurement system is having on the very things it's there to help improve (which is organisational performance, in case it's not obvious)? What we're talking about here is measuring the performance of your performance measurement process.

Yes, it probably feels a bit like your brain is bending back onto itself, but there are some very good reasons why measuring your measurement process is so worthy a cause. You can do things like:

better convince people that performance measurement is worth doing
improve your measurement process like you'd improve any other process
evaluate different approaches to performance measurement to find the best
make the accountability of your Planning & Performance team more objective

But first, you'll need to think about how you go about measuring a performance measurement process. A sensible place to start is to decide what are the results you most want from measuring performance, then design measures for these results. And to give you a head start, here are some ideas about the main types of results you should consider.

result #1: people understand their role in achieving organisational goals

Performance measurement is (or should be) connected to the planning process like your ears and eyes are connected to your head. When measurement is done properly (that is, measures are designed and not brainstormed), it makes the goals of the organisation much less fluffy and much more tangible. And it puts persuasive pictures in the minds of people of the future they're going to help make real.

result #2: people find it easy to measure what matters

You know that the best way to get buy-in from people is to get out of their way, don't you? People will only love their performance measures if they conceived them and brought them into the world themselves. To do this, obviously they need to know what steps to take to design measures and how to decide what is worth designing measures for and what is not.

result #3: the measures are used (to improve organisational performance)

A great measurement process isn't about selecting measures. It's about bringing them to life and making sure they get used. You want people using them regularly, using them constructively, using them to test their prior decisions and actions, and using them to prioritise where they spend money for the good of the whole organisation.

result #4: organisational goals are achieved faster

A reasonable performance measurement process will mean you are measuring your goals. But an outstanding performance measurement process help you achieve those goals quicker than other processes. But you want to make sure that there aren't any unintended consequences, like there often can be when you're trying to do things faster.

result #5: the measures used create more value than they cost

The return on investment of your performance measurement process is just as important as the return on any other investment your organisation makes. If you've got a fabulous performance measurement process, then use of the measures has produced more savings or other value for your organisation than the cost of creating, reporting and using them.

how important is it to you to know?

If you want to know how good your performance measures are, really, a survey of staff or stakeholders just can't cut it. You have to put more elbow-grease into it than that, which means you must decide what impact you want performance measurement to have, and design measures that give you the evidence of that impact, specifically.

#46 How To Design Great Measures

2007-07-11T08:16:00.004+10:00

Are you guilty of using the following methods as your approach to measure selection:

brainstorming with your team in a one-hour session during your two-day planning workshop?
trawling the internet or other places to find out what others like you measure?
asking your IT guy or gal what data you have and creating measures from that?
hoping someone will tell you (maybe a consultant or a stakeholder)?

These aren't approaches to measure selection. They are just ways to gather ideas for what to measure. None of these methods include any kind of overt and deliberate evaluation of which measures are the best measures. And you're probably wondering why your organisation has so many meaningless measures! Plus, on the flip side, these methods may have left you high and dry without any viable options for measuring some of those less tangible results like culture or sustainability or engagement or confidence.

If this is your burden - having no logical, practical way of choosing or designing the measures that can truly convince you that you're making the differences you need to in the world - then here are three tips that might make your life a little easier:

measure the result, not the action
what you can observe and describe, you can measure
don't measure it just because it's easy

measure the result, not the action

"Ensure that people with a disability do not experience discrimination and have their particular needs for services and support acknowledged and met." Such an inspirational and noble goal is so easily cheapened by a measure like "Establishment of an effective Advisory Council on Disability". Such measures track the activity associated with the initiatives hypothesised to produce the results implied by such wonderful goals. They can't let us know how much or how frequently people with a disability experience discrimination.

No doubt you're going to want to monitor activities in your organisation, but what meaning does that have unless you are first monitoring the results those activities exist to produce or influence?

what you can observe and describe, you can measure

Why is it so hard to measure results like "achieving equi-marginal efficiency for trade-exposed industries on a least-cost trajectory within a general equilibrium model"? The answer is not because no-one has thought of the right measure yet! If we go back to basics, measuring is about observing and collecting specific information about something. If you don't know how to recognise when that something is happening, you can't know where and when and how to collect information about it, can you?

So before you can measure "achieving equi-marginal efficiency for trade-exposed industries on a least-cost trajectory within a general equilibrium model", you need to know what equi-marginal efficiency and least-cost trajectories look and feel and sound like when they're happening. Even more, you need to be able to describe it in words that are evocative, words that conjure rich and detailed and shared pictures in the minds of people before they select or design measures.

don't measure it just because it's easy

It's easier to select more viable candidates for measuring a result when you describe it richly and without weasel words. In fact, you can end up with so many candidate measures that you might be tempted to pick those that are the easiest to bring to life. You already have the data, it would be a waste not to use it, no-one's got the time to collect more data. But you'd be falling into one of the deepest traps of organisational performance management: not making sure that your organisation has the data it really needs.

You'll need to think about more than just the feasibility of each potential measure in deciding on the best ones. Measures are meaningful when they have strong relevance to the result you want them to evidence.

use your brain when you design your measures

Three basic steps to better measures: focus on the result and not the activities, articulate clearly what that result looks like, and shortlist your potential measures by balancing feasibility with strength of relevance. Yes it will take a little more time that you have probably been giving to measure selection, but it will save you loads more time than you have probably been wasting managing with the wrong measures.

If you want to get some step by step help (plus examples and templates) to design your measures, visit http://www.pumphowtokits.com/measuredesign.htm

#45 Why Do We Measure, Anyway?

2007-07-11T08:16:00.003+10:00

Why do we measure organisational performance? The first answers that pop into your head might be:

you can't manage what you don't measure
what you measure gets done
we have to be accountable
they have to be held accountable
they told us to

These aren't the answers to the question this article asks. The reasons why so many organisations - particularly high performing organisations - measure things are more authentic, more fundamental and more motivating than those listed above.

to avoid knowing too late

At a government agency executive meeting I attended, participants were evaluating whether an end of year revenue target had been met. No it hadn't, and they did have lots of reasons why, most of which were how the market was changing and all their competitors were facing similar revenue downturns. If they'd had this kind of conversation more frequently throughout the year, perhaps they would have had time to create some strategies to better understand what was happening in their market and find new avenues of revenue generation.

Annual evaluation, or end-of-project evaluation is always too late to give you choices about changing your course. Are targets just about playing numbers games, or do they really represent important changes to ensure future health? The above organisation is no longer in existence. Perhaps if they'd treated their revenue target more seriously, they might still be around.

Frequently reported measures can give us early warning signs about whether what we are doing is actually making the differences it's supposed to, early enough that we have the chance to modify or stop doing it if the intended results are not forthcoming.

to avoid knowing too little

My friend works in a wholesale technology business that operates out of two cities over 1000km apart, with a staff of about 25 people and they sell approximately 50 product lines. The directors of this company only measure typical balance sheet stuff. Their staff complain incessantly about product returns, warranty service workload and availability of spare parts. Do they measure any of these non-financial things? No. They reckon they don't need to, because it's a small business and they can see what's going on by walking around. But the same simple problems that plagued them six years ago are still plaguing them.

Can you be everywhere at the same time, all the time in your organisation? Of course not. Most of what goes on in our organisations our physical senses (sight, hearing, touch, etc...) can't absorb or even detect with sufficient reliability for us to understand them.

A small suite of performance measures help us know far more about what is going on with the health of our organisation's processes, than our own eyes and ears ever could, with any reasonable amount of reliability.

to know the right things

A manager in the rail freight industry faced a typical problem for that industry several years ago: they were running out of capacity to move all their customers' produce. The typical solution to this problem is to invest in more rollingstock. Millions and millions of dollars worth. But he didn't take the typical solution. Instead, he measured and studied the way the system worked until he discovered that it wasn't how many wagons you had, but how quickly you could cycle those wagons through, that impacted the capacity. So he didn't need to buy new wagons because he did find a way to cycle the wagons through the system much faster, ending up with even more capacity than they actually needed.

How well do the decision makers in your organisation learn about what works and what doesn't work in fixing performance problems? Trial and error? Following traditional, already-proven strategies? How much real learning do they do about the real leverage points of unacceptable performance?

Well chosen performance measures, that monitor the root causes of the most important organisational health results, are measures that focus us on the things we really need to know. They help us break away from knowing things that really don't make much of a difference.

why do you measure performance?

If you aren't measuring to know enough about the right things, and frequently enough to do something about them, then perhaps you're not actually measuring performance?

#44 The Social Life of Performance Data

2007-07-11T08:16:00.001+10:00

One of my clients is drowning in dozens of reports collectively containing over 100 measures. Where he expects two measures from separate reports to have the same values, they don't. Where he expects a measure's value to be accepted by his customer, it is disputed. Where he thinks he's looking at the right measure to answer his question, someone warns him no. The tangle of reports and measures is unwieldy, but has become the dogma of decision-making. Untangling them all into a streamlined sensible suite of reports is not as simple as setting up a swanky scorecard.

Data quality worries most users of performance measures. There are an obscene number of reported measures that only generate dialogue about how unreliable the underlying data is. But what can you do about the quality of performance data? I've heard some performance measure experts proclaim that performance data must have 100% integrity. Hogwash! It never will, and here are some of the reasons why.

performance data is gathered by people

A vast proportion of our performance measures rely on data that has been touched at least once by human hands. People design data collection forms and processes, people fill out those forms, people enter the data from the forms into computer databases, people extract and manipulate data out of databases, people filter and analyse the data to produce performance measures.

So human error and misunderstanding, ambiguity or absence of clear data definitions, ad hoc data collection and analysis processes, and vague measure definitions (the calculation of measure values) all contribute to the low confidence people have in reported measures.

How many of your performance measures are defined in enough detail to avoid miscalculation or use of the wrong data? How many of your data collection processes are documented consistently and ingrained into work practices? How many of your people that collect data have been trained to do it according to the documented process? Does your organisation have a data dictionary that is available outside of the IT team?

people know that performance data can sting

Unfortunately many of our organisations are still carrying the burden of a blame culture. People can still remember (or are still experiencing) the use of data as a big stick to humiliate, take resources away from, demote or sack the so-called poor performers. We know in this kind of environment people swing into self-preservation mode (it's only natural) and weigh up their choices: cop another whack with the data stick or sweep that nasty data under the rug?

Managers and decision-makers need to earn the trust of employees again, that data will not be used against anyone. Performance measures and data need to be seen more often being used to honestly assess performance of systems and processes, more often being used to explore root causes and learn from the past, more often being used to stimulate dialogue about how the future can be influenced.

How many of your managers and decision-makers look for root causes of undesirable performance in the systems and processes (as opposed to the people)? How many performance measures are supported by diagnostic measures of causal factors (as opposed to just slice-and-dice the data into smaller fragments)? Have you got an automatic improvement process that kicks in when a performance measure reveals a problem?

data has no meaning apart from its context

An event must occur before data can be produced. And the data is the product of the event being observed and interpreted and coded. When people are doing the observing (as opposed to a machine such as a temperature gauge), the person unconsciously - and occasionally consciously - applies filters that affect how the event is interpreted and how it is coded.

These filters are influenced by beliefs the person has about the event, their interactions and relationships with others around them, their physical and mental health on the day, what they are thinking about at the time, their values and priorities regarding their work, and the list goes on.

Have you explored the context around the types of performance data you collect? Have you thought about the factors that might influence the way someone interprets and codes what they observe when they are capturing performance data? Do you have guidelines and examples in your data collection instructions to help data collectors capture quality data?

don't just rely on technical solutions to data integrity problems

Yes, there's certainly more to the social life of data than the three parts discussed here. Most of them can be discovered and dealt with through better communication among the people involved in data capture: from designing measures to developing data collection processes, to collecting data, to storing and analysing it. Don't rely just on the technical solutions - think through what needs to change in the social systems surrounding data. And be concerned more with how much integrity your decisions can survive with, as opposed to 100% integrity.

#43 Three Approaches to Traffic Lights

2007-07-11T08:14:00.000+10:00

Traffic lights – the decoration de rigueur for performance dashboards and reports. Have you gotten more carried away with the decoration, than with the rigueur? Take a look at these four common approaches to traffic lights, and see if you’ve got some room for improvement.

approach 1: % difference from month to month

When this month is 10% worse than last month, the traffic light turns red. When it’s 5% worse than last month, the traffic light turns amber. When it’s 10% better than last month, the traffic light turns green. Obviously, this approach works for time periods other than a month, and for cut-offs other than 10% and 5%.

Such traffic lights encourage us, usually, to ask questions like “what caused such a big difference?” In turn, such questions encourage us, usually, to find some way to explain the difference. If we’re clever, we’ll already have added a comment to the performance measure explaining that the difference is due to something outside our control. If we’re not so clever, we’ll be putting up different explanations every month, and have a list as long as Santa Claus’ of improvement projects.

There’s no advantage I can see to this approach to traffic lighting. It tends to encourage us to knee-jerk react to data, tamper with business processes or blaming something we don’t have to do anything about. Time gets wasted chasing problems that aren’t there and we miss problems that are.

approach 2: up and down, good and bad

When some performance measure values increase, it’s a good thing (like revenue, satisfaction and on-time performance). There are others whose values decrease and it’s a good thing (like rework, cycle time and pollution). Combine this with whether there’s an upward change or downward change in actual performance values and you get a complex range of traffic light signals to deal with: upward change that is good, upward change that is bad, downward change that is good, downward change that is bad. This “solution” probably resulted from a confusion that erupted when upward and downward arrows were chosen as the traffic light symbols.

When we sort out the confusion, these multi-faceted traffic lights encourage us to ask questions like “what’s behind the trend?” and the trend is concluded from maybe 3 consecutive points of data. Marginally better than approach # 1, and only just.

Any system of traffic lighting that moves us away from point to point comparisons (the essence of approach # 1) is a step in a good direction. But we still risk drawing the wrong conclusions from trend analysis that is based on not nearly enough data to be valid. And does upward and downward really matter nearly as much as good and bad?

approach 3: statistically valid signals

Statistical process control is an analysis method that discerns variation that is typical from variation that signifies change has occurred. It’s like filtering the signals from the noise, something the other two approaches don’t do (they assume that any arbitrary difference is a signal, irrespective of the typical size of differences over time). The signals are defined from a set of rules that test the probability that a difference is due to just normal variability (no change) versus atypical variability (change). Signals include sudden shifts in performance, gradual shifts in performance and instability in performance.

When our attention is moved from point to point variations to patterns in variation over time, we ask questions like “what caused that shift in performance to occur at that time?” and “why is performance so chaotic and unstable?” and “what do we have to focus on improving to improve the overall average level of performance?”.

These questions seek root causes, not symptomatic causes. They lead us to find the solutions that don’t just fix next month’s performance, but fundamentally improve the baseline performance level further into the future.

#42 Techniques For Cause Analysis

2007-07-11T08:07:00.000+10:00

Measuring performance results is a great thing to do, but understanding the causes of those results is at least as worthwhile. Understanding causes means you have information about how to exercise more influence (or control) over those results. If you want your results to improve, you've got to change the right things about the process or activity or function that produces those results.

Understanding the real causes of performance results means taking a more rigourous approach than knee-jerk reacting to hearsay, opinion or gut feel. Here are some basic techniques to help you navigate through the stages of cause analysis:

find the likely causes, and measure the incidence of each
assess the nature and size of the cause's impact
check for interaction with other causal factors

Technique #1: flow charting

It's impossible to do any kind of serious cause analysis unless you can actually trawl through all the factors that have some kind of potential impact on your performance result, and sift out those factors that have the most dominant impact. Flow charting the process or activity or function whose results you are measuring, is a great way to systematically trawl through all the potential causes of those results. There is software available for flow charting, but hand-drawn charts are quick and easy.

Technique #2: cause-effect diagrams

After flow charting your process and identifying what can sometimes be dozens of potential causes, you can have long lists that contain duplicates and related causes. Cause-effect diagrams (or fishbone diagrams) are a great way to collate and organise potential causes as you identify them, clustering related causes together so you can more clearly see the themes, and more easily discuss the most likely causes. There is software available for cause-effect diagrams, but again hand-drawn diagrams can do the job well enough.

Technique #3: Pareto charts

When you then go and count or measure how often or how much each likely cause is associated with your results, Pareto charts can then help you rank the causes and highlight those that have the biggest impact. You're now getting the stage where you have between 2 and 5 (roughly) causal factors you may wish to learn even more about. In Microsoft Excel, just use a vertical bar chart on your data, after sorting it from biggest to smallest.

Technique #4: scatter plots

When you arrived at the few causes that have the biggest impact on your performance result, it can be useful to know just how big that impact is. Scatter plots are an easy and visual way to explore when the cause variable changes, how much and in which direction the performance result changes. Scatter plots are one of the charts available in Microsoft Excel.

Technique #5: correlation coefficients

To get more a quantitative measure of the impact of a causal factor on your result, you can calculate a correlation coefficient which will give you a value between 0 and 1 indicating the strength of the relationship between your causal factor and the result. A postive value means that an increase in your causal factor will likely lead to an increase in your result, and a negative value means that an increase in your causal factor will likely lead to a decrease in your result. In Microsoft Excel, use the CORREL function to calculate your correlation coefficient.

Technique #6: regression analysis

Regression analysis goes a step further, and builds a mathematical model you could use to predict a result based on a change in your causal factor. Knowing this can help you set achievable targets for improvement, and estimate realistically what resources you're really going to need to get that improvement. In Microsoft Excel, create your scatter plot between your causal factor and performance result, then add a trend line, with the options of showing the equation and R-squared value on the chart (the R-squared value is a measure the reliability of the equation).

There are certainly more statistical techniques that can help you with cause analysis (such as multi-variate regression, experimental design and analysis of variance or ANOVA), but those provided above will still bring some valuable rigour to your performance improvement efforts.

#41 Drowning in Activity Measures

2007-06-17T03:04:00.000+10:00

If you were to ask me the most common mistake I see with performance measurement, I’d probably have to say it’s the use of activity measures. What I mean by activity measures is this:

Completion of system audit
Implement policy review by June 2007
Number of training programs conducted

It’s when people count up whether or not, or how many times, they’ve done a particular type of task. Why do I think activity measures are a performance measurement mistake? Here are five reasons.

Reason #1: activity measures often aren’t really measures

Why aren’t “completion of system audit” and “implement policy review by June 2007” really measures? Because they’re events. They’re milestones to reach, usually within a project or implementation plan of some kind. They are activities. Measures are evidence of the degree to which something is occurring, through time. Measures are supposed to be regular and ongoing feedback that we can use to adjust our course so we continue heading where we need to go.

If you want people to improve their ability to meet deadlines, like the milestones described above, then you don’t measure a single deadline as “met” or “not met”. You get tonnes better information if you track the proportion of deadlines met as time goes by, months and years. And thus, activity measures like these milestones are really just the pieces of data that would comprise a measure that has everything to do with someone’s on-time performance and nothing to do with the nature of the activity itself. That leads us to reason #2.

Reason #2: activity measures don’t measure performance

Many times I’ve seen activity measures being used as evidence of the achievement of strategic results. For example, one organisation (through the use of this measure in their strategic plan) seems to believe that the number of people trained is evidence of how committed their staff are. Activity measures are evidence of nothing more than the activity having occurred. The result of the activities is another matter entirely.

Evidence. Feedback. These are important words when it comes to measurement. Good evidence is convincing, and will convince the right people that a particular result is really happening. Good feedback is regular, and when it is regular enough (not annual) it helps people change their activities to stay on track to influencing their desired results to happen. Of course, there’s the presupposition that people have results that they want to happen.

Reason #3: activity measures drive the wrong behaviour

A measure that monitors the amount of activity being done sends the message to do more of that activity. These measures don’t send the message to do the right activity, and they don’t send the message to do the activity well.

That said, you can still devise some good activity measures that encourage the kind of behaviour you want. For example, one of my own personal performance measures is the number of articles and books I read each week. Because I hate wasting time reading things that don’t add to or challenge my knowledge, I know that this measure is going to drive me to read lots of valuable literature. So if you carefully think through the behaviours you want and the unintended consequences that might surround those behaviours, you can still use activity measures to improve performance.

Reason #4: activity measures waste data collection resources

Generally it’s really easy and quick to measure activities. We have easy access to the data, it’s not too hard to collect, and most organisations have systems for capturing a good deal of it. And it’s data that can rarely inform any important decision. The more activity measures we have, the less meaningful data about results we seem to have.

Letting go of our plethora of activity measures has the potential to free up time, energy and mental focus to start collecting and using data that gives us measures of results. Letting go is hard, but trying to improve organisational performance using activity measures is much harder.

Reason #5: activity measures breed learned helplessness

If there is one thing in our lives that we can control most, it’s what we do. Our direct activities are more within our control than the results of those activities, for example. Using activity measures to assess our own performance is usually associated with a desire to measure only what is in our circle of control.

What a waste! What if we all turn up to work and just accept responsibility for our circle of control, and don’t accept any responsibility for our relationships with other people, for our impact on the activities and work of other people, for the impact of our activities on the world around us? We all have a circle of influence, and we only really perform when we start measuring and improving what lies in our circle of influence.

#40 Advantages of Samples

2007-06-17T03:00:00.001+10:00

For the last 3 years I have been helping a client take a sampling approach to measuring the accuracy of their inventory records. The measure is the net error rate, based on the size of the difference between their electronic inventory records, and the actual inventory that is held at the storage locations.

you can save money and time

Sampling the inventory storage locations instead of trying to do a full stock take saves this client millions (probably more). They have thousands of storage locations, carrying many, many thousands of inventory line items. Counting them all would require an army. And even though they didn’t used to try to count them all, they are now spending less time and less money than they used to. Where can you save time and money by using samples to measure, instead of trying to measure everything?

you can improve the integrity of your data

And even though they are spending less time and money than they used to, because of the way we designed the survey, they are getting much more integrity in their data. The secret here is in segmenting the storage locations and the line items by value, designing sample sizes in each segment that accommodate the variability within each segment, and randomly selected the samples. The random selection is very important! Are you selecting samples that aren’t random? If so, your data is most likely biased, and could be misleading your decisions.

you can measure more frequently

Because measuring with samples is less costly than measuring everything, you have the option to measure more frequently. Often it is just as easy to do monthly “pulse surveys” of small random samples of customers, as annual surveys of larger samples. But with annual measures, you have to wait a long time to work out if change is happening. Monthly measures (or other more frequent timeframes) more powerfully show you emerging and sudden change, and you can then respond before it’s too late.

you can refute the squeaky wheels

Well designed (and randomly selected) samples are not subject to the bias that some of the more familiar data collection methods have. The old inventory accuracy measurement was biased because auditors would choose to measure the sites they suspected had the worst accuracy. Now the client can put any specific accusations about poor inventory management into context. His recommendations about improving inventory management are objective and considered, rather than knee-jerk reactions. Who are your squeaky wheels, how much do they influence decision making and resource consumption, and how could a well designed sample approach help?

#39 Getting People Involved in Measuring Performance

2007-06-17T02:59:00.000+10:00

Getting people involved in measuring and improving performance is one of the greatest challenges (and greatest enablers) in designing measures that lead to improvement. But just inviting them to a workshop, or telling them to come up with measures, rarely works. Here are my favourite ways to authentically involve people, in a way that has meaning for them.

idea #1: ask them what their biggest obstacles are to doing their job well

What will get people's attention more than talking about what bugs them the most? And what better a place to start involving them in performance improvement than in helping them improve what matters to them? Even if what bugs them isn't strategically important right now, it's a valuable exercise that will lead to them thinking more easily on what 'bugs the organisation' (the biggest obstacles to the organising performing well).

idea #2: ask them to give feedback on someone else's measures

Aside from the obvious value that comes from getting feedback from others, this approach makes it safe for people to start getting familiar with performance measures, without feeling imposed. As part of my PuMP Implementer Program, we use a measure gallery as a way gathering feedback from the wider organisation on the measures a team has just freshly designed. These measure galleries have been known to generate lots of interest in other parts of the organisation to further explore performance measures for themselves.

idea #3: coach them as individual people, rather than facilitating or teaching them as a group

The group approach, where you get people in the room and walk them through designing measures, is certainly very efficient. But it can leave people behind - people that are worried about a problem on the job, people that are starting from less experience with measurement, people that feel cynical about measurement ("It's just another big stick!"). Outside of workshop time, you may well need to share a coffee or chat over the phone to support a few participants that are feeling less than committed to the process.

idea #4: ask them why they aren't as involved as you'd like

Often we just assume that people don't care, or haven't got the time, or just have a bad attitude, and that's why they won't get involved in measuring performance. (And you know what assumptions are, don't you?) Instead, just ask. Not everyone will feel confident to give you an honest answer, but some will. And the conversation you can then have to explore their objectives, answer their questions, take on their ideas, could be just the opportunity you need to get them more involved.

idea #5: role model the design and use of great measures

Your actions speak louder than your words. You can't expect anyone to get involved in something you aren't involved in yourself. So establishing a few great performance measures that will help you improve performance, and then using those measures to make performance improve, can be a great way to show others the value of doing it too.

#38 Measuring the Impact of Initiatives

2007-06-17T02:58:00.001+10:00

One of my business goals is to increase subscribers to the mezhermnt Handy Hints ezine, so I can get lots of useful information out to lots of people, and also help people get to know me and the PuMP approach to performance measurement.

Obviously I can't control whether someone joins the ezine list - it is unethical to simply add people to the list without their permission (do you recall the confirmation you had to give in order to be added to the mezhermnt Handy Hints list?). But I can influence a few things that increase the number of people that find out about it, and even the proportion of those people that go to the next step and sign up.

So whether your improvement initiatives are small like mine, or much larger and more complex, there are a few good tips to consider when you measure the impact your improvement initiatives have on the intended results.

tip #1: start with some baseline data

The performance measure for building my list is the number of new subscribers. Before starting any list building initiatives, subscriptions were averaging about 10 per week. That's my measure's baseline.

What's your performance measure's baseline? Did you measure it before you began your improvement initiatives? Can you establish the baseline from historic data, or estimate where it was at that time? Can you use correlated data to calculate roughly where it was?

tip #2: pilot test each initiative separately

I don't just implement all the possible improvement initiatives for list building at once (it is tempting, but not sensible). Why? Because I want to first know how effective each strategy is for my situation, and then only invest in the strategies that work best. For solo professionals and large organisations alike, time and resources are limited and must be invested where they get the highest return!

My initiatives include Google Ads that appear when someone searches for "kpi" or "balanced scorecard", improving my website design to list higher in internet search engines, and publishing free articles on the web. For 10 weeks I tested Google Ads, in isolation of any other initiative. The effect was that subscriptions lifted to an average of 105 per week, an impact of 95 subscribers per week (not too shabby a result).

Are you in the habit of jumping into several solutions and actions before really testing the size of impact of each? Yes testing is slower, but you'll just waste time and money on initiatives that don't really work, when you have no way of knowing either way.

tip #3: use diagnostic indicators too

There are a few other indicators that are also useful to give more information about the ezine sign-up process. One is the click through rate of people that become aware of my ezine, who then sign up. By adjusting the wording of the Google Ad, I can finetune its relevance to people who search for information on KPIs. Of the 3 Google Ads I tested, one achieved a click through rate of 0.3%, and the other two were equal at 2.9%.

To decide which of the two better ads to go with, I needed more information. Another diagnostic measure is the position that Google ranks my ad along with other ads for the same search keywords. One ad averaged around position 5, and the other around position 4. Now I know which ad performs the best, and what size of impact it is capable of having on my performance measure of subscriptions.

Do you know which of your initiatives are most successful, and by how much? Have you tested variations of your initiatives to pinpoint what makes them more successful? Diagnostic indicators can be designed before you test your initiatives, but often you discover helpful diagnostic indicators during your testing too.

#37 Realistic Target Setting - Part 2

2007-06-17T02:57:00.001+10:00

The last 3 of the 6 most common worries about setting targets for performance measures are:

challenge 4: Anticipating the consequences of achieving and not achieving the target.
challenge 5: Finding the courage to go beyond your comfort zone.
challenge 6: Having the wherewithal to change whatever must change for the target to be accomplished.

Here are my ideas and learnings about overcoming them.

idea #4: keep one eye on the target, and one eye on the bigger picture

Even if you had enough foresight to explore the unintended consequences of achieving your target before you locked it into your plan, the world will still change later on. I once heard a story about a rail organisation that placed more importance on on-time running of trains than any other performance outcome. So much so, that one day, due to pressure risking the train running late, the driver omitted an important safety check to save time. The train derailed because of a braking problem that the safety check would have easily picked up.

Every now and then, ask your self "is this target still a good idea?" and "if we miss it, what's likely to happen?" and "if we achieve it, what's likely to happen?". It's okay to change a target that is no longer going to serve its original purpose. Is this check a part of your regular performance review process?

idea #7: give yourself (and your staff) permission to learn by not achieving targets

You are not supposed to achieve every goal or target you ever set. And if you do, then it's probably because you aren't challenging yourself enough. You're staying inside your comfort zone, inside of what you know works, what you know you can accomplish. That's not what improvement is about. There is no learning without failing, no improvement without learning.

If you want to jump over a creek without landing in the water and getting your shoes all wet, then don't aim for the far bank of the creek. Aim for a metre or so beyond it. Set the target further than you think you can achieve. That way, you'll be less likely to land in the water, and more likely to land even further than you thought possible. Somehow, our strides are longer when our eyes focus further ahead.

idea #6: do some preliminary scoping of "how-to" before locking in the target

If you and your team do not yet possess the target setting and achieving prowess of an Olympic athlete, then avoid setting any kind of target without first exploring a range of ideas of how you might go about achieving it. A very innovative manager I know has for years used simulation software to model his business processes (freight). The model simulates the steps in the process, the variability in the time each step takes, the variability in market demand, resource constraints, and much more. He can then make changes in the model to simulate changes like investing in more equipment, or changing a step, or removing a constraint (like a policy). So before he spends a single dollar, he can get a good idea about which strategies are going to work best to reach his targets.

What's wrong with taking an iterative approach to finding the right target? Scope a little and set the first target value. Explore what it might take to achieve that, then revise that value if necessary. Start the more detailed action planning to get a stronger idea of resource implications, and revise the value again if necessary.

#36 Realistic Target Setting - Part 1

2007-06-17T02:56:00.000+10:00

Some of the most common worries about setting targets for performance measures are:

challenge 1: Striking that sensitive balance between making the target achievable but also a stretch.
challenge 2: Creating that sense of urgency that will motivate people to hunger after the target.
challenge 3: Having a measure or means of monitoring progress as the target timeframe approaches.

I'd like to share some ideas with you, about how to lessen the burden when you come face to face with worries like these.

idea #1: don't strike a balance between achievable and stretch - do both

What I've learned is that it takes practice and confidence-building to achieve a target or goal. Why not set at least two or three targets for any single performance improvement? The first one is shorter term and not very challenging, for the purpose of building target-accomplishing momentum. The interim target is an opportunity to build more capability and confidence to stretch. The last one is the stretchy target, which you might have no idea of how to reach at this point in time, but be in a better position to know after you've achieved the interim target.

idea #2: use vivid and specific language to describe the world after the target is accomplished

Numbers alone are hardly enough to motivate anyone. So handing a team a performance measure + target value + timeframe won't likely be enough motivation. Have you ever tried telling the story about what the world (or at least your part of it) is like after the target is met? Colour, sound, movement, emotion, expression, behaviour, shape, rhythm and all those other sensory experiences emblazon the meaning of the target into the minds and hearts of those setting out to achieve it. Motivation from within is the best kind.

idea #3: make sure your measure can be monitored at least 6 times within the target timeframe

Design your measure so you can calculate it as regularly as is feasible, and then set a target timeframe that accommodates frequent enough feedback to increase your chances of staying on track. For example, monitor your measure weekly or monthly for a 1 to 2 year target timeframe. Yes, sometimes you just can't get data this frequently, but that doesn't change the fact that a single point of data says nothing. Is it worth setting a target that you cannot honestly know is achieved?

Stay tuned next month for the next 3 challenges of target setting!

#35 The Good, The Bad, And The Ugly

2007-06-17T02:53:00.000+10:00

There are lots of so-called “measures” that people choose to monitor business results. Some are good, some bad and some downright ugly! This is one of the most colossal mistakes I see people making with performance measures: to claim as a performance measure something that absolutely is not a measure of performance at all.

Here are three of the so-called performance measures that I really dislike most:

"win the BlahBlah Award"

The award might be a customer service award, or environmental award, or workplace health and safety award. Why do I dislike awards as measures? The winning of an award is an event, and can’t give regular, ongoing feedback that can inform decision making and improvement - to use it as evidence of business performance assumes that the criteria for the award correlate directly to the business’s priorities and strategy. And just think about the kind of behaviour and culture this kind of "measure" would encourage... everyone aiming to impress the judges of the award and taking their eyes off their real stakeholders.

"complete BlahBlah Project by June 2007"

Projects such as implementing a customer relationship management system, or upgrading a maintenance facility, or running a new employee training program are typically put in the KPI column of business plans. They are next to useless as evidence or feedback about business performance. Finishing a project by a particular date is an action, not an outcome, and thus provides no evidence whatsoever of the result that the project should have achieved. But they are the most common type of "measure". My theory is that it's because we are an activity culture - we have been duped into the false belief that as long as we do things (and finish them on time and to budget) then we have succeeded. A little more scientific thinking would go a long way: we need to use measures to test our hypotheses that the actions we have chosen in fact do produce the results we intended. So in reality, measures like these are actually strategies - the means we have chosen to achieve the results we want for our business.

"Annual BlahBlah Survey"

The survey could be an employee survey or a market survey or a community reputation survey - who knows? Irrespective, surveys are just data collection processes, not measures. The measures come from the data the survey collects and the measures must be very clearly designed and defined in order to ensure the survey collects the right data. Way too much money is wasted on surveys that ask irrelevant questions, and collect data that is never used. I guess having my foundations as a survey statistician makes me particularly frustrated by measures of this type. I'd just love to see more people demonstrating that they can discern the difference between data and measures!

#34 Zen, And The Art of Performance Measurement

2007-06-17T02:32:00.000+10:00

I just love the book "Zen and the Art of Motorcycle Maintenance" by Robert M. Pirsig, in part because I love philosophy, in part because I love trail bikes and in part because I am keenly interested in the issues of Quality versus Quantity (a major theme of this book). I'm just about to start reading it for the third time, because each of the last two times I drew new and different meaning from it. Anything philosophical awakens in me the almost overwhelming awareness that we are each part of something bigger than just ourselves, bigger than our day to day activities, our beliefs, our intentions and dreams and fears and penchants. Everything we "know" is relative - relative to the experiences we have had, relative to what we believe about the world, relative to our assumptions about the intentions of others, relative to what we have noticed and learned through our lives (and relative to much more too). Our "knowledge" is a mud map, not a satellite image from Google Earth and most certainly not the territory itself.

It's not hard to see then, why different people behave differently in response to the same performance measurement activity. And that's one of the big reasons why buy-in is such an elusive state to attain.

how people can react to performance measurement

Someone's map of reality influences the way they feel and act around performance measurement. Someone who is used to being blamed for things will feel defensive and fearful around performance measurement. They may throw up unlimited objections as to why performance measurement isn't needed or how they haven't got time to collect all the data. Someone who has put a lot of time (perhaps even blood, sweat and tears) into collecting performance data and never seen anything come from it will feel cynical and frustrated by performance measurement. They will at best bring their body to any new performance measurement initiative, leaving behind their heart and mind. Someone who has been frequently rewarded for outstanding performance would feel very comfortable and engaged around their existing performance measures, but may feel very nervous at the prospect of changing those performance measures.

These are just simple examples. And I'm sure you can imagine a selection of the these people in your own organisation. I've seen a selection of these people in just about every team I have ever facilitated through performance measurement activities. But getting these team members to a state of buy-in is something I seem to consistently achieve. How do I do this?

the art of performance measurement and buy-in

Performance measurement certainly does have (and need) a substantial technical base. Our performance measures would be a waste of time if they weren't linked to strategy, clearly defined, calculated consistently and using good quality data, or presented in a way that encouraged valid interpretation. However, our performance measures are also a waste of time if people involved in the measurement process (selecting measures, bringing them to life, or using them) don't buy-in to their measures, don't have a strong sense of owning those measures. This is the non-technical or human base to performance measurement, and without it, the technical base isn't enough.

Getting buy-in is, to me, more an art than a science. It's not about following a set of steps that will lead you to a state of buy-in. It's about creating and holding the space for people to safely explore what performance measurement can mean for them, personally. And creating and holding the space for this can mean adopting attitudes and behaviours like:

don't educate people in performance measurement - facilitate them through an action learning cycle that combines a little theory (such as techniques) and a lot of implementation (or even pilot testing)

don't tell people what they should measure - do show people a process to follow that can help them decide what is worth measuring themselves

don't be the judge, jury and executioner of people's measures - do suggest that people invite open feedback from all stakeholders about their chosen measures

don't micro-manage performance - do give people the time to use their measures to understand their performance and take the initiative to improve it themselves

don't blame people for poor performance results - do encourage people to analyse the causes, take corrective action and learn from this

don't assume that performance measurement is about control - do believe that performance measurement is about connecting people to something meaningful for themselves as well as the organisation (better control is a by-product of this)

How people respond to performance measures has a huge amount to do with how they connect themselves to a bigger picture. I'm curious about your ideas on this notion. Do you agree? Do you disagree? Do you have more ideas for how to create and hold space for people to buy-in to performance measurement? If you do have something to say, please send me an email!

#33 Compared to What?

2007-06-17T02:26:00.000+10:00

One of my favourite authors on subjects relating to performance measurement is Edward Tufte, who has written many books on the visual communication of information, including statistical information as it pertains to decision making (all his books, courses, musings and such are at www.edwardtufte.com). Why is he one of my favourite authors? He has extensive knowledge, he communicates this knowledge incredibly and entertainingly well, and he draws on many intrinsically interesting historic cases to illustrate his points. A quick browse of his website will confirm that for you.

[read my review of one of Edward Tufte's books. "The Visual Display of Quantitative Information" at amazon.com]

As Tufte puts it, "the deep, fundamental question in statistical analysis is Compared with what?" Information only has meaning in context, and quantitative information in particular runs such a high risk of misinterpretation in the absence of context. There are several types of context for performance measures to help you mitigate this risk of misinterpretation, three of which we discuss here:

the context of history

The easiest way to present your performance measures with some context is to add as much historical data as you have available (within reason) each time you report the measure. For example, if you are measuring things like revenue, expenses, profit, order cycle time, on-time supplier delivery, outstanding bills, rework, and so on, then try to report at least 20 points of historic values for such measures. Even less frequently measured results can benefit from the context of history, for example, if you have been running customer satisfaction surveys for a few years, then report your current customer satisfaction rating along with all the satisfaction ratings from previous years. And I can hear you say, "but the survey we use now is different from the one we used to use!" So we need an additional type of context...

the context of changes

There is no reason why you can't add to your graphs events such as when a customer survey was redesigned, or when a new product stream was launched, or when the ordering process was streamlined, or when you moved from ad hoc purchasing to formal supplier agreements. These events are markers in history that usually correlate to a sudden or gradual change in the level of your performance results. For example, after the new product was launched you may well have seen revenue start to climb, or after the ordering process was streamlined you probably saw a sudden "step change" reduction in the order cycle time. Adding events to your performance graphs can help you and others interpret why specific changes in the level of performance occurred. But of course, how do you know which events to put on your graphs? A little bit more context...

the context of causation

Some of our strategies or initiatives will have an impact on our performance results, and others will not (even if we intended them to). Some trends or patterns of behaviour in the market or in our industry or societies will also have an impact on our performance results, and others will not (even if you expected them to). The only factors that will influence our performance results are those that have a causal relationship with those results. For example, educating customers on how to use our order form may have been born from the hope of reducing errors that hold up the ordering process, but in reality had little impact. However, redesigning the order form to make it faster and more intuitive from our customers' point of view had the impact we intended, because it addressed one of the root causes of errors on orders. Correlations between the implementation progress of your initiatives and the changing results in your performance measures can give you clues about which initiatives are working, and which may not be. And that brings us to the last type of context for performance measures that we'll discuss here...

the context of contrast

To shed some light on why certain initiatives work and others do not, it can help to analyse your data a bit deeper to find where the successful initiatives failed, and where the failing initiatives succeeded. These investigations can uncover additional factors associated with the performance results you are getting, and thus give you more clues to increase the power you have over managing that performance. For example, perhaps the education of customers in how to use our original order form worked mostly for those customers ordering technical products: they are technically minded, and found the order form easy to understand once they had been shown how it worked. But the majority of customers are not technically minded - most of our products are designed to make things easy for our customers, and these products attract customers that like things to be made easy for them. No education in the world was ever going to change their mindset.

in conclusion

These are not the only forms of context you can surround your performance measures with, but I do hope they give you something to play with. And I'm very curious about your own ideas and examples of how you've used context to improve the rigour of your performance analysis and decision making. If you do, please send me an email!

#32 Are Your Measures Informing or Deforming Your Decisions?

2007-06-17T02:25:00.000+10:00

For some, the sound of the word "statistics" coming from someone's mouth impulses them to run far and fast in the opposite direction. Our society seems to be almost proudly fearful of anything to do with mathematics, especially its branch of statistical analysis. "Lies, damn lies and statistics!" they scoff. What a horrible foundation on which to build adequately informed decision-making. Particularly with performance measurement, due to its potent influence on business decision-making, the way performance data is often analysed produces a grotesque parody of useful and usable information.

What can we do about this? One simple step in a good direction is to be aware of how different types of performance data analysis relate to the different types of performance management questions we ask. How does your organisation compare to the following suggestions?

business question 1: Is this performance result getting better (or worse, or not changing)?

For example:

is the accuracy of our bills improving?

is cycle time reducing?

is our cost of inventory stabilising?

are we getting better at retaining our best staff?

Analyses that can help answer these questions:

use trend over time graphical methods, such as line charts, run charts and statistical process control charts

avoid linear trend lines - in the vast majority of cases they explain very little of the pattern of change, and rarely is anything so simple it can be explained by a straight line

avoid tables comparing this month to last month - two points of data are totally inadequate in accurately evaluating change, due to a natural phenomenon known as variation - you need a good time series of around 20 points

business question 2: What are the main reasons this result is happening?

For example:

what are the main types of errors on bills?

what are the 20% of problems that cause 80% of our cycle time blowouts?

what is the total cost of each of the slowest moving inventory items?

what are the reasons that staff leave us?

Analyses that can help answer these questions:

use a bar chart of causal factors or reasons

even better is to use a Pareto chart (most visually effective when the bars are horizontal, instead of vertical and ordered from largest to smallest)

avoid tables of numbers as they are much more difficult to interpret than the visual impact of charts

avoid pie charts, as they are designed to compare a part with it's whole, and encourage misleading visual comparisons between the slices

business question 3: Is this result really related to that result?

For example:

is the extent of errors on bills related to the workload of our billing staff?

what is the best lead indicator of cycle time: total throughput, customer ordering lead time or supplier on-time performance?

if we increase the percentage of inventory items on auto-order to 60%, what percentage change could we expect in inventory costs as a result?

to what extent is employee turnover related to employee job satisfaction?

Analyses that can help answer these questions:

use regression analysis to get a quantitative model of the relationship between two measures, also useful to predict changes in one measure as a result of changes in another

use correlation analysis to get a measure of the strength of the relationship between the two results (called a correlation coefficient)

more simply, use a scatter plot of the two measures

sometimes okay to use a line chart of the two measures in time series and visually look for direct or lagged correlations in their patterns over time

avoid relying on gut feel and hearsay - any relationship needs to be objectively tested

business question 4: How big is this result for us, compared to the same result for them?

For example:

what is our bill accuracy level like across each of our product streams?

how does our cycle time compare to our competitors?

what is the relative cost of each category of inventory we hold?

is employee retention different between our departments?

Analyses that can help answer these questions:

use vertical bar charts, with the bars ordered in a way that is logical to the classes or categories you are comparing

avoid pie charts and tables, for the same reasons listed above

So there are at least two things you could start doing now to ensure that you are analysing your performance data in the most useful way:

1) work out which types of business questions you're asking (or need to ask)

2) choose the types of analyses that are best suited to answering those questions

It's far more worthwhile to use the simplest analysis method well, than to use a more sophisticated analysis method poorly.

#31 Performance Conversations

2007-06-17T02:22:00.000+10:00

We all know that having performance measures isn't worth the paper they are printed on if they don't get used to drive performance improvement. But for many, there is a great leap from having performance reports to actually putting them to good use, and that's because the conversations that are had to interpret performance results don't have the conviction they really need. Here are some samples of conversation snippets I have heard over many years of performance measurement coaching and consulting. Which level does your organisation best fit?

signs of a poor performance conversation

Some of the signs that your performance conversations are next to a complete waste of time include hearing statements like these being made by those participating in the conversation:

"I think this project was a great success because last week I was talking to so-and-so and they said..."

"Well we did have to face quite a few challenges, and in the context of that, we've actually done quite well."

"You can see that this initiative was worthwhile because our annual customer survey result shows improvement."

"No we didn't really have any good measures, because it really isn't possible to measure this."

"Yes but our goals just weren't the right ones, and we had to change tact along the way."

"You know, it's a very difficult goal we set, and we really didn't think we'd achieve it. But we're heading in the right direction."

"Everyone else is suffering in this volatile environment. We're not doing as badly as some."

"I just know it worked. I know the area and I can see things changing."

signs of a constructive performance conversation

Your performance conversations are faring a little better than those above when you can hear a few comments like these being made (and listened to):

"Okay, so let's take a look at the measures we designed for this project and see what they're doing."

"A few people did complain about the change in our process, but when you look at the survey results, the majority of people have said they like it better than before."

"These measures are showing some improvement over the past few months, and we believe it's due to our new initiative."

"We intended for costs to reduce, but we just didn't seem to get that to happen. Maybe we should check that the project was implemented properly..."

"In hindsight, we set way too difficult a target without really knowing what it would take to reach it. And not reaching it caused some cynicism too."

"So it seems that we set some goals that were really outside our control to achieve, so let's focus on the parts we can still influence."

signs of a strong performance conversation

And great performance conversations show signs of people talking like this:

"Four of our project outcome measures are showing statistically significant improvement, and the other three are showing no significant change at all. The project lead measures suggest this is because our training is working, but we still haven't managed to make the system available to everyone."

"Just as our lead indicators suggested a few months ago, we are starting to see performance stabilise around our targeted level."

"The pilot areas where we introduced the new work procedure are consistently showing 23% higher productivity than in the areas where we are still using the old procedure."

"Yes we are getting higher satisfaction ratings, but even our competitors are too, and they haven't tried to improve anything. Correlation analysis shows it has more to do with recent media coverage in our industry than with our new incentive scheme."

"No, there is very little evidence to suggest this project is working. Costs haven't changed, and cycle time is still averaging around 10 days. It's time to hit the pause button, find out why it isn't working and either pull the pin or redesign the project."

"It was unfortunate that fuel prices rose so suddenly when we were trying to reduce costs, but we re-scoped the project to also find ways to reduce our mileage and dependence on fuel."

"So we just identified an important question we can't answer with the data we have. What information will help us answer this, and how can we get it?"

Welcome to the mezhermnt Handy Hints blog!

2007-06-17T02:14:00.001+10:00

G'day, and welcome to the official blog for the archives of the free ezine "mezhermnt Handy Hints". It's my twice-monthly subscriber-based email newsletter about business performance measurement - specifically featuring tips to make it easy, rigourous and even fun!

First things first - how do you pronounce "mezhermnt"? Well, it was my brainwave years ago (I might be starting to regret it now) of the phonetic spelling for "measurement"!

I hope you enjoy reading these short articles. I'm trying to keep them conversational and practical, so be sure you give me lots of feedback about the degree to which I am actually achieving that goal!

Smiles,

Stacey Barr
the Performance Measure Specialist
http://www.staceybarr.com
http://www.mezhermnt.com