Right Measures, Measured Right

…is a phrase I often use when talking about measuring stuff in performance systems. Although I deride arbitrary numerical targets at every turn, I genuinely believe it’s essential to measure things, as long as they are the right things. So, first of all, that begs the question, “What are the ‘right things’?”

Fair question, sometimes followed by, “…and how do you measure them ‘right’?” (Okay, so that should say ‘properly’, not ‘right’ – sorry Grammar Police, I just think it’s catchier). Anyway, another fair question.

So let me try and answer.

First of all, the right measures are the ones derived from purpose. Purpose is what the system is there to achieve.

cockpit dials - labels

For instance, here’s some dials in a cockpit. I’ve marked what a few of them tell the pilot. You can see that it’s pretty useful stuff – how fast the plane is flying, what direction it’s heading, how the oil pressure is doing, and so on. All linked to purpose – in this case to fly passengers safely to their destination. The method by which the information is relayed to the pilot is useful too – the dials are capable of indicating change in real time, enabling our pilot to respond accordingly and make adjustments if necessary.

Right measures, measured right.

Therefore, the right measures, presented in a meaningful and readable format, enable the user to understand how the system is performing, as well as identify what needs to be done to attain purpose successfully.

Now, what about if you use the wrong measures?

cockpit dials - silly labels

Well here’s the same set of dials, this time configured to measure the wrong things. Ok so it’s a pretty daft example, but it makes the point. Using the wrong measures means it’s impossible to establish if you’re achieving purpose. There’s no point counting the wrong things, just because they’re easy to count. Furthermore, if you’re using the wrong measures, don’t expect them to encourage the right behaviours.

Next, what about if you measure the right things, but in the wrong way? Well, this happens:

cockpit dials - binary version

Here’s the ‘binary comparison’ version of the cockpit dials. All the same measures as in the original configuration, but presented in a way that can’t tell you anything useful. For more on binary comparisons click here, or just talk to me for more than five minutes.

Finally, for the benefit of those people who interpret my ‘no targets’ message as ‘no performance management’, have a read of this blog which explains the distinction between targets, measures and priorities. Relax – I’m not some anti-measurement anarchist.

This is what would happen if we didn’t measure anything:

cockpit dials - blank

I wouldn’t want to be a passenger on that plane.

There’s nothing pink and fluffy about doing measurement properly. It’s just better use of information. Much better than trying to guess whether something’s going up or down because you’ve compared it to last month. And taking the targets out means that people can focus on purpose, rather than the targets.

Enjoy the flight.

About these ads

About InspGuilfoyle

I am a serving Police Inspector and systems thinker. I am passionate about doing the right thing in policing. I have a big problem with numerical targets, unnecessary bureaucracy, and anything else that stops police officers from providing the best possible service. I believe that by adopting a systems approach, policing can be transformed beyond the wildest expectations of many.
This entry was posted in Systems thinking and tagged , , , , , , . Bookmark the permalink.

23 Responses to Right Measures, Measured Right

  1. Roger Flint says:

    Brilliant

  2. Rebecca says:

    So, what if you can’t define what the system is intended to achieve, but you ‘know’ it’s the right thing to do?

  3. Pingback: Right Measures, Measured Right | Policing news ...

  4. Gav Beresford says:

    Can’t argue with that!

    Don’t forget – we could also go and look at how others fly their planes and in doing so end up in completely the wrong place, quite literally in this example!

  5. patently says:

    First of all, the right measures are the ones derived from purpose […] There’s no point counting the wrong things, just because they’re easy to count.

    Couldn’t agree more. Now, are you going to tell the Safety Camera Partnerships about the difference between speed and safety, or should I? ;-)

  6. Eileen says:

    Really like the way you have presented your argument – very impactful. I spend most of my time helping health and social care systems through their improvement journeys. There tends to be a love-hate relationship with measurement for improvement. Getting the right balance feels like the holy grail!

  7. smbrookes says:

    An excellent way of getting the point across. My favourite quotation is from Albert Einstein which was written on a note attached to his office wall (not a post-it note!) “Count what counts and not what can be counted”. Well done Simon. Your illustrations show this very well!

  8. Chris Close says:

    That was ok Simon so long as the process does not outweigh the value of the outcome

  9. Pingback: Right Measures, Measured Right | Quality of Hea...

  10. Tim says:

    Excellent stuff !

  11. Daniel Rolfe says:

    Hi Inspector and all.

    I love the stuff outlined in this blog. You have a real knack of getting the message across and I have also found your book very useful in learning more about systems thinking and measures.

    I have one little bug in my head though. It’s about all numerical targets being arbitrary. I was hoping you might have a view on this.

    At the start of a Premier League season Liverpool FC manager Brendan Rogers targets a points haul of 95. Brendan knows that this number is the highest ever points tally in Premier League history and achieving it is highly likely to make Liverpool champions.

    My questions are: Is this target arbitrary since it’s based on evidence? Even if it is arbitrary, is it dysfunctional?

    It’s the only scenario I can think of that causes me to question the logic of all numerical targets being arbitrary and having the potential to be dysfunctional.

    Don’t worry. I’m not going to start pulling targets out of the sky, it’s this sole example that I can’t quite resolve in my head.

    • Tim says:

      So Brendan Rogers sets a target of 95 points. Is it based on evidence? What evidence is the achievement of this target based on? Last year’s performance? Don’t think so.
      How will the target of 95 points influence the players’ performance?
      If they’re going into a game with Man City – will they think – “ahh this is a game we don’t have to win – we can drop points and still get 95 points, as long as we beat Norwich and draw with Man Utd.” If they do this – it’s dysfunctional because every Manager/player/fan knows that you need to go out and try to win every game. Teams that target a draw often come to grief.
      If the players don’t think like this and go out to try and win every game then the points target is irrelevant.
      Either way the 95 points target is at best a guess or a hope. If it affects the method then it will be dysfunctional – if it doesn’t then it’s irrelevant.

      That’s how I see it.

    • Hi Daniel,
      It’s a fair question. A couple of things spring to mind in reply. One is ‘why is aiming for 95 points better than aiming to win every game?’ Aiming to be the best possible – in this case by trying to win every game – is equivalent to aiming for 100% / aiming for perfection, which is where the manager should aspire towards. The purpose of the club is to be successful so why would it set the bar any lower?

      Also, whatever has been achieved in the past by whatever team will have been done under slightly different system conditions – whilst it may have been the top flight football division at the time (under whatever name), whoever has gotten 95 points previously has done so with different players, teams and under different circumstances than this season. so there can be no equitable comparison to the maximum points scored then against what could be possible this season. Thi smakes the selection of any point below 100% arbitrary. (I see Tim has suggested some of the dysfunctions that might end up affecting the balance of the table so I won’t cover old ground there).

      The team might come top of the table with 88 points – would they care if they didn’t get the 95? Of course not. How about if they got 95 and hit the target but came 4th because others scored more? Look at this for a real life example of hitting the target but failing>>> http://www.northamptonchron.co.uk/sport/champions-league-final-loss-harder-to-take-than-relegation-for-barnet-boss-davids-1-5042759

      For more, have a look at Nick Bailey’s comment on ‘Sopt the Difference’ http://inspguilfoyle.wordpress.com/2013/01/31/spot-the-difference/ about a football team.

      Hope this makes sense – cheers for reading the blog.

      All the best,

      Simon

  12. Julie Guy says:

    Sounds like you would like the Improvement Model, are you familiar with it?

      • Julie Guy says:

        Ahh now there’s a question. I’m new to it and as I understand it there are many different models but they all have the same philosophy’ish’ would Plan, Do, Study, Act answer which one?

      • Deming’s PDSA. Quality. Also good (and virtually the same) is Joiner’s PDCA model. Deming purists may lynch me for saying this but my favourite is John Seddon’s Check-Plan-Do cycle. As long as they are applied with due understanding of the system (and a preparedness to challenge management norms) they can all be extremely useful.

  13. Pingback: Stick Child’s Guide to Systems Thinking | InspGuilfoyle

  14. Pingback: Three Different Things | InspGuilfoyle

  15. Pingback: in which I consider the safety of airlines | theputneydebates

  16. Pingback: Find The Treasure | InspGuilfoyle

  17. Pingback: Avoidable Harm | InspGuilfoyle

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s