Message From The Skies

What’s this?

Lightning control chart - binary comparison version

These strange lights were seen above my house last night. (Well, sort of…)

It’s actually a photo of a lightning bolt stretching across the sky from left to right, as seen through the eyes of someone who only understands binary comparisons.

You see, I’ve blogged about binary comparisons before. I grumble that comparing just two isolated numbers (e.g. ‘this year compared to last year’) is misleading. I moan that it’s impossible to say whether the numbers are going up, going down, or staying the same. I gripe that when used as a method for assessing performance, crime statistics, or anything else, the use of binary comparisons is a big fat waste of time.

Look at the unedited version of the lightning bolt to see why…

Lightning control chart

Beautiful, isn’t it?

When I saw this, I thought of a control chart. The top and bottom of the frame would be the control limits. The lightning bolt would be the zig zags of a data series over time. Sad, aren’t I?

But there’s a point. Viewing data in this way gives it history; context. You can see what’s ‘normal’. It’s possible to identify whether the overall picture suggests there are any trends or unusual spikes, or, as in this case, the numbers are consistent over time.

Simply taking any two points from this lightning bolt and comparing them against each other will give a different result. Look –

Lightning control chart with arrows

See? Depending on which two points you pick, you get different directions of travel. (For more on control charts, have a look at the ‘measures’ section of this blog post). Therefore, if you’re still using numeric tables stuffed full of binary comparisons, such as the one below, then effectively what you’re doing is trying to see the whole lightning bolt by looking at those two isolated dots in the picture at the top of this post.

Binary comparison crime table

As you can see, this table is constructed of lots of little snapshots of lightning. It’s impossible to ascertain anything at all about crime trends. What makes it worse is the use of percentages to emphasise the binary comparisons – even more so when really small numbers are compared. That’s why you get a 100% reduction in one crime type and a 400% increase in another.

It astounds me that these things are still used by anybody. They lend themselves to misguided assumptions and conclusions about stuff which isn’t actually there. This means that any decision based on a binary comparison is pure guesswork. Because of normal variation in any data set, the chances of picking two isolated points and them actually being part of a trend is miniscule.

So, here’s the deal. If you stop comparing the two tiny pieces of lightning, you will see the total picture, thereby leading to informed decision making, evidence-based responses, and better results all round. It will also prevent knee-jerk reactions, reduce costs and waste, plus help you understand what is actually happening out there.

Binary comparisons vs control charts – one is blunt, misleading and completely useless; the other is simple, practical, and the most reliable method of understanding data known about since the 1920s. What’s not to like?

Now, here’s that picture again…

Lightning control chart

Don’t mess with nature!

About InspGuilfoyle

I am a serving Police Inspector and systems thinker. I am passionate about doing the right thing in policing. I dislike numerical targets and unnecessary bureaucracy.
This entry was posted in Systems thinking and tagged , , , , , , , . Bookmark the permalink.

8 Responses to Message From The Skies

  1. geoff elliott says:

    ……Binary comparisons vs control charts – one is blunt, misleading and completely useless; the other is simple, practical, and the most reliable method of understanding data known about since the 1920s.

    What’s not to like: lots and lots. Frequentist stats does not work when applied to a service organisation.

  2. Thanks for your comment Geoff. Here’s a suggestion – Rather than popping up on my comments pages (and emailing me) just to tell me I’m wrong, why not share your perspective so we can all learn from you? I’d be really keen to understand why you believe what I’ve written about wouldn’t work in service organisations. Surely you’re not endorsing binary comparisons? Cheers, Simon.

  3. Robert long says:

    Aren’t Control charts only useful if the data is normally distributed or ratehr if you assume data is normally distributed? If this is the case, then since most real life data dosn’t appear to be normally distributed, a control chart could be misleading too.

    Not that I am a fan of Binary Comparisons, though gold love them if they say the right thing!

    • Control charts are only useful if the process in question is stable. Whether or not the data are normally distributed will be visible when they are placed into a control chart. Chicken / egg thing I guess, although I’ve found that lots of real life data do tend to be normally distributed, meaning control charts can be really useful.

  4. Pingback: Find The Treasure | InspGuilfoyle

  5. Pingback: Comfort Blanket | InspGuilfoyle

  6. Pingback: Freeze Frame | InspGuilfoyle

  7. Pingback: What have the Romans ever done for us!! | Squire to the Giants

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s