Where Are The Stick People?

It’s been quite a few months since I last published a blog post and I’m often asked what’s happened to Stick Child and his friends. Well, they’re still about, but I have to admit that having written over 100 blog posts now I’m finding it hard to find new ways to communicate the same simple messages. Maybe I’m getting lazy, but more recently I’ve been directing interested parties towards previous posts, or the ‘useful articles’ that cover some of the main limitations of numerical targets, league tables and binary comparisons.

I guess if something outrageously new occurs to me, I might well feel compelled to blog about it, but I know I’ve been sounding like a broken record for a long time now. As Stick Child’s original mission was to explain basic performance management concepts to the under-10s, I suppose if you haven’t got it by now then you might as well give up anyway.

Also, you may remember in mid-2015, the then Home Secretary (and now Prime Minister), Theresa May, commissioned Chief Superintendent Irene Curtis (President of the Superintendents Association of England and Wales at the time) to carry out a far-reaching review into the use of targets in UK policing. The final report was published several months ago (December 2015) and contains some practical, ready-to-implement recommendations for Chief Constables, Police and Crime Commissioners, Her Majesty’s Inspectorate of Constabulary (HMIC) and the Home Office.

Mrs. Curtis’s review is probably the closest thing to a ‘Useful Guide for Better Police Performance Management’ that anyone could ever hope for. Crucially, it recognises that the dysfunctional behaviour long associated with the use of numerical targets is not the only major problem within UK police performance management – it also highlights, for example, the limitations of simplistic binary comparisons, the dreaded ‘reds and greens’ and the meaningless data presentation formats that lead to entirely predictable knee-jerk management reactions.

The report emphasises the importance of culture, people, leadership, trust, autonomy and the need for a different mindset towards performance measurement and management. It proposes simple and credible alternatives, as well as a straightforward set of guiding principles for effective police performance measurement that could be easily adopted tomorrow. It even provides case studies featuring current good practice in a handful of UK police forces. I strongly recommend that anyone who cares about policing takes the time to read it. The report can be found here.

There’s been a lot of support for the review – some forces (and senior individuals) have actively embraced its recommendations, yet several months after its publication, progress seems slow and the national picture is haphazard, to say the least. What’s needed now is some tangible and coordinated action on the part of the College of Policing and other recommendation owners, in actively promoting and implementing the recommendations – like I’ve said, a lot of them are practically ready to go now. It just needs the will to make it happen.

What’s more, none of the proposed alternatives or guiding principles endorsed within the review document are the slightest bit contentious or even particularly complicated. Only those who remain wedded to numerical targets, fixated on simplistic league tables, or obsessed with meaningless binary comparisons might struggle with the content. Take a look for yourself and you’ll wonder, “Seriously, what’s not to like!?”

Neither the Stick People, nor the review, are going away. Ultimately, either the review’s recommendations will be implemented as part of a coherent, coordinated national programme  (alongside the necessary  genuine endorsement, visible leadership and active support from the top), or they’ll continue to be taken up in a piecemeal fashion by those leaders bold enough to do so, simply because it’s the right thing to do. It’d be a shame if something so important was left to a handful of individuals who happen to care.

Fundamentally, Mrs Curtis’s review represents an opportunity to completely transform UK police performance management within a very short timeframe – it would be criminal to squander it.

Right, the Stick People have stick people stuff to attend to and I’m going to lie down in a darkened room.

Anyway, here’s a few blog posts to be getting on with…

Three Different Things – The difference between targets, measures and priorities.

Right Measures, Measured Right – Why, and how, to measure stuff properly.

Avoidable Harm – One of my exasperated attempts at trying to explain why numerical targets are arbitrary.

The Wrong Conversation – Why numerical targets are completely the wrong starting point for trying to understand performance.

Nonsense – Total sense of humour failure about the uselessness of binary comparisons.

It’s Criminal – Why crime rates aren’t even a police performance measure anyway.

…and there’s plenty more where they came from. Happy reading!

 

 

The Professionals

Imagine a police force that handed out firearms to its officers without training them.

Or one that chose to opt out of established good practice for investigating serious crime.

Or one that refused to work with partner agencies because it didn’t feel like it.

Or a force that allowed untrained officers to pursue criminals at high speed in fast cars.

The consequences would be predictable and these sorts of irresponsible actions would quite rightly attract criticism and incredulity.

So why is it any different for performance management? How can individuals be expected to use performance information wisely when they don’t fully understand it because they’ve never been trained? How can some managers be allowed to ignore the widespread evidence that dysfunction is likely to result from certain practices, such as the use of numerical targets, binary comparisons or league tables? Why are untrained people handed such responsibility and just expected to ask the right questions about the right things?

It’s time to professionalise police performance management.

Leaders at all levels must be equipped to ask different questions, using legitimate forms of performance information as a starting point for understanding the whole system – they also need to stop asking for products that are fundamentally illegitimate, simply because they are comfortable with the format.

Decision makers require trusted expert analysts to act as specialist tactical advisors, providing them with fit-for-purpose, contextualised performance information. It’s no different to calling in expertise from hostage negotiators, search specialists, firearms or public order tac advisors.

The police service needs clear national guidance and bespoke training programmes that provide leaders with a real understanding of performance measurement and management, along with its implications. We need to completely reframe and reorient the performance conversation from one that fixates on simplistic concerns about whether something appears to be ‘good’ or ‘bad’, to one that uses performance information as an evidence base for making good decisions, understanding, learning and improving. We need to stop trying to out-do each other, blaming individuals and using the wrong mechanisms to ask the wrong questions about the wrong things.

Finally, dare I say it, but there’s probably even a place for HMIC, ensuring that forces don’t flagrantly disregard good performance practices simply because they feel like opting out – I’m sure there would be outcry if a force chose to ignore its responsibilities to protect vulnerable children, or let untrained officers loose with firearms, and quite rightly so.

Managing complex investigations, running firearms operations and driving fast cars requires high levels of training, professionalism, and responsibility.

It shouldn’t be any different for performance management.

Chain Reaction

No, nothing to do with an old Diana Ross track, but the title of a presentation I gave at the UK’s first ever TEDx policing event in Leeds yesterday – #OldBillRebuilt, ably organised by Gareth Stubbs.

Hopefully, the footage will be uploaded to the TEDx website pretty soon, so you can cringe at my bad jokes and unpolished delivery.

In the meantime, these are the slides, along with a running commentary…

Here we have a few of the Stick People, including Stick Cop, Stick Child (next to Stick Cop) and a few of his friends.

This is the title page, with a stock photo of a chain and a guest appearance from Stick Teacher. It would be wise to do as he says.

Okay, so now we have the first challenge… this is a bowl of my home made Vietnamese beef broth (Pho). The Stick Person likes the look of it and wants to have it for his lunch. Now all he needs to do is decide what the ideal implement would be to get the soup out of the bowl and into his tummy.

Right, so here’s a range of options. He could use a fork, but although you might hear some senior managers say, “I’ve always used a fork to eat soup and it’s got me where I am today!” it’s probably not a very good option as it’s likely to result in an angry Stick Person.

Then there’s the knife. Indeed, many would argue that a knife would be a perfectly legitimate option as you could balance a few drops of soup on the flat side. Some would also say there’s nothing wrong with using a knife to lift droplets of soup from the bowl in an excruciatingly slow manner, as people should be free to use whichever implements they want to when it comes to eating soup. Nevertheless, the result = angry Stick Person.

Then of course we have the spoon – designed for precisely this type of situation, fit for purpose, easy to use, gets the job done. This is likely to result in an empty bowl and a happy Stick Person with a full tummy.

Now let’s see how this applies to the world of performance management. Wake up! I know it might not seem to be the most enthralling subject matter, but stay with me – it’s important.

Okay, so here’s what happens when binary comparisons, league tables and numerical targets are used as a means of presenting performance information. What’s more, this type of chain reaction is highly predictable. It’s science, kids.

Here’s some alternatives. Control charts and contextualised peer comparisons (stuff which provides statistically meaningful information about comparative peer performance) aren’t the be-all-and-end-all, but at least they aren’t rubbish, like the things in the previous slide. Binary comparisons, league tables and numerical targets are the knives and forks of the performance management soup-eating world; they cause unsightly dribbles down your best T-shirt and leave your tasty soup to go cold.

However, when people use alternative types of performance information, they tend to interpret the data accurately, make better decisions, don’t knee jerk about stuff that probably isn’t even happening, or bully people over meaningless data. This is clearly better for everyone. Cue a happy Stick Person.

Where do you dream up this stuff from Simon? Well, despite the crude drawings, flippant commentary and simplistic headlines, it’s all from proper, rigorous and extensive research. ‘Evidence-based policing’, if you will. Here’s a couple of academic articles of mine that have been published recently. They are crammed full of impenetrable stats and clever tests. If you have difficulty sleeping or just refuse to accept what the slides with the red and green arrows are telling you, then you should read them.

And there you have it…Back to the soup!

THE END.

Limited edition bonus material: As a special extra, here’s some photos from the event; one of speakers and audience members (you can play ‘spot the twitter user’ if you like) and another featuring me looking shifty in a variety of poses and bad light.

Freeze Frame

Watching the TV programme ‘The Force’ last night (which follows the work of officers from Greater Manchester Police), I thought of yet another analogy to help explain why binary comparisons are rubbish. (Yes, I know I go on about this, but there are still people out there who appear to be ‘hard of learning’ when it comes to this subject).

Anyway, the episode featured the policing of a protest by the English Defence League, including sequences from a state-of-the-art control room where the officer in charge was able to watch events unfolding on banks of CCTV screens. This meant he was able to make decisions dynamically in response to changing events and fresh information, as it came in.

Using the inevitable #StickPeople to contrive a tenuous link to the use of binary comparisons, here we can see the police boss running an operation to prevent disorder involving the fringe ‘S.D.L.’ (Stick Defence League) – an angry and directionless group who sometimes cause problems at their own events for no apparent reason.*

The boss uses the live footage from the CCTV screens, radio messages and other incoming information to make decisions about what to do, and the event passes off peacefully. Everyone’s happy. This is because all he’s doing is the equivalent of using meaningful and multifaceted performance information to understand what’s occurring over time, so informed decisions can be made about how to respond (or not respond).

Contrast this with the situation below, where the real time information is switched off and all he has to go on are occasional isolated ‘freeze frames’.

It’s pretty obvious really, but the latter case is equivalent to using binary comparisons, where isolated numbers are compared to ‘the same period last year’ etc.

Why anyone still does this is beyond me.

* Disclaimer: This blog is just a bit of light-hearted fun. The ‘SDL’ are entirely fictitious and even if they were weren’t, they would have the right to protest peacefully. Any similarity to anything else, ever, is purely coincidental. And binary comparisons are still rubbish.

 

Short Circuit

This is Stick Boss. Stick Boss is basically a good guy who cares about his organisation and wants the best for people. Unfortunately, he has a problem with his brain. This means that just as he’s on the cusp of understanding how to make best use of his performance information, his brain short circuits and he reverts to the use of arbitrary numerical targets (and other associated abominations).

Let’s have a look inside his head to see what goes on in there…

As you can see, the content of the different segments of his brain is pretty standard. Unfortunately, the ‘numerical targets’ quadrant interferes with his thought processes in certain circumstances. Take the following example…

Stick Boss is bright enough to understand that different parts of his organisational system influence each other, helping or hindering service delivery. Even some stuff that wouldn’t be obvious at first can affect how well the front line performs. Therefore, Stick Boss is keen to understand where the opportunities for improvement lie, so he ensures his performance measurement system reaches into these dark backlots to extract useful information that can aid his decision making.

This is good. He’s got a clear understanding of his organisation’s purpose and recognises the need to have access to performance information that measures the right things.

But then the brain defect kicks in.

Instead of using the right measures in the right way, the short circuit in his brain overrides the need to present performance information in a usable, contextualised format, and supplants it with an ill-conceived reflex that results in his potentially useful data being corrupted by arbitrary numerical targets and binary comparisons. What a missed opportunity.

Like I said, Stick Boss isn’t a bad guy, but his brain problem prevents him from realising the damage he is causing by relying on fundamentally illegitimate performance practices. So, if you know people like him, please help.

Maths Class

Here’s Stick Teacher. Today Stick Teacher is going to run through a few sums with the Stick Children. Adults might find his case studies useful too. Stick Teacher is going to demonstrate how to save money whilst providing a better service – what’s not to like? 🙂

To do this, he’ll use the example of a generic public sector call centre (*police / ambulance / fire / tax office / housing department / other). *Delete as applicable.

The idea that these call centres save money seems to be based on the premise that if you put a lot of people in a central location and give them a specific function that forms part of the overall service provided, it’s cheaper.

Let’s try and understand how this concept works, using characters that Stick Teacher has invented to help explain things to the Stick Children…

Here’s Employee 1. Employee 1 is an enthusiastic call taker and a hard worker, who always assists the public the very best he can with his level of training and experience.

And this is Employee 2. She is also enthusiastic and hard-working, but more senior, more highly trained and experienced; plus, she is empowered by the organisation to make decisions about stuff.

Here we go then…

Case Study: Option One

For this example, Employee 1 answers a call, asks basic details and creates an electronic record. He is paid 10 Stick Coins per hour (this is the currency in Stick Land) and spends 15 minutes dealing with the call.

Time elapsed so far = 15 minutes.

Total cost so far = 2.50 Stick Coins.

Employee 1 then forwards the electronic record to Employee 2; being more senior, more highly trained and experienced, she is paid 20 Stick Coins per hour. Her role is to review the electronic records, identify any additional actions and make a decision about what to do.

On this occasion, she identifies that Employee 1 has covered most of the basics, but thinks it would be helpful to ask a few more questions and do some background checks before a resource is allocated to deal with the case, so she details these requirements on the electronic record and returns it to Employee 1 to complete.

She spends 15 minutes doing this (costing a further 5 Stick Coins).

Time elapsed so far = 30 minutes.

Total cost so far = 7.50 Stick Coins.

Employee 1 then completes these tasks and sends the case to the appropriate resource. This takes another 15 minutes, costing another 2.50 Stick Coins.

Total time elapsed = 45 minutes

Call rate per hour = 1.33

Total cost = 10 Stick Coins

So, that’s the traditional call centre model then.

Case Study: Option Two

An alternative approach would be put Employee 2 at the point of contact with service users, where she can maximise her skills and experience to do a more thorough job from the outset, thereby negating the requirement to pass the job backwards and forwards for review and remedial actions. Due to her enhanced ability, she is able to deal with the entire case in 15 minutes.

Total time elapsed = 15 minutes

Call rate per hour = 4

Total cost = 5 Stick Coins

So, although it costs twice as much to employ her, the job gets done more quickly and effectively. Oh, and the whole process costs half as much as in Option One.

Case Study: Option Three

But perhaps your organisation wouldn’t want to use a raft of ‘Employee 2s’ at a higher rate of pay, so a third option could be to employ Employee 3, who is just as enthusiastic and hard-working as his colleagues. Employee 3 is quite capable of handling 95% of calls without assistance; he is more highly trained than Employee 1, but isn’t trained to the highest level of specialism, like Employee 2.  This means he is paid 16 Stick Coins per hour. He is able to deal with most cases to the same level as Employee 2, and at the same rate.  Therefore, he deals with this call in 15 minutes, costing 4 Stick Coins.

Total time elapsed = 15 minutes

Call rate per hour = 4

Total cost = 4 Stick Coins

On those few occasions where Employee 2’s specialist training is required to make decisions on the most complex cases, this model would allow for a small number of such experts to be available, commensurate with the demand for their enhanced skills.

So, this model is also faster and costs much less than the traditional model; the only consideration is there will be a handful of occasions where some calls need to be passed to Employee 2 to complete. If the cost / benefit considerations of this model are acceptable, then this configuration could be the most cost-effective of all three.

Stick Teacher’s Conclusions

Therefore, the result of today’s lesson for the Stick Children is that conventional wisdom about things like centralised functions is not always that wise. Whichever way you look at it, Option One is actually the slowest and most expensive to operate; Option Two is the ideal model from the caller’s perspective, and Option Three is probably the most cost-effective and suitable configuration for the real world.

The illusion of savings under Option One comes about because of the focus on cost-per-operator, without understanding end-to-end flow or whether purpose is achieved. It builds in waste, unnecessary handovers, disempowers staff, creates delays and COSTS MORE, whilst providing a WORSE service!

Therefore, maybe more people should try one of the other options instead.

If the Stick Children can understand this, so can you.

Detox

Whilst flicking through TV channels recently, I came across a consumer advice programme, where the reporter was researching the effectiveness of self-administered ‘detox’ products. The TV crew followed her for three days as she consumed nothing but ‘specially formulated’ fruit and vegetable juice drinks of various colours. At the end of the experiment, she visited a nutrition expert to discuss the results.

I have recreated the conversation that followed, with the help of my Stick People…

The reporter seemed surprised…

The expert’s assessment wasn’t that well-received…

The exchange made me chuckle, as it reminded me of this sort of thing…

Standby for the chap on the left to say, “Thanks Stick Child – that’s really helpful. I know what I must do!”

Or perhaps…

Oh dear. As writer Aldous Huxley said in 1927:

“Facts do not cease to exist because they are ignored”.

 

Comfort Blanket

When Stick Child was a bit younger, he had a little comfort blanket. Here he is with it, dreaming about smashing up some arbitrary numerical targets. (That’s supposed to be the moon in the window by the way – not a banana).

Stick Child believed that his comfort blanket had magical powers – it kept monsters away at night and helped him sleep soundly. It felt soft and warm, so if he was ever sad he would hold it against the side of his face. It was familiar and reassuring.

Some adults adopt a similar approach.

Despite evidence that binary comparisons are incapable of telling us anything about trends or trajectories, are prone to causing unwarranted assumptions, consistently impair decision making and lead to unnecessary action as a direct result, some adults still cling to them like a comfort blanket. (The same applies to numerical targets and league tables).

In case of any confusion, here’s a simple guide:

So, to conclude: binary comparisons do not have magical powers and any ‘reassurance’ they appear to provide is false.

Comfort blankets can be great for children, but adults need to let go.

The Wrong Conversation

Stick Child has been getting increasingly irritated by the slack methods some adults use to present information about really important stuff, such as how long it takes for patients to be seen in Accident and Emergency (A&E) departments. Many a time recently he’s had to do a facepalm at the way hospitals and other institutions are judged and compared against each other, supposedly to inform the public about how well each is performing.

The problem is that the starting point of the conversation – the frame by which performance is judged – is totally wrong. Plenty has been written about the arbitrary nature of numerical targets and their propensity for triggering dysfunctional behaviour, so we’ll leave that to one side for now, and just look at why using them as the focal point for judging performance simply means people engage in the wrong conversation.

Stick Child has drawn a couple of charts, which plot the distribution curves of A&E admission times for two hospitals.

 

As you can see, the curves are different. Hospital ‘A’ manages to get 95% of patients seen within 4 hours, after which, a steep drop off in the curve indicates the remaining 5% are seen before 4 hours and 30 minutes has elapsed.

Hospital ‘B’ also manages to see 95% of patients within 4 hours, but the tail of the curve beyond this point is much longer, meaning that the remaining 5% of patients take much longer to be seen – some waits are as long as 7 hours. It’s also apparent that Hospital ‘A’ sees more patients during the early stages of their wait than Hospital ‘B’. This is evidenced by the fact that Hospital ‘B’s curve is weighted more to the right.

So, its quite clear that the patterns of waiting times are different for the two hospitals.

Not according to the target.

According to the target, the two hospitals’ performance is exactly the same. This means that opportunities to understand why some patients wait up to 7 hours in Hospital ‘B’ are missed. It means that managers don’t get the chance to understand their system, as the usefulness of A&E admission time data is undermined by using the target as a focal point, thereby degrading potentially useful information into a simplistic PASS / FAIL scenario.

Now consider what would happen if Hospital ‘A’s distribution curve actually showed that 94.9% of patients were seen within 4 hours, whilst Hospital ‘B’ achieved 95.1%.

Yep, despite the fact that Hospital ‘A’ demonstrates better overall performance, it FAILS, whilst Hospital ‘B’ PASSES. This fixation on the target and a binary definition of ‘good’ or ‘bad’ performance means no one learns anything about either hospital.

That’s the real FAIL.

Then there’s those convincing-looking charts which look authoritative, but actually spew out what can only be referred to as pseudo performance data, such as this:

Looks good, doesn’t it? Well it’s not. All it does is tell you the percentage of cases where performance has crossed that invisible and imaginary dividing line between ‘good’ and ‘bad’, as defined by the target. A chart that tells you about a target to hit a target. Utter waste of time.

If you’ve got the data, simply plot them and learn from them. Why throw in a target and thereby replace the richness of potentially useful performance information with a meaningless and mind-numbing YES / NO game? Seriously, why?

It drives the wrong conversation.

Avoidable Harm

I recently had a conversation on Twitter about a national campaign called ‘Sign up to Safety’, which aims to reduce avoidable harm in the NHS. Now, avoidable harm is clearly something worth tackling. The sticking point for me was that they have a numerical target to reduce avoidable harm by 50%.

What I can’t understand is why anyone would aim to reduce avoidable harm by 50% – if it’s avoidable, we should avoid it! Not just some of it. Why would you want to be ‘half-safe’?

It’s like deliberately planning to retain the other 50% of harm! Which, of course, sounds silly – because it is silly.

There’s a lot that’s silly (and harmful) about such targets, such as the assumption that a target is necessary to make people want to reduce harm in the first place. If they know that reducing harm is important (i.e. a priority), then the target is irrelevant. It might even be possible to measure some types of harm reduction, so that’s good too, because then you have measures to help you understand how your harm reduction efforts are going. The target is still irrelevant though.

Anyway, why is the target 50%? How was this determined? Why not 55%, or 70%, or 81.648%? If it was set at 50% because it was deemed attainable, then what’s the point of the target, because you’re gonna attain it anyway, right?

Why is a 49.999% reduction a failure, whereas a 50.001% reduction a success? These invisible dividing lines between ‘good’ and ‘bad’ simply don’t exist in the real world. If you could reduce more harm than 50% then you would, wouldn’t you? If so, the target is irrelevant. If you wouldn’t, then why not?

How about if you reduced all the harm you possibly could, but this only amounted to 35% less harm? Have you failed? Why? What about if you had it within your gift to reduce harm by around 80-90%, but only reduced it by 55%? You’ve exceeded the target, but is this good?

Then there’s the stuff about method. How does a numerical target set at any level help you identify and address harm reduction opportunities? It doesn’t, because targets don’t provide a method.

Also, as I’ve said before, it’s better to aim for 100% (i.e. perfection) than just a fraction of your true goal. You’d then measure, learn and improve as you go along. Yes, in many domains (such as harm or crime reduction) it may not ultimately be possible to completely eradicate the object of your reduction efforts, but this shouldn’t stop anyone from trying.

Let me give you a few examples using the Stick People, to try and demonstrate why numerical targets like the 50% target for avoidable harm are pointless (not to mention arbitrary and prone to causing dysfunctional behaviour).

Here’s Stick Doctor. Today, Stick Doctor encountered two opportunities to reduce harm in her hospital. Guess how many she addressed? (Clue: It wasn’t one).

This is Stick Cop. Stick Cop currently has four investigations in his in-tray. He’s decided to investigate all of them to the best of his ability. Not just two.

Here’s Stick Child. Stick Child saw one opportunity to help a group of under-10s get their heads around some basic performance management concepts. He didn’t stop half way through.

Get  it now?

If you have a worthwhile priority, just focus on that. Measure your progress, using the right measures in the right way. Learn and improve. You don’t need the target.

Reduce avoidable harm by reducing numerical targets!

By 100%.