Do Something Different

Tyn ChurchI recently came across this blog post by Bernard Rix, the Acting Chief Executive of the Confederation of Police and Crime Commissioners (CoPaCC). Bernard has been thematically analysing the draft Police and Crime Plans that Police and Crime Commissioners (PCCs) are producing for the year ahead. The particular post that caught my eye was, as you might guess, about PCC’s different approaches to performance management.

What struck me was the wide spectrum of approaches proposed within the various plans. Some were particularly encouraging – I have copied a couple of quotes below:

  • “Policing is complex, and the aim must be to do the right thing for victims and communities without slavishly adhering to indicators and targets where there is a danger of chasing a target and missing the point.”
  • “I will not be setting any specific numerical targets in this Plan. I believe that a focus on targets can distort the system and, due to possible ‘gaming’, lead to unintended consequences that can deflect from attaining the desired outcomes.”

I could have written that second one myself. Nice choice of words!

This is great to see and I sincerely hope that these bold intentions translate into meaningful performance frameworks that keep policing activity focused on doing the right thing, whilst ensuring transparency and accountability.

However, at the other end of the scale, some plans are littered with numerical targets and language that is synonymous with the type of performance management that police and public alike had desperately hoped was on the way out. One quote reads:

  • “I have set clear ‘hard’ targets and I will hold the Chief Constable to account for reaching these targets in the coming year. I have also included, where appropriate, additional ‘stretch’ targets to indicate the aspirational goals through Year 1 of this plan and into Year 2.”

Facepalm.

facepalmNow, if you are a regular reader of my blog, you’ll know that I often thrown in a bit of mischievous humour from time-to-time, poking fun at daft performance indicators such as numerical targets for cleaning up dog poo and the like. This time I’m going to tone down the trademark jocularity because I want to get a serious message across, before it’s too late.

So… If you are a PCC and you intend to include numerical targets in your Police and Crime Plan, I don’t doubt it’s being done with the best of intentions, but please consider the following points:

It’s Guesswork

finger in the airHave a think about this: If your plan includes targets such as ‘reduce robberies by 10%’, ‘attain 85% public satisfaction’, or ‘no more than 300 road deaths’, firstly, ask yourself where these numbers came from. Chances are, the ‘science’ involved boils down to little more than the classic ‘finger in the air method’. This is because all numerical targets are arbitrary.

It doesn’t stop people trying though. ‘Methods’ range from the pseudo-scientific, to the extremely simplistic, as well as the outright unfathomable; at the more ‘sophisticated’ end of the scale they include aggregating multiple data sources, attempting to predict trajectories and even applying algorithms; conversely, the good old ‘let’s look at last year’s / someone else’s figures, then add or subtract a few percent’ method is also very popular. Whichever method is used to conjure up the numbers, the final numerical target will always be entirely whimsical.

Furthermore, these types of targets inadvertently send out the message that a 10% reduction is the best that can be hoped for, that the plan aims for 15% of the public to be dissatisfied, and that ‘only’ 299 road deaths are okay. If that sounds silly, it’s because it is silly. Whilst it is possible to estimate the range within which data will continue to fall if the operating conditions of a system remain unchanged, it is not possible to set a precise numerical target within that data range. For all the effort and expertise of stock market traders, I know of none who can predict the exact point that the FTSE index will close at tomorrow.

Targets Don’t Actually Work

Targets are implemented on the assumption that they ‘motivate’ people and thereby improve performance. This is fundamentally misguided for a couple of main reasons. First of all, it ignores the reality of why people strive to do a good job – i.e. intrinsic motivation. (For more on this, read my canny tale of talking deer). Secondly, and from a purely practical perspective, targets (along with many traditional performance management methods) focus on the individual, rather than the system. This is a deadly error, as the system is where the overwhelming majority of performance derives from. I reproduce a pie chart from a previous blog post to illustrate this:

94-6 pie chart

Let me be clear that targets cannot and do not work on the system – therefore, even if they were a non-arbitrary, side effect-free, perfect tool for improving individuals’ performance, managers who rely upon them are concentrating on squeezing a fraction of a percent out of the thin slice of the pie, whilst missing the huge opportunities for positively influencing the way in which the overall system operates. Why would anyone do this???!

The chart below represents the performance of one aspect of a system. It could be response times, the public satisfaction rate, recorded crime, or anything else you choose.

Control chart

The pattern of data indicate that the process is stable (i.e. no data points fall outside of the control limits depicted by dotted lines). This means that if this process continues to operate under the prevailing system conditions, the data will continue to fall within these boundaries ad infinitum. Now, if you felt that public satisfaction rates were too low (or crime was too high or whatever), a common reaction is to set a target for improvement. Let’s see what happens when this occurs:

Control chart with target

Here, the target is represented by the red ‘X’. As you can see, it has no effect on the spread of data. This is because the data do not know it is there. The data plod on regardless within the constraints of prevailing system conditions. Setting a target within the existing range of data just means that sometimes the target is hit and sometimes it isn’t, regardless of constant effort. Now, if you were to set a target outside of one of the control limits (say your response times ranged between 6 and 20 minutes and you thought a 5 minute response time target would be nice), then this is beyond the capabilities of the system and can therefore not be achieved unless the system itself is reconfigured. Holding individuals to account for something that is a direct result of the system design is pointless and counterproductive.

Special Offer: Adverse Consequences Free With Every Target!

Finally, no matter how well-intentioned targets are, there is none that is immune from causing dysfunctional behaviour. This is fact. There is a range of evidence worldwide that demonstrates this. It’s already known. Have a look at these articles for a start:

Guilfoyle (2012) – On Target? Public Sector Performance Management: Recurrent Themes, Consequences and Questions.

Rothstein (2008) – Holding Accountability to Account.

Bevan and Hood (2006) – What’s Measured is What Matters: Targets and Gaming in the English Public Healthcare System.

I’m not going to go into the dysfunctional behaviour that targets cause, as we would be here all day – the consequences of target-driven performance management are well-documented in these articles, elsewhere on my blog pages, and in the news, as I’m sure you are already aware.

Here’s some common pitfalls to watch out for, which people sometimes think will mitigate the adverse effects of target-driven performance management:

  • Simply implementing fewer targets. Well, this is exactly the sort of thing that systems thinker John Seddon is referring to in his point, ‘Doing less of the wrong thing is not the same as doing the right thing’.
  • Assuming that it’s not the targets that are the problem, but the way they’re implemented. Sorry, but the pathogens are built into the target. There is no way to effectively implement something that is inherently defective.
  • Increasing ‘accountability mechanisms’ to counteract the dysfunctional behaviour likely to be caused by the targets. This misses the point. It also generates unnecessary bureaucracy and waste. It’s like bursting a balloon then blaming the pin.

The Solution

It’s easy to criticise, but criticism is unhelpful if no alternative is offered. Well, the crux of my alternative is in removing numerical targets and replacing them with meaningful, purpose-derived measures that actually tell you how your system is performing. I cover this proposition and explain the important distinction between targets, measures and priorities in my blog ‘Spot the Difference’. Go on, have a read.

My final thought on the targets issue is that I don’t necessarily believe people install targets because they’re bad people – targets are just so commonplace in conventional management practice that it’s easy to adopt them unquestioningly. They appear to be a plausible method for achieving what we all want – better performance. There has been some really constructive and lively debate on the subject recently, on social media and beyond, involving members of the public, PCCs, and police officers of all ranks. This suggests to me that people really care. I certainly do.

As recently as this morning, someone posted a comment on one of my other blog posts which read:

“…and now I think the challenge is to convince PCCs”.

I hope this goes some way toward doing so. If you are in a position to grasp this unique opportunity to take police performance management beyond the dark days of hitting the target whilst missing the point, then if you haven’t already, please be bold enough to do something different.

‘Appendix One’

To rebalance the proportion of lightheartedness per blog post required to meet my self-imposed arbitrary numerical target for jocularity, I’ll return to the mysterious photo at the start. The only reason it’s there is because it was taken in the beautiful city of Prague, where most of this post was written. Here’s another, taken in my favourite bar there, alongside two of the barstaff.

Prague bar staff edit

More Krušovice Černe please!

Advertisements

About InspGuilfoyle

I am a serving Police Inspector and systems thinker. I am passionate about doing the right thing in policing. I dislike numerical targets and unnecessary bureaucracy.
This entry was posted in Systems thinking and tagged , , , , , , . Bookmark the permalink.

10 Responses to Do Something Different

  1. bernardrix says:

    Reblogged this on Bernard Rix – mostly re policing… and commented:
    A fascinating commentary on my earlier blog on PCCs’ varying approaches to performance management!

  2. On the one hand we have an ever growing body of evidence to show that targets are not only ineffective at improving performance, but also drive some very poor and undesirable effects. Whether it’s crime figures, or hospital treatment data, or school exam performance, or finance industry sales and bonus targets, or late trains, or child neglect, or any other target regime anyone cares to mention.

    On the other hand, we continue to have managers and people in authority persisting with the mantra of targets. In their view, there’s nothing wrong with targets – it’s just the employees who are bad (require more time spent in training, introduce targets for professional standards), the managers who are weak (more training, leadership standards), the checking and inspection regime which is weak (employ more inspectors), the data collection and analysis system which is poor (introduce more forms, buy a new IT system, employ more data collectors and analysts). But there is no alternative – how else do you keep staff in check?

    The latest and perhaps craziest iteration is the proposal to make it a criminal offence to fiddle figures (see link:
    http://www.telegraph.co.uk/health/healthnews/9934211/New-criminal-offence-to-stop-NHS-hospitals-fiddling-figures-to-be-introduced.html).

    The mind boggles…

  3. Another delightful post! I too hope that the obsession with “targets” is relegated to the circular file where it belongs! Keep up the great work and humor!

  4. Pingback: Do Something Different | Police and Crime Commissioners | Scoop.it

  5. yo mo says:

    Just when I thought my mood could sink no lower, Confederation of Police and Crime Commissioners. Not content with foisting a bunch of clueless, political appointees* into a manufactured office to ‘oversee’ policing there is now another layer as well. Shoot me now.

    [*Yes they were elected, but a turnout of less than 20% is hardly a mandate from the electorate, especially given that recent votes with a higher turnout have been dismissed as irrelevant.

    Bernard, if you are reading this it is nothing personal, the job came up, you went for it and were successful, fair play. My despair is with the system, not the players, [those found wanting will reveal themselves anyway].

    And if you are going to tackle some of the pointless collection of irrelevant and largely bogus statistics, then I will be properly impressed. But you are trying to change a practise that has been in place for over 20 years, and most of those that dabble in it know no better.]

    bold intentions
    translate
    meaningful
    performance frameworks
    focused
    transparency
    accountability

    Blimey, Boss one more and I’m on a full house here,.
    [Though I’m guessing the prize will be a homoeopathic quantity of beer delivered via the nearest cold water tap].

    I think the debate around “all numerical targets are arbitrary” needs to evolve since it is demonstrably false and does not convey the true problem. I think “hitting the target but missing the point” is a better working analysis of the effect. But it needs refining or expanding.

    Turning to the main thrust.
    If one seeks to change anything, there has to be a ‘baseline’ or ‘reference point’ established in order to assess the effect of any change, be it positive or negative. To give a mechanistic example, it might be decided to reduce the process speed in order to establish the impact, on say the finish of a product. And if successful then the process equipment would be changed to achieve this. [In the case given, to avoid a bottleneck the capacity at that point would be increased to maintain the throughput, by adding extra process capacity in parallel].

    Looking at the examples cited:
    ….’reduce robberies by 10%’ …. the 10% is pointless since the objective is to reduce robberies to zero, it might be practically impossible to get to zero, but if you never try you will never know.

    ….‘attain 85% public satisfaction’ …. well this is a very odd thing to have as a objective/goal/target/aim/ambition. It depends who is defined as “The Public” and how the question is phrased. If for example the sample includes those that have been arrested then survey is wrongly conceived and the result meaningless.

    ….‘no more than 300 road deaths’ …. OK a bit of hyperbole there, no-one would actually say that out loud if anyone was listening, they would be ripped to shreds, but they general point about reduction applies.

    Targets and comparisons as invidious as they may be are unavoidable, there exists no other system to establish the effectiveness of an ‘intervention’ [for want of a better word]. And to rail against them to the exclusion of all else is a mistake.

    What to use instead? A very difficult question. But let stick my neck out and have a stab at a single ‘functional’ definition of one example of good performance, [because they are bloody difficult to define, without splitting a complex function into unrecognisable sub-fractions of the whole] . This is no more than nebulous approximation to be completely torn apart, mocked and held up to ridicule on the way to finding better definition(s).

    An event, could be any thing but let’s use a football match: Did everybody get to the match safely? Were they safe in the ground, and did they get safely home.? Could apply to anything, a night out, going to the shops.

    On motivation, motivated people will respond to appropriate targets be cause they are motivated people. When a target is wrong, they will resist, point out the flaws, but if people above them insist, they will at best pay lip service to it, or ignore it completely. If they are compelled to do it then they will, grudgingly and to the absolute letter. Unmotivated people couldn’t care less, what ever the target is, appropriate or not, unless there is punishment attached.

    “Targets do not work on the system” , this really depends on the system components, if ‘the system’ contains “meat puppets” then it can produce all sorts of odd effects. Including a complete system collapse, which is why using a mechanistic approach (system thinking) to ‘system engineering’ where some or many parts of the system are not under reliable control can be a big mistake.

    [Even in a purely mechanical system, attempting a system change can have very bad consequences, as individual components have ‘performance envelopes’ and once these are exceeded, the system may become non-linear, fluctuate erratically or just fall apart. Sometimes with a bloody great bang, and bits fly around the room, with smoke and flames as extra entertainment. ]

    You and your wacky charts, I’m not going to pull it apart because it brings out all the ‘true believers’ who are shocked that I’m slaughtering ‘holy cows’ again. But there is something very odd going on with these. [I did break open a spreadsheet a few months back, but had to abandon it for lack of time.]

    Let’s take your key point, back to the manufacturing environment where these charts belong. The process is running and producing results as shown in your chart between control lines.

    And I’m in charge of it [it’s my fantasy kingdom after all], one day my Boss rocks up and says there has been a change in the specification, and draws a new specification limit below the control limit. Perfectly legitimate, customer demand may have changed or there might be a functional reason related to a wider performance issue in the final assembly of which this is just a part.

    Without even running the process it is apparent that the process needs to be modified. Can the process be changed to meet the new specification limits? Remember specification limits not control levels'[they exist implicitly on all control charts, but are not always drawn]. Possibly, but it will require addition resources of one form or another.

    Now if we fold that analogy back into the Policing model, then people can insist an a reduction in X or Y form of crime. But they have to put up additional resources to achieve it. Which obviously translates into hours worked, and money spent.

    Finally, I did read your Oxford paper, the ‘rot’ goes back a lot further than you describe, and has its roots in some work that was done in the 70’s that didn’t come into flower until the mid to late 80s. And then only affected the manufacturing sector, it didn’t really get its teeth into the non-manufacturing sector until the early 90s, and from there spread like a nasty disease into the public sector. With all the unwanted effects now experienced.

    Long story, short analysis, the public sector is sufficiently different to the private that it requires a very different approach in managing its ‘performance’. It needs a methodology completely different from the private sector. It is big enough to develop its own measures and should do so rather than mindlessly copying those used in the private sector. Which is not the paradigm of efficiency that those who have never experienced it believe it to be.

    “Na zdravi”

    • A lot of good points.

      Targets do work, but in the public sector they don’t work in the way that the bosses would intend them to. Put a target in place, link it to rewards/sanctions, and you end up with skewed behaviours. Apart from the arbitrary nonsense of aiming for 15% of victims to be dissatisfied (oops, sorry, I meant to say 85% satisfied), the focus on hitting the numbers, as you say, leads to activity which means you may hit the target but miss the point. And yes, there have been targets for road deaths. The last government set national targets for it, linked to funding.

      We are all very comfortable with targets. I’ll reward myself with a beer when I finish mowing the lawn. I’ll have a rest once I’ve finished this last piece of work. I’ll pay the builders when they’ve finished what they were asked to do.

      The difference, I think, is the degree of control. In manufacturing, you set the specification, and you manufacture to that specification. If the specification (target) changes, you need to change something in the production process to meet the new spec. But you control most, if not all, of the process.

      In services, and in the public sector, you have much less control over the process. To what degree can the police ‘control’ the number of fatal accidents, for example? If you take that as an example, some of the biggest improvements over recent years have been through car design, wearing seat belts, and improvements in emergency health care. The evidence on speed cameras is still not clear. If you look at general trends, the weather seems to have a big influence on the number of fatalities in any one year. A single tragedy, such as coach crashing with multiple fatalities, can really distort ‘performance’. When you look at the individual causes of accidents, there is often very little (if anything) that the police could have done to prevent it (eg, a driver suffers a heart attack at the wheel).

      However, the performance management logic kicked in anyway. Most accidents are down to driver error, excess speed, drink/drugs, using mobile phone, etc. Therefore, if there is enforcement activity against those factors, driver behaviour should improve, and the number of accidents should reduce. But there are a lot of ‘ifs’ with that logic.

      Most services, whether private or public, deal with individuals. What works for one person, may not work for the next. Targets assume that the outcome can be controlled through the process. With services, targets distort the behaviours and lead to dysfunctional processes. You can only work on those elements you can control.

  6. Dave Hasney says:

    Unlike you, I’m no expert on the prevailing problems. I’m not a scientist, an economist, a statistician or a bloody politician (thankfully), like many who join the police I just wanted to help other people enjoy a better life and rid them of factors that blight their existence. Throughout my thirty year service I became increasingly dissatisfied with the ‘system’ and how it became a plaything for self-promotion. That’s why I started to study that ‘system’ in more depth and subsequently proved to myself (and others) that I could also obtain academic qualifications, to back up what I was saying…
    My worry is that; despite how simple to understand our observations on the current policing management ‘systems’ we are unlikely to see major change for the better. What we have now is so intrinsically embroiled in the self-promotion and self-importance of those responsible for management of the ‘systems’ – we need a sea-change in the ethics and methods of those people. We need those managers to love the system they are part of and what it is designed to do i.e. provide ‘service’ to others. The processes contained in that ‘system’ are not there to enhance the career aspirations of self-indulgent and self-important individuals!

    • Dave – like you, I joined the police to help people, and agree that significant changes are required for the sort of things I argue for to take root. I do sense an undercurrent of interest in the systems approach at the moment and remain hopeful for the future.. Keep reading the blog and thanks for your ongoing contributions!

  7. Pingback: Do Something Different | Policing news | Scoop.it

  8. Sean Flanagan says:

    I think that targets were created by a government that was more concerned with soundbites and attention-grabbing headlines. I have absolutely no faith in any of the political parties. I have no time for trashy state-controlled tabloids or government ministers who, in their infinite wisdom, think they have all the answers but can’t be bothered to speak to people on the ground. Well-meaning politicians? Sorry but I haven’t seen many of those and I doubt I ever will. Governments may change but the lies remain the same.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s