Data Done Properly

In contrast to all the mangled, misinterpreted, unpublished and skewed data there are beacons of hope. This time it emanates from none other than Local Government, not known for leadership in these areas. So pleasing to see.

I saw this LinkedIn post by Steven Johnson and immediately commented. It is such a delight to see data being used properly, from gathering to analysis.

 

Annotation 2019-03-19 074815

That is all. Have a wonderful Tuesday.

 

PS: No conflict of interest as I have never even met Steven!

Advertisements

Data – The Fog of Promises, and What To Do About It

Calculating the value of data is something I have been thinking about a lot. Data, any and all, seems to be relentlessly hoovered up whenever we use any form of connected device. Who had ever heard of data as a mainstream topic twenty years ago? Nowadays, we have seen Mark Zuckerberg answering to Congress in the States and countless articles based around what Google and Apple know about us. Some people are laissez-faire about it whilst others veer towards the downright paranoid.

zuck in congress

Organisations collect data, they hoard data and (hopefully) guard these vast amounts of data that they collect. Why? Because it is valuable. It is useful. Apparently. However, who in a company actually gets down to the nitty-gritty of this and can measure and express the Return on Data that this feverish collection and hoarding actually brings to the organisation?

In 2015 Doug Laney from Gartner wrote about data in financial terms. How it can affect the value of a takeover target, if they have a vast unexploited data store, for example. Were that to be monetised then what is it worth? Does this mean the buyer is getting a fantastic deal or when it seems to be overvalued on traditional metrics is the difference made up by the value of their data? Herein lies a real problem as the difficulty in valuing data stems from several reasons.

Firstly, that there is no firm formula to do so, because to some that data is just wasted storage and to others it is gold. Whereas a physical asset, such as a piece of land, is such a mainstream asset that it is far easier to value. With data, the  great big lump of bits and bytes only has value if the owner knows how to extract information and insight from it, and use that effectively to make them more competitive or to sell to someone else in a finished and usable form. People have had a stab at it by trying to vary old maths and make it new maths. I found the following on the Internet:

ROD_Definition

Though his looks like an elegant formula,  the Gain from Data metric is subject to so many other variables, primarily time,  it is almost impossible to calculate so simply, makinf the formula impossible to scale. It only serves to highlight just how the temporal aspect of data value is so important. Depending on what it is, it may be very time limited, making it useful only in a very brief window. Think of data like a paper currency that can burst into flames at any moment.burning-money-png-2

One second it has the face value and the next it is ashes.

In contrast, a piece of land is just there. No more land is being created, whereas data creation is never-ending: limited only by our ability to get it and store it.

Secondly, the technical aspects are crucial. What form is it held in, on what type of database, where is it held (there are massive regulatory differences around the world), have the data owners consented to its use, by whom, how old is it, how consistent is it and so on. If I can’t use in my company for my purposes then it is just Ones and Zeros on a hard drive somewhere, merely cluttering up the ether. Utterly without value.

The fact remains that extraordinary amounts of data are being recorded about us, all of the time. I recently holidayed in Norway and in ten days I didn’t use one bit of hard currency. All card, all the time. I navigated around using Google Maps. I checked TripAdvisor and used Uber, as well as uploading countless photos to Facebook for family abroad to see. In doing so I must have left an enormous digital smear across the Norwegian landscape. Me and the thousands of other tourists on holiday at the same time. Can you imagine the quantity of data generated by me and the billions of other people using connected services every single day?

To be able to achieve a RoD that makes all the efforts and costs at collection and storage worthwhile, several things need to happen and I can only really see that these can happen under the guidance and direction of a very senior – if not on the Board type senior – individual who guides a team with specific responsibilities. Call them a Chief Data Officer (CDO).

Ideally the value of data is considered so important that the CDO is on the Board. The CDO would need to have close ties with Marketing and Strategy functions to understand how they intend to use resources to achieve them, and whether existing data is useful or new data needs to be acquired. Additionally,  they need to know how to shape and deliver it to them in a worthwhile manner. Then there needs to be a real-time feedback loop – Sales? – in order to assess the efficacy of the deployed data as well, as a direct line between them and the technical functions of the company. The sort of things CIO deals with, especially storage and access. The CFO will have demands on their funds from the CDO. They need to be able to understand the RoD and how it is affecting the bottom line, the share price, their partners and so on.

businessman staring into fog

Most importantly of all is someone who can see through the Fog of Promise that all this data is purported to hold. The RoD that can be achieved if only they used it ‘properly’ is the sort of golden thread that is so often sold to them. Correlation does not equal causation. I’ll repeat that: correlation DOES NOT equal causation. Falling into the Feynman Trap is something that affects the best and the brightest (Famously, Jim Collins did this in Good to Great). Usually when they become mesmerised by their own belief in the infallibility of data.

The CDO not only ensures the data is valued correctly, they are responsible for preventing their company being led down a rabbit-hole of promise of the jam-tomorrow variety. The sunken cost fallacy remains as relevant today as it ever was and sometimes the emperor is indeed naked.

 

 

Algorithms – A Bluffers Guide

A breakdown and simplification of some current tech speak

The word ‘algorithm’ is uttered with a degree of reverence these days. It is the black magic behind AI and Machine Learning and is a favourite thing to go rogue in a modern plot line. The actors merely blame the bad algorithm when a computer goes crazy in a dystopian sci-fi catastrophe.   The decision making requirements that we are faced with in the modern commercial world far exceed our capacity in many instances because our brains evolved for a very different sort of world. A world of small groups where we rarely met anyone very  different from ourselves. We had significantly shorter lives and our main priorities were sex and survival. These days there is hugely increased complexity and nuance yet the evolved desire for rapid choice-making hasn’t left us.  Faced with these pressures we turn to computers for help. Computers helping humans is so pervasive and permeates almost all aspects of life. Such a rapid change has occurred in the last lifetime as the evolution of computing capacity increases exponentially. Your mobile telephone has vastly greater computing power than all three computers on the first Space Shuttle. Think about it for a moment. Your phone possesses all the computing power required to fire you into space. This incredible capability means that people have been fascinated with the idea that a computer can be turned from a dumb machine into a thinking machine (thinking as we do) since the dawn of the first machine. However, computational power is one thing. How to make it work as an independent thinking machine is another thing all together. One of the key things you need to do this you need an algorithm.

Algorithms: the rules needed for machine thinking. 

Algorithm Just to clear this up. Machines DO NOT think. Computers can process a huge volume of information, really really quickly because they are unbelievably fast calculators. The hardware is just a superfast counting machine with a screen. Algorithms are not hard to conceive, if you think of them like this; an algorithm is what you need to cook supper for your family. Few families eat the same thing for every meal of every day so there are constraints and variables. Imagine there are four of you. One is a vegetarian, one is on a low-fat diet and the other too aren’t that fussy but do have preferences. You want to provide them with a nutritious and tasty meal that ensures everyone enjoys the experience, including you.   Let’s imagine that you are 45 and have cooked for the same people many times before (almost daily) and as a consequence you have learnt a lot about what works and what doesn’t. However, this week is different and you haven’t had time to shop and the other three did the shopping for you. You open the cupboard doors and have a peer in the fridge and freezer to get an idea of what is available for you to cook with. Within about 30 seconds of taking stock of the cupboard contents, the fridge contents, the available utensils to cook with, any time constraints, the dietary preferences and so on you decide on a meal. You cook it, serve it and everyone eats. They get up from the table appropriately nourished leaving the process to be repeated the next day. What allowed you to do this was an algorithm in you head. Call it the ‘cooking for family’ algorithm.  Algorith wordcloud   Pause for a moment though and think about how simple it can sound and actually how the thinking and actions required was so incredibly, amazingly, mind-blowingly complex and nuanced.

 A quick note as to where this can go wrong

Simply put, computers are not people. Computers are superb for making decisions that do not require any emotion, ethics, bias and the like. Eventually a computer beat a Chess Grandmaster and uit did it by sheer computational brute force. However, to take the supper example: the cook knows the audience at a level a computer can’t match. All the calculations from an algortihm and it can’t know from someone’s face if they are the special kind of tired that a Wednesday can make them, so putting any kind of pie down for dessert would mean the world to them. And the others would see that a pie was not only what was needed but was a very thoughtful gesture thereby elevating the cook in the eyes of the other three and making an intangible but felt contribution to them too.  The aim is to have algorithms teach themselves by learning from mistakes in order to achieve the desired outcome of the programmer(s). They try,  but they are far from perfect and because we expect perfection from computers, in a way that is different from our expectations of one another, then mistakes are not easily forgiven. Algorithm 2

Data Ethics For Business

We exist in an increasingly data driven world. More and more, we are encouraged or directed to ‘listen to the data’ above all else. After all, the data doesn’t lie. Does it?

bigdatawordmap-1264x736-672x372

Data Ethics in business is the name of the practice used to ensure that the data being used to make high-value commercial decisions is of the highest quality possible. However, there is a catch. Human beings are the catch. We have  gut-instinct, prejudices, experience, belief systems, conditioning, ego, expectation, deceit, vested interests etc. These behavioural biases all stand to cloud the data story, and usually do.

A high-value commercial decision does not necessarily have immediate financial consequences. Although, in commercial terms, a sub-optimal outcome is invariably linked with financial loss. In the first instance, the immediate effects of a high-value decision can be on organisational morale or have reputational consequences.

responsibility

When a high-value decision is to be made there are invariably advocates and detractors. Both camps like to believe that they are acting in the service of a cause greater than themselves. Occasionally, some of the actors cloud the story because their self-interest is what really matters to them, and they try hard to mask that with the veneer of the greater good. Hence the term ‘Data Story’, because behind the bare numbers and pretty graphics  there is an entire story.

The concept of conducting a pre-mortem examination of the entire data story to model what can go wrong is becoming more important for senior decision makers. It is getting increasingly difficult to use the traditional internally appointed devil’s advocate as, due to the inherent complexity of understanding a data story, this function needs to be performed by subject matter experts. Although the responsibility for decision-making always falls on the Senior Management, they want to do it with a full breakdown of the many facets of the data story.

BigData-wordcloud-2

 

In order to achieve this, individuals with a unique blend of talents, experience and inquisitiveness must be used. People with absolute objectivity and discretion, who don’t rely on inductive reasoning. Ones who are robust enough to operate independently, diplomatically and discreetly and have executive backing to interrogate all the data sources, ask the difficult questions and highlight any gaps, inconsistencies, irregularities. From this they can provide a report for the Executive Sponsor(s) with questions to ask and inquiries to make so a well-informed decision can be made.

After all, when there is  lots at stake, no one wants to be remembered as the person that screwed-up and tried to blame the data?

Why is data dangerous?

In the words of @RorySutherland: “The data made me do it” is the 21st Century equivalent of “I was only obeying orders”. The growing power and influence of Data Science touches everyone’s lives. Sutherland also remarks: “Markets are complex and there can be more than one right answer. People in business prefer the pretence of ‘definitive’ because if you can show you’ve done the ‘only right thing’ you have covered yourself in event of failure”. These are all attempts at Plausible Deniability, and they are weak.

For the record, plain old data is not dangerous, you are unlikely to be hit by an errant Spearmans Rho, or a rogue Control variable that detached itself from an analysis. Data is just a record of the measurable values of something that has happened in the past. Digital exhaust, if you will. Like speed in a car, it is the inappropriate use of it that causes issues.

zuck-data

Doing the right thing often sees people becoming  enslaved to Type 1 and Type 2 data, because they are the easy parts. You can hire experts, who can count well, use the software and understand how to tease out knowledge from the data points. What the majority can’t do, or may even do intentionally, is to manipulate the presentation, context and language used when presenting their findings. This is the Type 3 data I talk about, that isn’t traditional data as we know it.

Type 3 data is the really dangerous stuff. The reason for this is our complete fallibility as human beings. This is nothing to be ashamed of, it is how we are made and conditioned. It is in fact, entirely, boringly, and ordinarily normal. I was recently told by a lawyer – I say this because she is pretty well-educated – that all statistics are a lie. She then cited the famous Mark Twain (nicked from Disraeli) saying of, “There are lies, damn lies and statistics”, as if this were all the proof she required. Interestingly, when I challenged her on this and made a case for accurate uses of statistics she refused to even acknowledge this. She was wedded to her belief and I must be wrong. Case closed.

statwordcloud

I think immersion in courtroom rhetoric may have been getting the better of her. However, this goes to show the just how dangerous we humans can be. Imagine being a client with a lawyer whose dogmatism may cause them to overlook or be able to question relevant statistical evidence? All stemming from a strongly held view that all statistics are lies. Professor Bobby Duffy recently wrote an excellent book called Perils of Perception and on p.100 he shows just how problematic this view can be.

My point is: If a person who is well-educated, and practising in a profession like law, can hold such a position, then it is not beyond any of us to do so, quite unwittingly. Until one is more familiar with the behavioural biases that we are all susceptible to, the way Type 1 and Type 2 data can be mis-represented (Type 3 data) and how that uses our in-built foibles to generate a reaction.

This is where someone who understands both of these areas, and can blend that knowledge into an expertise which is useful, can help you. When important decisions on strategy, direction and spending  are conditional on interpreting data from others, you want to get it right first time. If not, you’ll be forced into, “The data made me do it”, and that rarely ends well.

burning money

 

 

 

 

Let’s break down today’s bad data usage – Yes Guardian newspaper, I mean you!

Overnight there was a report published in The Guardian newspaper entitled “Met police’s use of force jumps 79% in one year”. I see the hysteria on Twitter – being whipped up and added to by the usual suspects who revel in the dog-whistle approach to political discourse – about the use of force by the Metropolitan Police being used disproportionately against black people.

“The Metropolitan police’s use of force has risen sharply in the last year, with black people far more likely to be subjected to such tactics than anyone else, the Guardian can reveal.”

Firstly: this is not an attempt to take sides. The police may be guilty of the accusation. Without correct and fair anaslysis of data it is impossible to tell. See a previous post about how to approach stories like this.

Secondly: the purpose of this article is to interrogate the findings of the Guardian’s reporting of this story. If this undermines the story then so be it. Do not conflate that with an endorsement of the police in London, for I do not know enough to comment about them. This is about the use of data.

The main thrust of the article is the “79% in a year” claim. It is what has been seized upon and retweeted with vigour. Nowhere does it appear that the people getting all worked up over this selective quote  have actually looked into the data.

“On 39% of occasions in which force was used by Met officers in the first five months of the financial year, it was used on black people, who constitute approximately 13% of London’s population.”

The first thing that struck me about this piece was the language and imagery used. Whilst the language is not the data, the way it is used certainly serves to alert you to the fact that they may be glossing over details in the pursuit of shock value. The Guardian is (was?) a credible broadsheet with a left-of-centre bias. Nevertheless, now they are giving away their content for free, they seem to be leaning towards the ‘clickbaity’ style of reportage, and that is a pity. Look at the graphic they have used. It is fairly emotive stuff. A white man pointing a weapon at you.

taser front view

In the first paragraphs the article uses words and phrases like, “jumps, risen sharply, most likely, on average, approximately, raised alarm, receiving end, stark figures, police culture” and the like.

These are written efforts to engage the enraged response (metaphorically speaking) part of our brains rather than the rational analysis. The System 1 reaction and not System 2 as Daniel Khanemann calls it in his book Thinking Fast and Slow.

Arguably, the alleged disproportionate use of force by police officers against black people is so serious an allegation that it warrants slowing down, taking a deep breath and analysing correctly?

Let’s break down the critical analysis a bit by asking some questions, and making some observations.

  • The only reference to the data used is “Guardian analysis of official figures“. This alone should sound the loudest alarms ringing in your head and set you into sceptical analysis mode. What figures, analysed by whom, what is their expertise, what were the controls used, compared to what, is there (perish the thought that a journalist is anything other than scrupulously impartial) an agenda on the part of the presenter of these figures?  [I think I may have found the data being used. See the bottom of the article for links. It certainly isn’t acknowledged in the article. This might lead a cynic to wonder if it may be being taken out of context and the journalists don’t want this easily checked for fear of undermining their credibility.]

 

  • Many people think of the word black being interchangeable (perhaps incorrectly) with people of colour. It turns out that the Guardian even mentions that ‘Asians’ and ‘Other’ are not part of this classification.

 

  • Are the figures generated by the ethnicity initially recorded by police codes for radio use or the self-defined ethnicity – 16+1 versus 9 – codes used by the subjects, even if they differ from the officer assesment. It doesn’t say.

 

  • In the last paragraph of the piece the most convoluted attempt at figures is used;  we witness the groups ‘Asian’ and ‘Other’ being rolled together to make a ‘52%’ claim sound more shocking. They need to decide how they portray things and stick to it. Previously the paper excluded ‘Asian’ and ‘Other’ from the ‘Black’ category and instead let them sit outside along with ‘White’ in order to use the five month and 79% figure on which the outrage is based.

 

  • There is no indication if these figures are split between reactive (responding to calls from the public), or proactive (the officers see something that they decide to investigate further). Proactive interventions are carefully considered by officers, they rarely steam in like you see in the movies. Things like back-up availability, whether they are single-crewed (and far more vulnerable), priorities like previous calls, outstanding paperwork (yes, really, there is a lot), their caseload and so on. Proactive policing is where a racist would shine as they would be able to target black people if that was their aim. From there they would need to engage and at least claim a veneer of credibility for their choice to use force. That wouldn’t last long as everyone would need to be in on it. These days that is very difficult.

 

  • Debra Coles from the charity Inquest is reported as saying: “This also provides yet more evidence about the overpolicing and criminalisation of people from black and minority communities. It begs important questions about structural racism and how this is embedded in policing practices.” – From other remarks in the article, when the Metropolitain Police were approached and asked for their view, it sounds like after losing some 20k officers the police are rarely proactive, mostly reactive. If only they had the time and resources to ‘overpolice’ anywhere.

 

  • What if, and I am trying to steer away from political and social commentary here for it is not my intention, the police respond to more incidents in places where there is a greater proportion of black people? I X amount of interactions involve use of foprce then is stands that the use of force against blacks is more likely. There is no doubt there is historical antipathy towards the police amongst much of the black community, especially in London. Previous generations of the Met (and other forces) have not been known for their even-handed approach towards the black community. Young men (for it is predominantly males)  in groups often feel that their masculinity is being challenged if an authority figure like a police officer lawfully requires them to do something. What if this leads to more physical  resistance which in turn leads to force having to be used? What if the white people, the Asians and the Others are more compliant when dealing with the police? What if, what if, what if? The fact is that these figures do not seem to be presented in a holistic manner. By that I mean controlling for variables such as age, gender, location, time, weather, changed police priorities, changed dynamics of interaction due to cuts in resources and so on.

 

  • The phrase ‘use of force’ is misused by the journalists and politicians. The police use a very specific definitionb, and it is not what the ordinary person thinks it may be. A voluntary handcuffing is a use of force. You know, the kind where the officer says something like, “for my own protection I am going to handcuff you.” and the subject complies. Perhaps a single-crewed female arresting a large male and having to drive him to custody herself. Merely drawing Captor (CS) spray needs to be recorded as a use of force. No one was sprayed, situation calmed down. Same as the drawing of a baton. Force is also shooting someone dead. There is a wide definition of force. Force, in police recording terms, does not mean taking the suspect to the ground in a violent bundle.

 

  • The whole method of recording has changed, a fact the paper skips neatly over. Too complicated to explain I imagine. The simple fact is that comparing these new figures generated and recorded one way with the past where they were not recorded in the same way, if at all, is simply invalid. It is far too soon to tell.

 

  • The politician, David Lammy MP, famous for trying to whip up stories like this to create indignation – I say this merely because he is a public figure who regularly tortures data or chooses to use tortured data –  betrays a lack of understanding when he talks about the criminal justice system and the police. The police in London are merely one small part of this national system. Saying there is systemic racism at each stage of the system in a piece targeted at the police in London does smack of trying to score wider points and not, in my opinion, worthy of inclusion. It weakens any point trying to be made. It is good to have Lammy on board for a bit more clickbait type appeal though. He has a large Twitter following and retweeted the article almost immediately. Surely not because he is mentioned in it.

 

  • As Matt Twist of the Met Police said, “…the figures should not be compared with population demographics. He said: “The collation of these figures is still in its early stages, and as this is new data, there are no previous benchmarks to compare it to. Therefore any conclusions drawn from them must be carefully looked at against this context, and should only be compared with those individuals who have had contact with officers, rather than the entire demographic of London.” You may think he is a police stooge but it does not make his statement incorrect.

 

  • The paper even says it is comparing  FY 2017/18 to FY 2018/19. This means that from April 6th 2017 to April 5th 2018 and similarly for 18/19. This is important because the new recording system was introduced from April 2017. The data being quoted is April to August 2017. It is being compared with April to August 2018. What happened to the seven months in between? Does this show a steady rise instead of a jump? Has anything else changed in this time? For example: the new system may not have started well and overlooked items or officer engagement was not what it should be, resulting in pressure from Borough Commanders down to record more accurately, leading to an apparent jump in incidents when it is actually a rise in adherence. Just ignoring a seven month gap is concerning. Why? An oversight or intentional?

 

Images are worth a thousand words. Misleading images are still far more impactful than poor descriptions. I reproduce this because it is a howler of a poor and misleading graphic.  The article is using the Financial Year for measurement, hence the mention of 2019 whilst we are in 2018.  Laying out the same images of London side-by-side implies that a comparison is about to be demonstrated.  However, the left hand image uses Westminster and mentions five other boroughs, none of which are referenced in the right hand image. This makes the image of questionable value, other than adding to the devaluation of credibility.  The source attribution should say that this is where the data was sourced, not presented in this way. I rather implies that this graphic is from the Met Police. It isn’t.

I think this was rather twisted to produce the graphics: Met Police Use of Force information.

howler of a bad graphic

Interestingly, the Met Police data gives this caveat, albeit buried on the third tab of their use of force stats page and not linkable by a URL. It explains areas where the data may be misinterpreted. The journalists don’t bother to tell us if they have taken this into account or not. We’ll never know.

CoversheetSo you can see that this article – and many stories on many topics – is riddled with inconsistencies. To me, I just dismiss it because it hasn’t got some of the basics right. It may be speaking a degree of truth, but that truth is devalued by the poor presentation.

Data is fine. Data is useful. Data is just digital exhaust. Data without context is just numbers and means nothing.

As I said previously, “Data only helps you take a problem apart and understand its pieces. It is not suited to put them back together again. The tool needed for that is the brain.”

Try to dissect stories quoting numbers. Be they in the press or someone making a commercial claim in order to influence your actions.

 

Here are some likely data sources for the story and for you to use when reading these type of stories using numbers to give credibility to their assertations.:

UK Government crime statistics

Metropolitan Police data

Office for National Statistics

UK Data Service video

 

PS: Anecdotally: I have known many types of officer. From 6’3″ tall Senegalese immigrants, who started as a PCSO in London and is now policing rural Oxfordshire, to short white people that are Reading natives who are born and bred and police there. They vary in their attitudes and actions because they are people. The huge majority want to make their communities better places. I do have no doubt that amongst them there are a few racist thugs, albeit a tiny and ever-decreasing amount. A bit like regular people I suppose. 

 

PPS: There may be typos. I try very hard to proofread. I am a terrible typist though.