Dirty Data

Westiminster recently feigned shock and horror that the UK's coppers cook the crime figures. But Simon Jenkins says we've known for years that the numbers are meaningless and they should be banned as "they spread confusion and fear".

But 'plod' is not alone in mis-classifying, mis-recording, ignoring or otherwise presenting data in a way that suits himself. We've had many financial trading scandals where banks apparently had no idea of the exposures they faced, either because transactions were concealed or perhaps no one was looking hard enough - the global financial crisis was a function of poor due diligence.

A possible root cause of the problem is that humans are involved too early in the data collection and reporting processes. Rarely are we responding to the 'raw' data, as opposed to figures that have been 'gathered' and 'rolled up' through a series of other people's filters, manipulations and interpretations (which are often taken out of context). It's puzzling why regulators' systems don't receive a feed of the actual trades straight from bank trading desks - or from peer-to-peer lending or crowdfunding platforms - rather than relying on periodic reporting of summary data.

Maybe GCHQ can help...

At any rate, we should focus more on 'clean' mechanisms for capturing and presenting raw data rather than someone else's interpretation of it.

Image from TraceyNolte.
 

What Happened To 'Class A' Political Journalism?

My appetite whetted by this week's local electoral melodrama, I've been searching for some Class A political journalism to feed my lust for pragmatism. 

There were little flashes of it from a few of the TV people. Michael Crick, who blew the lid off the Andrew Mitchell stitch-up, was rude as hell to Farrago, no doubt furious at having stuck to him like a leech in the hope of discovering anything coherent and coming up empty-handed. That left the usually mild-mannered Gary Gibbon to go after the rest of the gang. Desperation set in after the AutomEtonian responded to every single question with the line that this week was simply about local councils. He genuinely seemed to forget he was the Prime Minister, and I guess it's easy to see why. This seemed to put Gary in such a foul mood that he went after Flash Nick and Millibore like a mortar crew on speed. Each prevarication was interrupted with a fresh round down the tube, and another explosion of disbelief at the factually-twisted response. 

The only problem with the Gibbon assault was the apparent premise of the questions on capital spending: that it's the job of the state to fill every hole in the infrastructural landscape. Creating a whole new mountain range out of UK public debt is strange medicine indeed, whatever the cause. Ironically, Flash Nick went closest to a straight response, saying that while they'd barely invested a bean of new public money, the coalition has done a great job of attracting private capital to public projects. If that's true, then let's hope they've overcome the planning fallacy, and the PFI vultures leave a little flesh on the state carcass for the rest of us. 

As for Ed, well... 

In the end, the howling in my soul could only be quieted by re-reading "Fear and Loathing on the Campaign Trail '72". Forty years on, nothing has changed. The vicious wheels of the party political machines are still flattening the best interests of the citizens into the road in the rush for power and patronage, and Thompson's substance-fuelled take on the political animal is so brutally right that the recognition will make you laugh like a hyena. This, for example, could have been written today:

"This also reinforced my contempt for the waterheads who ran Big Ed's campaign like a gang of junkies trying to send a rocket to the moon to check out rumours that the craters were full of smack."

Now why doesn't anyone write about politics like that anymore?

Is it merely because today's journalists are sober, or have they abandoned hope that we can produce anything different to the current stage-managed pantomime?

Thatcher Failed To Make It Personal

Whether you loved or loathed her, you have to be impressed that 23 years after she was hunted out of office Margaret Thatcher's funeral is as divisive as a Poll Tax riot.

Clearly Britain has failed to 'move on' from the Thatcher years, which suggests to me that the work she started was on the right track but is seriously incomplete. I mean, if her policies had been just plain wrong-headed or disastrous, Britain would have dropped them like hot coals - or the notion of 'light touch' banking regulation. Instead, we're still trying to balance Thatcher's blast of economic reality with its personal and social impact.

Whatever your politics, it's clear from all the recent commentary that Thatcher was focused solely on improving the way the failing British economy 'works'. She spent her energy arguing relentlessly with people about the nature of the problems, their causes and the improvements that should be made to resolve them. The resulting policies obviously appeared 'right wing', but this was largely by comparison with the dogmatic lunacy espoused by the economic lemmings in charge of the Labour Party and trade unions at the time. Their policies seemed predicated on the private sector operating as a charity for the public sector, rather than economic sustainability. Thatcher's opponents were not arguing either on the same rational terms or with the same rigour. Her disciplined approach ruthlessly exposed dogma, from both left and right, and homed in on the most feasible economic solution. Then she rammed it home...

While Britain's reward was increased productivity and employment, far too many of its people were ill-equipped to cope with this fairly brutal brand of politics. Thatcher is infamous for the quote that "there's no such thing as society" which is often unfairly given without the qualification she gave it. But even the full quote reveals a serious flaw in her approach:

"They are casting their problems at society. And, you know, there's no such thing as society. There are individual men and women and there are families. And no government can do anything except through people, and people must look after themselves first. It is our duty to look after ourselves and then, also, to look after our neighbours."  Women's Own, 1987.

Thatcher's words "and then" raise the issue of when, which we naturally interpret as 'when we have enough for ourselves'. But enough is never enough. Our society is obsessed with personal rights and entitlements, rather than the duties and obligations which must be performed if those entitlements are to be delivered. After all, who ultimately bears the responsibility for delivering everyone's rights and entitlements if not each of us personally? Thatcher was right to the extent that the state cannot perform our personal obligations for us - ultimately, it can only act as a facilitator for our own endeavours - but it was a mistake to assume that society would automatically benefit if each of us looked after ourselves as a first step. Perhaps this was as much a flawed belief in the 'efficient markets hypothesis' as that of Alan Greenspan (and Gordon Brown) a decade later.

At any rate, we are now faced with the fact that, in Thatcher's own terms, we are not looking after a fairly large number of our neighbours. While it's worth noting that Thatcher's governments produced consumer-oriented legislation such as the first Data Protection Act (1984), the Hospital Complaints Procedure Act (1985) and the Consumer Protection Act (1987), it took British society several more decades to establish even a basic sense of 'customer service', and most of the UK's institutions are still not designed around the 'customer'.

In my view, we will continue to struggle with significant social imbalances until we grasp the idea that society and the economy only 'work' if each of us - whether acting as individuals or employees of corporations or the public sector - acts in ways that are sustainable for both ourselves and society at the same time. It's not a matter of looking after ourselves first "and then" our neighbours, as an afterthought. Our activities have to be aligned to be sustainable. And only by focusing on our duties and obligations to everyone else will we secure our own rights and entitlements. That is the fundamental concept behind what I would call “the Personal State”. It's time we built it.

Image from DelhiNewsRecord.

LSE Gets It: More Pragmatism, Less Politics

Having recently made the same point, I'm encouraged to see the London School of Economics setting out in detail some of the ways in which the UK could benefit if pragmatic political consensus were to replace party-political dogma. 

However, it would be wrong to think that this approach is only needed in the areas of education, infrastructure and innovation, on which the LSE's report focuses in particular. It's a general shift in attitude that is required in every aspect of our lives. 

This doesn't simply mean that politicians and civil servants should adopt a different top-down attitude. It means inverting the institutional narrative altogether. Politicians and the public sector must adopt a pragmatic, bottom-up view of what works and what does not work at the individual level, for the common good. The public sector must monitor and disclose publicly whether - and, if so, how - its activities, regulations and incentives distort all kinds of local and national markets in favour of private and public sector institutions, thereby constraining innovation and competition. Critically, this extends to the wasteful way in which the public sector purchases its own goods and services.

In practical terms, that shift in attitude requires the civil service and politicians to focus on obtaining data, defining problems, measuring their scale, analysing root causes and implementing lasting solutions. After all, hard choices are easier for more people to accept when they can be shown to be driven by harsh reality rather than party political dogma.

While, fortunately, there's plenty of evidence to suggest that this change is already underway as part of longer term trends discussed on this blog, the voices of institutions like the LSE are critical to those trends becoming mainstream behaviour sooner.  Let's hope similar reports follow from others shortly.

Pragmatism Grows At Night

In "India Grows at Night" the writer and commentator Gurcharan Das shares his insights into how India's growing, pragmatic middle class can achieve the country's necessary political and economic reforms. While inspired by Das's presence in Tahrir Square two years ago, these insights also resonate with the plight of Western democracies whose growth is inhibited by extractive private and public sector institutions.

The title of the book comes from Das's belief that India's knowledge economy powered her economic growth because: 

"Bureaucrats did not know how to regulate it and could not choke it with red tape, in the way they stifled India's industrial revolution through licences, permits and inspectors... India's knowledge economy literally grew at night while the government slept."

But India's problems are not over. Das explains that the "puzzle is... how can a vibrant democracy with a rising economy and an energetic civil society have allowed the state and governance to decay"?  He then describes the evolution of the Indian state from before British rule until today, tracing the tensions between social and official structures, and the shortcomings of the political system and key market failures.

Despite different starting points, this 'decay' also awaits Western democracies who have not been alert to the need for ongoing political and economic reforms. There's an ominous familiarity, for instance, in the complaint that the Indian state is preoccupied with the quantity of schools and other public services rather than their quality - "which is what really drives shared prosperity." The problems in our financial system are well rehearsed.

Das's description of the reasons for India's institutional decay is also echoed in Phillip Blond's explanation of the 'political bankruptcy' in Western countries. I understand them both to be saying that right wing policies allow the concentration of wealth amongst relatively few extractive institutions and their management and investors, rather than creating an environment in which widespread entrepreneurship can flourish. Meanwhile, left wing policies that are designed to 'redistribute' income through taxation and public spending are grossly inefficient by comparison to markets. The self-interest of partisan politics has gone too far, and legislators have no real commitment to the common good. Electoral battles fought along social and cultural lines distract everyone from critical long term issues, as well as being dangerously divisive. As a result, we lack appropriate regulatory frameworks and incentives to address market problems that stifle innovation and competition. Not only does institutional decay reflect the bankruptcy of dogma-ridden political parties, but as that decay constrains growth the economy itself drifts into liquidation.

Das argues that successful reforms will only be achieved through more active political participation by the members of the rising middle class, since they are the most conscious of the problems and the most impatient for the necessary reforms. He argues that the intransigence of existing Indian political parties creates the need for an entirely new, 'bottom-up', liberal political party. Das explains that this is a 'classical' rather than a 'social' liberalism - tolerant on social and cultural matters, yet wary of state intervention where the private sector and the market can be more effective.

This also seems to reflect the "renewed political idealism" and "participative democracy" for which Phillip Blond argues. 

In UK terms, this would seems to place Das's vision for a 'liberal party' somewhere between the Tories and the Liberal Democrats. And it seems quite telling that UK voters have forced those two political parties into coalition.

However, I disagree that the formation of a new political party or even a new political idealism is a necessary pre-condition for achieving political and economic reform.

As discussed in Lipstick On a Pig, the bottom-up approach that Das refers to has already been unleashed, largely enabled by the Internet's 'architecture of participation'. The 'Arab Spring' and developments in sub-Saharan Africa emphasise both the global nature of this phenomenon and its effective political impact. This process of 'democratisation' requires no more structure than the social media and a city square, and its power lies in the fact that it isn't confined to politics or economics. Greater transparency, knowledge and reform in one area creates the desire for change elsewhere. The result is both seismic and chaotic, yet significant reform is bound to be 'messy', not orderly and neat. As a result, I've suggested we're seeing the evolution of a "personal state" in which we're acting pragmatically as individuals in a highly collaborative fashion through the services of facilitators, rather than passively relying on our institutions to set the pace of reform.

New political parties and ideals might well emerge in this environment, but they will be a symptom of reforms achieved by each of us acting personally, not the cause.   

The Personal State

This decade is not going well for Britain’s institutions. The 2010 election did not magically restore our faith in a scandal-ridden Parliament. Bail-outs failed to improve the conduct of UK banks. Our public sector finances are in an appalling state. And as more sunlight has revealed the self-serving conduct of our mountainous bureaucracies, the gradual melting of our trust in them has become an avalanche. We want to know how rotten our institutions really are. More importantly, however, we want new models that work. 

As explained in “Lipstick On a Pig”, this plunge in faith in our institutions coincides with trends that are democratising the means of producing goods and services. Using digital technology we are personalising the one-size-fits-all experience traditionally offered by the likes of record labels, publishers, retailers, banks and political parties, and manufacturing our own physical products using desktop industrial machines. Rather than merely accepting what is ordained from the top down, both individually and as members of the ‘crowd’ we are shaping products, markets and political policies to solve the problems we encounter in our day-to-day activities. 

This process of ‘democratisation’ is being facilitated by organisations that are intently focused on helping us solve those problems. I call these organisations ‘facilitators’ to distinguish them from ‘institutions’, which exist to solve their own problems at our expense. The characteristics that I believe mark an organisation as being either a facilitator or an institution fall within broader themes of alignment, openness, flexibility, transparency and responsibility. In other words, a 'facilitator' solves its customers’ problems openly, flexibly and transparently, and takes responsibility for the impact of its activities on the wider community and society. 

Why are these features so critical? You might argue, for example, that focusing on ‘creating shareholder value’ or maximising management and staff compensation have proved to be more successful for some organisations than focusing on customers. As Anthony Hilton, Financial Editor of the Evening Standard, once said, “The City has done very well over the past 50 years dreaming up any old product and shoving it down peoples' throats.” 

But if that’s such a successful strategy, why are those City firms suddenly the subject of scandal after scandal and fine after fine for mis-selling and other misconduct? Why aren’t they able to recover quickly from their mistakes and move on? Why is Parliament labouring over new banking and financial services legislation? Why are people taking to the streets in protest? 

Because these firms are not 'facilitators'. 

In “Lipstick on a Pig” I explored the distinction between facilitators and institutions in the context of financial services, which then marked the latest consumer frontier. That sector also provides a great illustration of how organisations that produce complex products with hidden fees that their own staff can neither explain nor justify to customers become hooked on revenue and profits that disappear when the regulators finally wake up. How clubbing together with competitors leaves the whole club vulnerable to the same event or the consequences of the same mistake. How ignoring complaints and covering up problems leaves an organisation unable to understand the causes of issues it needs to fix. And how, when it finally emerges that the institution is not managed in the interests of the wider community, that community will no longer support it.

Since then, however, the frontier has expanded to confront the public sector and how society works – or doesn’t - as a whole. So I've been focused on the extent to which the public sector shares the same institutional characteristics that afflict our banks, and how facilitators are emerging in that wider context to help people solve their day-to-day problems that are being ignored. 

Whether an organisation is a facilitator or an institution is ultimately a matter of personal judgement for each of its customers. You might consider that a supplier is on the cusp of either category. Some will shift categories over time - although the drift from facilitator to institution appears to be easier than reform the other way. Some may never be reformed. Instead, they will gradually wither away while alternative models grow around them. 

Ultimately, however, the success or failure of our institutions and the facilitators that replace them is down to each of us. We are obsessed with ‘our rights’, but we must also realise that each of us bears responsibility for the wellbeing of everyone else. With our rights come duties and obligations that each of us must perform personally. The state cannot perform these obligations for us. The state can only act as a facilitator for our own endeavour. This is “the Personal State”. 

The Personal State is a simple concept. But it is of course a hugely complex dynamic, fraught with deeply-rooted life and death problems. For it to operate effectively, each of us must act pragmatically - in an informed way, rather than by adopting “uninformed, stupid practice”. That means no longer describing problems in terms of political dogma and propaganda. It means thinking critically and practically to identify and solve real problems. It means praising what works and explaining what doesn’t. It means spending, saving and investing our money in productive ways, and declining state benefits we don’t need. It means finding ways to improve the efficiency and productivity of the public sector to reduce public spending. Of course we must punish the gross mismanagement of our institutions and other violations of public trust. Yet we must also encourage entrepreneurs to engage in survivable trial and error, in order to promote innovation, competition and growth. In short, we must help each other wherever we can. 

Now a state like that would be worthy of some lipstick.

Image from Makeup Artist.

The Complex Job of Producing Simple Financial Services

There has been a futile tendency amongst regulators to view 'simple' retail financial services as merely 'basic' or 'vanilla' versions of existing products. And the Treasury's aspirations in this area have not improved, at least as of February 2012.

Perhaps ironically, the route to simplicity and transparency is much harder than producing a complex product that no consumer really understands. To produce a simple retail financial service involves first understanding the complexity of the consumer problem being addressed, then figuring out the simplest, most consumable service that will solve it. That's the role of a facilitator. By contrast, those producing complex products are unlikely to be focused on the consumer's problem in the first place, let alone understand it - they're focused primarily on solving their own problems at consumers' expense.

The path to simplicity involves disruptive innovation and critical thought to remove not only the complexity but also the intermediaries ('institutions') who've failed to solve consumers' problems cost-effectively to date. Trial and error, testing and learning, flexibility and adaptability are all key characteristics of this process. Yet our financial services framework is intolerant of them. In fact, a new service could launch and undergo several iterations in the time it takes to get it authorised by the regulators in the first place. Tiny factual differences have seismic regulatory implications in the type of permission or licence needed, and this adds to the time-lag and legal advice involved. We must figure out how to make the process of decent innovation easier.

Having waded through all this treacle to actually produce an innovative, simple product, there is then the challenge of explaining very simply how it works - and on a low budget. I recall Richard Duvall saying that £60m was spent on the launch of Egg, while the marketing spend dedicated to the launch of Zopa was £35k (though the phenomenal PR benefit from launching something truly new was priceless). But perhaps that isn't as much of a disadvantage as you might expect. While we see plenty of pointless, fluffy TV ads for expensive banking products, we don't see much marketing effort devoted to the 'basic' versions. As Professor Devlin found in his research for the Treasury, low fees and ease of switching has dampened traditional institutions' enthusiasm for creating and marketing simple stuff when there's so much money to be made from complexity and inertia.

At any rate, we need to celebrate the really simple, clear and transparent explanations of how our new financial services work. I've set out some examples below from my own experience. Some relate to services that are in beta or brand new, some established. To demonstrate the ability of TV journalists to explain things incredibly simply, I've also included just one piece of excellent coverage that removed the need for the business concerned to produce videos of its own.

Other top-tips on great explanatory clips are welcome.

Nutmeg




Abundance Generation




Elexu



Zopa



Funding Circle




PeopleFundIt

Short Churches?

Ever since protesters were forced by police to retreat from the London Stock Exchange to occupy St Paul's Churchyard, I've been fascinated by the effect of the global financial crisis on our Christian institutions.

While the Vatican has seized the opportunity to issue its statement on 'reform to the international financial and monetary systems', the Church of England, of course, was terribly embarrassed to be caught up in it all. Incapable of grasping a real opportunity to shape people's thinking, instead St Paul's initially offered to convene a nice cosy debate. Then the Cathedral's 'canon chancellor' resigned ahead of the Bishop of London's threat of eviction, which was followed shortly after by the resignation of the dean of St Paul's. Finally stirred into action, the Archbishop of Canterbury called for "robust public discussion" about the possibility of a so-called 'Robin Hood tax' on financial transactions.

The Vatican's statement is typically grand, and I've not had the time to consider it all, but here's an extract of some concrete proposals:

"a) taxation measures on financial transactions through fair but modulated rates with charges proportionate to the complexity of the operations, especially those made on the “secondary” market. Such taxation would be very useful in promoting global development and sustainability according to the principles of social justice and solidarity. It could also contribute to the creation of a world reserve fund to support the economies of the countries hit by crisis as well as the recovery of their monetary and financial system;

b) forms of recapitalization of banks with public funds making the support conditional on “virtuous” behaviours aimed at developing the “real economy”;

c) the definition of the domains of ordinary credit and of Investment Banking. This distinction would allow a more effective management of the “shadow markets” which have no controls and limits."

However, I wonder whether our religious institutions could be a bit more active in the reform of the financial system, rather than pontificating from the sidelines? Their wealth and tax-free status has not gone unnoticed, and there's plenty they can do on the investment front. The Church of England's ethical investment policy is here, for example. And it has lent stocks to short sellers. But that's not what I'd call active. 

Having previously suggested that short selling would be a useful regulatory tool, and that we could do with a secular version of the old Devil's Advocate, perhaps these are areas where the churches can help, along with voting at AGMs on executive compensation, for example. In fact, sometimes billed as the "shock troops of the Vatican" or "God's Marines", maybe there's a calling for highly-trained Jesuit priests on the trading desk of an ethical hedge fund, short selling the stocks of companies that the faithful believe are operating unethically. 

I wonder how they would rate Mr Blankfein's efforts?

Image from NJ.com.

Bad Data In...

The IMF seems to be having a bad year. First it was accused of 'groupthink'. Then it was rumoured (again) to be considering Gordon Brown as its leader.

Now it's The Economist's turn to sink the slipper. It points out that the IMF's forecast of China's current-account surplus assumes a steadily depreciating yuan, and a widening surplus. That provides ammunition for protectionists to impose tariffs on Chinese goods. Yet China's current-account surplus has actually declined since 2006.

But let's not just pick on the IMF. In "Botox and Beancounting", The Economist also points out the cosmetic effect of US official measures of government debt, productivity and economic growth compared to European measures. "The snag comes if investors fail to grasp that official national figures can show the American economy in an overly flattering light."

Of course, none of these is an isolated incident. There are also fundamental problems in comparing corporate financial data, for instance, given differing accounting standards.

And we live in a world where auditors are still trying to figure out what "scepticism" means...

But we don't have a problem detecting bad data. Plenty of people warned others about what Madoff's fraud, for example, and investment analysts routinely uncover issues such as The Economist has reported. Short-sellers make this their business. No, as David Einhorn elucidated in "Fooling Some of the People All of the Time", the problem is how to give the same weight of publicity to the prudent interpretation of the data as is given to the release of the data itself.

The media and social media clearly play a significant role. But even if we create an Office of the Devil's Advocate, ultimately each of us must accept responsibility for thinking critically about the data we're given if we're to avoid making some big mistakes.

Image from TraceyNolte.