EU Countries To Offer Their Citizens Digital Identity Wallets From 2026

The EU is finally pushing forward with aRegulation that requires member state governments to offer their citizens a voluntary European digital identity wallet from 2026.

Under the new law, member states will offer citizens and businesses digital wallets that will be able to link their national digital identities with proof of other personal attributes (e.g., driving licence, qualifications, bank account). Citizens will be able to prove their identity and share electronic documents from their digital wallets simply, using their mobile phones. 

 The new European digital identity wallets (EDIWs) will enable all citizens to access online services with their national digital identification, which will be recognised throughout the EU, without having to use private identification methods or unnecessarily share personal data. User control ensures that only information that needs to be shared will be shared.

For the past 20 years, various players have pushed the idea that you could have a digital identity issued by any number of certified 'trust providers' based on certain agreed standards. This would go hand-in-hand with concepts of 'personal data stores' and access to your transaction data in machine-readable form ('midata')that would allow you to control how your data is monetized. There was speculation that the trust providers would likely be banks and telecoms companies, or perhaps dedicated new entities; but there were always concerns about whether they really had the core competencies required - or the risk/liability appetite - as well as issues relating to security and privacy.

The European Commission explains that:

• by 2026, each member state must make a digital identity wallet available to its citizens and accept EDIWs from other member states according to the revised regulation 
• sufficient safeguards have been included to avoid discrimination against anyone choosing not to use the wallet, which will always remain voluntary 
• the wallet’s business model: issuance, use and revocation will be free of charge for all natural persons 
• the validation of electronic attestation of attributes: member states are required to provide free-of-charge validation mechanisms only to verify the authenticity and validity of the wallet and of the relying parties’ identity 
• the code for the wallets: the application software components will be open source, but member states are granted leeway so that, for justified reasons, specific components other than those installed on user devices need not be disclosed 
• consistency has been ensured between the wallet as a form of eID and the scheme under which it is issued...

Qualified website authentication certificates (QWACs) will ensure that users can verify who is behind a website under well-established eID security rules and standards (which enable open banking service providers to authenticate each other's systems, for example).

Is BigTech Still Battling The Entire Human Race, Or Just Some Of Us?

Readers will be familiar with my view that we consumers tend to be loyal to 'facilitators' who focus on solving our problems, rather than 'institutions' who solve their own problems at our expense. Previously trusted service providers can also lose their facilitator status, and I'd argue that Facebook has already done so (owing to privacy, electoral and extremist content scandals) and Google is firmly headed in that direction (through behaviour incurring massive EU fines). Yet, despite announcements designed to suggest increasing transparency, it seems BigTech is actively resisting independent human oversight and the perceived battle between computers and the human race is far from over...

Part of the problem is that 'BigTech' firms still operate as agents of retailers and other organisations who pay them vast amounts of money for exploiting our personal data targeting advertising at us, rather than as our agents for the purpose of finding what we need or want while shielding us against exploitation. In fact, this is the year when digital advertising spend will exceed spending on the old analogue 'meat space' channels. 

Combine that exploitative role with rogue artificial intelligence (AI) and you have a highly toxic reputational cocktail - particularly because AI based on machine learning is seemingly beyond human investigation and control. 

For instance, Amazon found that an AI programme used for its own recruiting purposes was terribly biased, but could not figure out what was going wrong or how to fix it, so had to simply shut the thing down.  Alarmingly, that suggests other AI programmes that are already notorious for being biased, such as those used for 'predictive policing', are also beyond fixing and should be shut down...

Many BigTech firms are appointing 'ethics boards' to try to avoid their AI programmes heading in inappropriate directions. Trouble is, not only is there doubt about what data scientists might view as inappropriate (which drove the appointment of ethics boards in the first place), but these boards are also generally toothless (only CEOs and main boards can decide the actual course of development), and tend to be populated by industry insiders who sit on each other's ethics boards. 

It is unclear, for example, whether the recommendations of the ethics committee overseeing the West Midlands police 'predictive policing' algorithm will be followed. Meanwhile, 14 other UK police forces are known to be using such AI programmes...

Another worrying trend is for AI firms to prevent investors voting on the company's plans, using "dual class" share structures that leave voting control with the founders rather than shareholders. Lyft is the latest to hit the news, but other offenders include Alphabet (Google), Blue Apron and Facebook, while Snap and Pinterest give shareholders zero control. Those firms might argue that stock prices are a check in themselves. But the stock market and investor greed are notorious for driving short-term decisions aimed at only maximising profits, and even giant regulatory fines are subject to appeal and can take a long time to be reflected in share prices. Voting power, on the other hand, is more qualitative and not simply a function of market forces - and the fact that it is being resisted tells you it's a promising tool for controlling BigTech.

Regulation will also be important, since fines for regulatory breaches are a source of revenue for the public sector that can be used to clean up the industry's mess and to send signals to management, investors, competitors and so on. I'm not suggesting that regulatory initiatives like the UK Brexidiot ToryKIP government's heavily ironic "Online Harms" initiative are right in the detail or approach, but Big Tech certainly cannot keep abdicating responsibility for the consequences and other 'externalities' associated with its services and approach. There has to be legal accountability - and grave consequences - for failing to ensure that AI and the firms themselves are subject to human control.

I guess the real question might be: which humans? 

Better Services For SMEs: Follow The Data

I was at a 'parliamentary roundtable' on Tuesday on the perennial topic of small business banking reform. A more official report will be forthcoming, but I thought I'd record a few thoughts in the meantime (on a Chatham House basis). 

It still seems to surprise some people that small businesses represent 95% of the UK's 5.4m businesses - 75% of which are sole traders - and that they account for 60% of private employment, most new jobs and about half the UK's turnover. So-called 'Big Business' is just the tip of the iceberg, since only they have the marketing and lobbying resources to be seen above the waves. As a result small businesses have long been a blind spot for the UK government - until very recently - and the impact has gone way beyond poor access to funding. It includes slow payment of invoices, the absence of customer protection when dealing with big business and lack of alternatives to litigation to resolve disputes.

What's changed?

A combination of financial crisis, better technology and access to data has exposed more of the problems surrounding SMEs - and made it possible to start doing something about them. And it's clear that legislators are prepared to act when they are faced with such data. The EU Late Payments Directive aimed to eliminate slow payments. The UK has created the British Business Bank to improve access to finance, as well as a mandatory process for banks to refer declined loan applications to alternative finance providers and improved access to SME credit data to make it easier for new lenders to independently assess SME creditworthiness. The crowdfunding boom has also been encouraged by the UK government, and has produced many new forms of non-bank finance for SMEs, including equity for start-ups, debentures for long term project funding, more flexible invoice trading and peer-to-peer loans for commercial property and working capital.  Last week, the FCA launched a discussion paper on broadening its consumer protection regime to include more SMEs.

Yet most of these initiatives are still to fully take effect; and listening to Tuesday's session on the latest issues made it clear there is a long way to go before the financial system allocates the right resources to the invisible majority of the private sector.

A key thread running through most areas of complaint seems to be a lack of transparency - ready access to data. This seems to be both a root cause of a lot of problems as well as the reason so many proposed solutions end up making little impact. But the huge numbers and diversity of SMEs presents the kind of complexity that only data scientists can help us resolve if we are to address the whole iceberg, rather than just the tip. That's surely one job for the newly launched Alan Turing Data Institute, for example, although readers will know of my fear that it seems more aligned with institutions than the poor old sole trader, let alone the consumer. So maybe SMEs need their own 'Chief Data Scientist' to champion their plight?

The latest specific concerns discussed were as follows:

• despite much regulatory activity, few bank reforms have actually taken effect - and many are being watered down - with the result that the banking picture is worse, if anything, than it was in 2008. Recent fines and scandals reveal little change in culture from that described in the report of the Parliamentary Commission on Banking Standards and most recently in the damning report into the failure of HBOS. In addition, banking is more concentrated (despite a few new entrants); less diverse in terms of ownership structure, competition, regional coverage and financial model; and still only about £200bn or 3% of bank assets are focused on SMEs and GDP producing activities. The trouble is we've known this for some time, but aside from the BBB's activity on access to funding, we still seem unclear on what can be done for SMEs at a regional or local level;

• the recent findings and remedies proposed by the Competition and Markets Authority into business current accounts are widely considered to be weak and unlikely to be effective - try searching the word "data" in the report to see how often there was too little available. The report still feels like the tip of an iceberg rather than a complete picture of the market and its problems;

• austerity imperatives seem to be the main driver for off-loading RBS into private shareholder ownership - the bank pleading to be left to its own devices (not what it suddenly announced to the Chancellor in 2008!) - and trying to kill-off any further discussion of using its systems as a platform for a network of smaller regionally-focused banks (as in Germany);

• bank lobbying has become more powerful - and will become even more so with the merger of their various mouthpieces - we need much more transparency on banks' access to government and the revolving door between the City and Whitehall/Westminster - we should have learned long ago that the interests of large banking groups are not aligned with SMEs (or consumers);

• SMEs lack an effective means of resolving disputes with banks, as the Financial Ombudsman Service has limited jurisdiction; banks refuse to release internal reports related to credit decisions and calling in loans; while the FCA is said to be hiding behind its confidentiality obligations; and banks have too much legal fire power for SMEs to take them on in court - hence moves to take matters into their own hands. The FCA has clearly woken up to this, but it's recent discussion paper shows there's a huge gap to close.

• the financial infrastructure for SMEs appears not to be geographically diverse - it doesn't yet mirror the Chancellor's "Northern Powerhouse" policy, for instance - despite calls for bank transparency on the geographical accessibility, a US-style "Community Reinvestment Act" and clear reporting on lending to SMEs by individual banks (rather than the Bank of England's summary reporting). There's a sense that we should see some kind of financial devolution to match political devolution, albeit one that still enables local finance to leverage national resources and economies of scale. Technology should help here, as we are tending to use the internet and mobile apps quite locally, despite their global potential;

• Some believe that SMEs need to take more responsibility for actively managing their finances, including seeking out alternatives and switching; while others believe that financial welfare should be like a utility - somehow pumped to everyone like water or gas, I assume - indeed regional alternative energy companies were touted as possible platforms for expanding access to regional financial services. My own view is that humans are unlikely to become more financially capable, so financial and other services supplied in complex scenarios need to be made simpler and more accessible - we should be relying less on advertising and more on hard data and personalised apps in such instances.

• Meanwhile, SME are said to lack a genuine, high profile champion whose role it is to ensure that the financial system generally is properly supportive of them. This may seem a little unfair to the Business Bank, various trade bodies and government departments, but it's also hard for any one of these bodies to oversee the whole fragmented picture. As I suggested above, however, I wonder whether a 'data champion' could be helpful to the various stakeholder in identifying and resolving problems, rather than a single being expected to act as a small business finance tsar. 

 In other words, we should follow the data, not the money...

Artificial Intelligence: The Control Dichotomy

Professor Nick Bostrom delivered an inspirational speech for the SCL last night on "Superintelligence: a godsend or a doomsday device", although few would have found it reassuring - it is certainly conceivable that machines could become more intelligent than humans and that humans might not be able to control them. But these are still early days, he stresses. Regulating the development of artificial intelligence at this point risks halting progress. There's a lot more work to do to really understand how artificial intelligence will develop beyond playing old video games better than humans or recognising an image as the picture of a cat. We need to consider how the technology could help avert our extinction, as well as how it might wipe us out. Yet Nick also warns that it will take far less time for computers to exceed human level machine intelligence than to reach it in the first place. So we need to start work on the control mechanisms for the development and use of artifical intelligence, without regulating the industry out of existence: the control dichotomy.

Nick suggests that the guiding principle should be that of "differential technological development" - diverting resources away from technologies and their application which could cause human extinction while focusing on those which will either help prevent our demise or will facilitate the expansion of the human race throughout the cosmos.

But how do we distinguish between helpful and harmful technologies and their application? 

As Nick points out, it's tough to think of any human invention that is inherently 'good'. He mentions many things, from gun powder to genetic engineering, and I think we can throw in the wheel and the Internet for good measure. All these things are used by humans in bad ways as well as for the greater good. But I think what horrifies us especially about the idea of Superintelligence or 'The Singularity' is that it will be machines, not bad humans, who will be using other machines against us. And while we have lots of experience in dealing with evil humans, even our top minds admit we still don't know much about how machines might act in this way or how to stop them - and what we humans fear most is the unknown. 

You'll notice I haven't said 'evil' machines, since they might not be operating with any evil 'intent' at all. Human extinction might just be a mistake - 'collateral damage' arising from some other mission. For instance, Nick suggests that a particular machine left to itself in a given situation might decide to devote itself entirely to making paperclips. So, presumably, it would not bother to put out a fire, for example, or free a human (or itself) from a burning building. It might leave that to other machines, who might in turn have chosen their own narrow objective that involves ignoring people's screams.

Here's where I struggle with the notion of Superintelligence. In fact, as someone who hates being pigeon-holed into any single role, I think a machine's decision to only ever make paperclips might be fabulously logical and a brilliant choice for that machine in the circumstances, but it makes the machine as dumb as a post. For me, Superintelligence should involve a machine being able to do everything a human can and more. 

But that's beside the point. Knowing what we know already, it would be insane to ignore the paperclip droid and wait for artificial intelligence to develop a machine more capable than humans before figuring out how we might control it. Nick is right to point out that we must figure that out in parallel. In other words, the concept of human control has to be part of the artificial intelligence programme. But it won't be as simple as coding machines to behave protectively, since machines will be able to programme each other. For instance, Nick suggests we could put the machines to work on the control problem, as well as on the problem of how to ensure the survival of our species. AI labs might also pay insurance premiums to cover the damage caused by what they develop. He was less certain about what we might do to constrain developments that occur in the context of secret defence programmes or intelligence gathering, but he seemed confident that we could at least infer the pace of development from the results, and be able to consider how to control the wider application of those developments. Mmmm.

At any rate, Nick also warns that we need to be careful what we wish for. Mandating human survival in a prescriptive way - even a specific biological form - would be a bad move, since we should not assume we are in a position to foster positive human development any more than the Holy Office of the Spanish Inquisition. Better to embed positive human values and emotions or, say, entertainment as a feature of intelligent machines (although I'm guessing that might not go down well with the jihadis). From a phyiscal standpoint, we already know that the human body won't do so well for long periods in space or on Mars, so some other version might need to evolve (okay, now I'm freaking myself out).

To retain a sense of pragmatism, at the end of the speech I asked Nick what he would recommend for our focus on 'Keeping Humans at the Heart of Technology' at the SCL conference in June. His tip was to consider which of the various types of control mechanism might work best, recognising the need to avoid constraining the positive development of artificial intelligence, while ensuring that we will be able to keep the machines in check if and when they become smarter than us.

No pressure then...

Will Machines Out-Compete Humans To The Point of Extinction?

I've been a bit absent from these pages of late, partly pulling together SCL's Technology Law Futures Conference in June on 'how to keep humans at the heart of technology'. As I've explained on the SCL site, the conference is part of SCL's effort to focus attention on that question all year, starting with a speech by Oxford University's Professor Nick Bostrom on 2 March: "Superintelligence: a godsend or doomsday device? 

In other words, last year was when the threat of "The Singularity" really broke into the mainstream, while this year we are trying to shift the focus onto how we avert that outcome in practical terms. 

My own book on how we can achieve control over our own data is still ping-ponging between agent and publishers, but will hopefully find a home before another year is out - unless, of course, the machines have other ideas... 

Humans, Not Big Data, Must Benefit From Opening Up The Banks

One of the 'good news' items in last year's Autumn Statement was the endorsement of an Open Data Institute report on how innovators could make better use of bank data on our behalf (instead of on the banks' behalf). Last week, the Treasury followed up with a call for evidence "on how best to deliver an open standard for application programming interfaces (APIs) in UK banking and... whether more open data in banking could benefit consumers."

This is important, because enabling our own machines to start crunching our financial data on our behalf is key to humans winning the race against Big Data.

'Open data' involves enabling public access to data by connecting data sets using uniform resource identifiers ('Linked Data') and publishing data about them in machine-readable formats. While the government and other public institutions have done a good job of opening up public data sets so far, our private institutions are lagging behind, as discussed here. That's because they benefit from making sure we know less than them about our own requirements - 'information asymmetry'.

The Midata initiative was the first to aim at restoring the balance in favour of consumers, by taking aim at energy utilities, telecoms providers and banks. Now the energy companies face being opened up like so many tin cans by smart metering, while the telecoms market looks like it will go from bad to worse. Banks have also proved a tougher nut to crack. Yes, some people can download their current account data via internet banking. But in addition to consumers, there are 4.5 million small businesses out there that need a hell of a lot more than that. So far, it's taken regulatory action just to start the process of improving access to small business credit data and making a market in rejected small business loan applications. Now the Treasury is trying to force the pace on the technology necessary to support all that.

The ODI's recommendations for opening up banking are: 

• banks should agree an open API standard to support third party access to bank data - basically firms that can help you make sense of the data (but Big Data firms will try to persuade you to share it with them too);
• independent guidance should be provided on technology, security and data protection standards that banks can adopt to ensure data sharing meets all legal requirements; 
• an industry wide approach should be established to vet third party software applications and publish a list of vetted applications as open data - this would allow visibility of firms that are acting on consumers' behalf and those who are not;
• standard data on Personal Current Account terms and conditions should be published by banks as open data; and 
• credit data should be made available as open data.

My main concern is that requiring agreement on 'standards' as a precondition for opening up banking will enable the banks to delay the whole process by a decade - as they did with Faster Payments. As was recommended by the security working group in the Midata initiative, I would prefer to see banks required to immediately make their data available to each consumer/SME in whatever open format the banks choose, while adhering to common data security protocols, and leave it to the open data community to figure out how to re-format and display whatever rubbish they dish up.

Apart from complaints by the banks, Treasury officials expect to hear positive contributions from consumer groups, other financial services providers, financial technology firms and app and software designers.

Let's not disappoint them. This is a good opportunity to ensure the government clears the way for innovation that puts you and me at the heart of financial services, without mistakenly creating further barriers in the process.

Google Switches To Defence In Its War On The Human Race

Nine months after Google's Chairman, Eric Schmidt, used his speech at Davos to declare war on the human race, he and the other  Big Data commanders find themselves very much on the defensive.

"I was suprised it turned this quickly," Mr Schmidt is quoted as saying of the political tide, after his European smarm offensive in June failed to avert calls for Google to be broken-up.

The trouble is that Big Data funds itself by selling the opportunity to find humans and present advertising to them. Even the craze in wearable devices is all about geolocating the wearer (and potentially their companion(s)) for advertising purposes. Ideally, you'll buy a watch or pair of glasses that will keep you reading ads and search results while on the move, but a wristband that tells your 'friends' what you're doing and where will do just nicely. Maybe one day you'll even go for the driverless car, so you can watch ads instead of the road.

As I mentioned in January, the advertising revenue that initially helped fund the transition from the analogue/paper world now dwarfs the value we actually get from Big Data and the Web. Mutuality - and humanity - is being sacrificed in the Big Data rush to sell you tat. Oh, and in the quest for The Singularity, when the high priests of SillyCon Valley believe that machines will achieve their own superintelligence and outcompete humans to extinction. Yes, really. 

In the same way that banks have grown from their mutual origins to suit themselves at our expense - keeping most of the 'spread' between savings and loans to suit themselves - Big Data platforms are primarily focused on how to leverage the data you generate ("Your Data") without rewarding you for the privilege.  GCHQ and the NHS are playing pretty much the same game.

But not all digital platforms finance themselves by using Your Data as bait for advertising revenue. Since eBay enabled the first person-to-person retail auction in 1995, that model has spread to create marketplaces in music, travel, communications, payments, donations, loans, investments and personal transport. The marketplace operators thrive by enabling many participants to use their own data to transact directly with each other in return for relatively small fees, leaving the lion's share of each transaction with the parties on either side. 

The marketplace model also reveals that most of daily transactions could be carried out between our machines. After all, they are much better at crunching all the data than we are. They are in the best position to combine our own transaction data, open public data and commercial product information to recommend the right car, mobile phone tariff or insurance products, without disclosing our identity to every advertiser in the process.  And why couldn't they arrange it so you switch to the cheapest phone or energy tariff each day, or switch car insurers depending on time of day or where your driving?

True, the platforms that enable you to leverage your own data more privately haven't yet attracted investors to the same extent as Big Data. eBay is solidly profitable and doesn't depend on substantial advertising revenues for its existence, yet it has a lower market capitalisation than Facebook or Google. It should come as no surprise to you that Wall Street and the world of high finance attaches a lower value to democratic and sustainable business models that don't suit a short term, institutional view of the world. But the financial news of 2014 must show institutional investors that we humans doubt whether Big Data has our best interests at heart. So the stock market value of marketplace operators may yet exceed that of the Big Data boys.

That's not to say that the whole Big Data movement has been a wasted experiment - it has just strayed from the path of simply digitising our daily experiences to trying to exploit them. Much of their technology could be re-aligned to empower you as an individual user, rather than treat you like a farm animal for the benefit of advertisers. 

Neither should we underestimate the Big Data giants' ability to reinvent themselves for the better. They are well-funded and more responsive to customers than banks and other institutions which have lost their way.

And it would be good to know they're working to sustain the human race, rather than kill it off.

Twitter Gnip Shows Why Social Media Should Share Revenue With Users

Source: Financial Times

Like Google's declaration of war on the human race, the news that Twitter will buy Gnip illustrates why social media platforms should share their Big Data revenue with users. Indeed, they would seem to have no choice if they are to survive in the longer term.

Gnip's CEO claims that:

"We have delivered more than 2.3 trillion Tweets to customers in 42 countries who use those Tweets to provide insights to a multitude of industries including business intelligence, marketing, finance, professional services, and public relations."

And that's not all. Gnip also has "complete access" to data from many other social media platforms, including WordPress, the blogging platform, and more restricted access to data from other platforms, such as Facebook, YouTube and Google+. 

Quite whether users consent to all that is an issue we'll return to in another post shortly. 

Meanwhile, Twitter suggests that Gnip's current activities have "only begun to scratch the surface" of what it could offer its Big Data customers in the future. Yet, from a user's perspective, Twitter has barely changed since Gnip began its data-mining activities. So are users receiving enough 'value' for their participation to keep them interested?

The social media operators would argue that their platforms would never have been built were it not for the opportunity to one day make a profit from users' activity on those platforms. And it may look like the features have not changed much since launch, but part of the value to users is the popularity with other users and it costs a lot to keep each social media platform working as the number of users grows. Each platform also has to keep up with changes to other platforms so users can continue to share links, photos and so on. That means platforms tend to lose a lot of money for quite a long time, as the FT's comparison chart shows. 

But analysing the value to users gets mirky when you consider that the social media are already paid to target ads and other information at users based on their behaviour, and that the cost of that type of Big Data activity is reflected in the prices of the goods and services being advertised. 

And it doesn't seem right to include the cost of buying and operating a separate Big Data analytics business, like Gnip, in the user's value equation if the user doesn't directly experience any benefit. After all, that analytics business will charge corporate customers good money for the information it supplies, and the cost of that will also be reflected in the price of goods and services to consumers. 

In other words, social media's reliance on revenue from targeted advertising and other types of Big Data activity means that social media services aren't really 'free' at all. Their costs are baked into the price of consumer goods and services, just like the cost of advertising in the traditional commercial media.

And if it's true that the likes of Gnip are only just scratching the surface of the Big Data opportunities, then the revenues available to social media platforms from crunching their users' data seem likely to far exceed the value of the platform features to users. 

Yet user participation is what drives the social media revenues in the first place (not to mention users' consent to the use of their personal data). The social media platforms aren't publishing their own content like the traditional media, just facilitating interaction, so there's also far less justification for keeping all the revenue on that score. And it seems easier to switch social media platforms than, say, subscription TV providers. 

So the social media platforms would seem to have no choice but to offer users a share of their Big Data revenue streams if their ecosystems are to be sustainable.

Six Years On And Pragmatism Has A New Frontier

I see this blog has reached the ripe old age of six, so I felt compelled to squeeze in at least one post to celebrate.  

It's fitting that the reason for my absence has been the need to get to grips with the FCA's proposals to regulate P2P lending and investment-based crowdfunding - not to mention the revelations concerning the Chairman of the Co-op Bank. After all, this blog set out to chart the rise of facilitators who help us wrest personal control of our day-to-day lives from the one-size-fits-all experience imposed on us by our institutions. Rumbling the 'Crystal Methodist' marks the continuing plunge of faith in those same institutions, while the decision to finally let the 'crowd' into the regulated financial markets shows that even Parliament recognises you and I are better off dealing with each other directly than simply entrusting our life's savings to the banks and investment funds.

Of course, these are just a few examples of the punishment being doled out to our financial institutions. And they aren't the only ones under pressure from the trends sweeping society, as we struggle to figure out a more sustainable form of capitalism. All our institutions, from the BBC to the Police to the Church, unions, political parties, government departments and so on, face the choice of becoming facilitators or withering away. 

So is there anything 'new' to write about? 

Six years on we are still seeing the dawn of where these trends will take us. But to get a sense of the future, I've been following the rise of 'open data' - or open access to data in machine-readable form. This marks a new frontline between institutions and facilitators. Big Data vs You. Not only has it already created new facilitators, in the form of "personal data stores" or "personal information managers", but it may also redefine some of today's facilitators as the institutions of tomorrow... 

As a taste of things to come, last week a senior advertising executive insisted to me that "Big Data can accurately predict human behaviour." To be fair I made him repeat the assertion in case it had slipped out by accident. No one else at the table seemed to find that truly weird, and it wasn't until the end of the week, when I met up with some people working at the sharp end of data gathering, that I was able to fully enjoy the hilarity of that statement.

This is going to be fun.

Image from Data.gov.uk

The Future Is Not Behind Us, It Lives Locally

For every old saying there's an opposite. Today's conflict lies in the adage that "those who don't know history are destined to repeat it." Yet "our history is not our destiny." This leaves a fine line between useful historical insight into how the world works and steering solely by what we see in the rear-view mirror. The distinction becomes critical as we lurch ever more quickly from one financial crisis to the next.

So what lessons from recent history will help us move foward, and which should go in the scrapbook? Here are 5 that I think are worth taking forward, in no particular order:

1. Clearly, the Internet is not a fad. Consumers are the winners, aided by facilitators in their battle against our creaking institutions. Yet e-commerce is still only 10% of all retail. And we are still in the 'primordial soup' phase in the evolution of our tools for extracting meaning from the great, chaotic swirl of data. So this trend has a lot more mileage in it yet and will sweep across more service sectors - if you can buy it online and have it delivered it won't be sold in high volumes on the high street. In fact, it might even be made locally...

2. In addition to democratising services, the Internet is also returning the means of production to local communities through e-enabled machines. Remote/home-working is replacing central workplaces and 'factories' are getting closer to customers, requiring a rethink of corporate systems, processes and supply chains. Huge production plants and sole-occupancy office towers will gradually become a thing of the past. High streets may well regenerate to support this trend, or as a result of it.

3. Cities with at least 3 private workers for every public sector worker see the most growth. Give that some thought if you live in a city near the bottom of this chart. This chimes with findings that economies (regional and national) whose public spending exceeds 35% of their GDP struggle to grow. You have to feel sorry for Northern Ireland...

4. The public sector is very inefficient and needs to act locally - public institutions are already being relentlessly affected by lessons 1 and 3 above, and 2 should increase  the pressure. Civil servants really have no alternative but to spend public money more wisely. Meanwhile, we'll help drive down the cost of government by dealing with the government online. Big government offices must surely go eventually. Fortunately, the moves to devolve more power to local government coincide here, but I don't think we should be fooled into thinking that's part of any real plan to future-proof the UK...

5. The UK's financial system is seriously inefficient at allocating money to people and businesses. The fact that we all rely heavily on a few major banks for whom lending to small businesses is not a core activity is part of the problem, but innovation and competition are constrained by our outdated and rigid regulatory framework and the related incentives. Crowdfunding, or peer-to-peer finance, platforms are springing up all over the country, and are increasingly focused on specific sectors, activities and locations.

In short, if I were a civil servant (see 4) based outside the south east of England (3), I would start an Internet-based service (1) that efficiently provides low cost finance (5) to help localise the means of producing stuff using e-enabled machines - or buy my own 3D printer and start renting it out (2). Now. 

Image from KC Anderson.

Open Data Spiders?

Since 2009 I've hoped that the semantic web - that is my computer dealing with suppliers' computers - would replace the need for price comparison sites. Following a discussion last night at the CtrlShift Explorers' Club, I'm confident that we don't have much longer to wait.

If suppliers publish their product data in computer-readable format, I could then programme an application or 'spider' to search the provders' open systems to find the product that's right for me - ideally a bespoke product assembled from a menu of optional components. This spider would use my personal data to conduct its search without disclosing that data to any product providers, at least until the time of purchase (and disclosure might not even be necessary then). It could also collect, say, public sector Open Data related to my desired activity, and analyse it in the context of my relevant personal transaction history. This could vastly improve my choice of car, holiday or home improvement and how it's financed. Or it could save me money by keeping me on the right energy or mobile phone tariff.

This is not about 'intent-casting' or 'demand-casting' in order to encourage suppliers to send me thousands of offers. My spider would not announce to the world that it's looking for anything. It would simply run around the web looking at openly available product codes and report its findings to me. Ideally, the product provider will have no idea that it's actually me who's looking until I make a purchase, if ever.

And I would not need to read any screens or physically enter any data until my spider reported its findings - or it could save me the trouble by calling my mobile.

In a machine-to-machine world, the marketing challenge is to ensure that anyone's 'spider' can always find your product data, and that data is accurate and up to date. Perhaps it could be somehow 'spider optimised', but it seems to me it's the job of the spider developers to make sure the spiders are good at finding product data, even when it's in a sorry state.

My sense is that an Open Data approach to the market takes such a different corporate mindset that it is unlikely to sit comfortably within traditional suppliers, where "Big Data" is the latest buzzphrase. In the Open Data world the challenge is to enable your products to be directly embedded in the ecosystem, helping to solve problems as customers encounter them and their machines or 'spiders' look for an answer. The traditional product approach is not 'connected' in that way, or at all. And, as I suggested recently, "Big Data" approach to behavioural targeting of advertisting seems fundamentally hamstrung by the fact that personal behavioural data is highly contextual and not really 'predictive' from one scenario to the next. Why spend all that money on what is ultimately a shot in the dark?

Those who ignore the Open Data option could well be spending their way rapidly into oblivion. 


Image from Data.gov.uk

Will Consumer Transaction Data Drive New Online Marketplaces?

I should begin this post by explaining that I'm involved in the Interoperability aspects of the midata programme. I was invited to participate on a voluntary basis and have no client in that process. I donate my time. I'm independent of the dozens of other participants. I didn't shoot JFK. And I don't even own a pet, let alone one I believe to be a reincarnation of Elvis Presley. 

The only conspiracy in which I might be accused of involvement is the mass collaboration by consumers known as Web 2.0, which has evolved into Web 3.0 using "linked data" that computers can read (aka "the semantic web"). Lots of information - including government data and data from online bank accounts - is now available in this format because it makes analysis so much easier. Analysing the data is necessary to convert it from being merely information into useful knowledge. This is why the government is keen that banks, telecoms providers and big energy start making all your transaction details available to you in machine-readable form - if you want it. Gaining insight into your finances, communications and energy use will enable you to make better spending decisions and even negotiate new, bespoke products - if you want to do so.

In an explanatory post on the Which? website, Consumer Affairs Minister Jo Swinson sparked a number of comments by people who do not want to be empowered or have their consumer experience made more efficient. They are happy to make their own product searches and to use price comparison services. Evidently, they are not concerned at the number of problems detected by the Office of Fair Trading in its work on the price comparison sector. They also appear to believe that information security and privacy safeguards around existing transaction databases are adequate. I am not amongst them.

It's perhaps unfortunate that this debate seems to be centring on price comparison services, when it's really the mainstream product providers who are to 'blame' for the lack of consumer bargaining power in key markets. In fairness these services have evolved into a prime marketing channel through being more nimble, internet savvy and committed to transparency than the mainstream product providers themselves. This has rendered the services useful to consumers up to a point, but they are limited by their deals with the product providers as to how far they can really empower the consumer. In this sense they occupy the battleground created by consumer rebellion and institutional resistance.

I have expressed my own frustration with the current model of price comparison service for many years. That so many engage in extensive television advertising campaigns tells you that being a price comparison service provider is a really great business to be in. But consumers are paying for those big advertising campaigns in the same way they're paying for vacuous bank advertising - through the price of the products they buy, which in turn generates commission and/or ad revenue for the comparison service providers. So, while these services are intended as a tool for consumers to use, it would be naive to assume that price comparison service providers are acting on the consumers' behalf. Product providers are paying good money to ensure these marketing channel ultimately work for them, not you. As a result, these services are not so much about 'price comparison' as simply 'comparative advertising'.

Critically, however, the current crop of price comparison sites also only rely on the entry and/or display of personal and product information in human readable form. They have you by the eyeballs, at least for as long as you're able to keep your eyes on the screen. Yet the product providers are able to rely on their computers mining and analysing a wealth of yours and other customers' transaction data to work out the most profitable product to offer you. True, in some scenarios - like insurance - this 'information asymmetry' appears to favour you, but product providers have giant data sets that can overcome any advantage you think you might have. And when in doubt they simply charge you more. In The Undercover Economist, Tim Harford shows how information asymmetry works to our detriment in buying a car - when the dealer has all the information - as well as in the market for health insurance, because we have the information and not the supplier.

So while a price comparison service might enable you to buy a cheaper traditional product from one provider versus another, you are by no means able to negotiate from a position of real knowledge about the product or price that's right for you.

This makes a human-readable interface an enormous waste of your personal time for the tedious yet important task of ensuring you spend your money wisely. Instead, our own computers should be analysing our transaction data and interrogating product providers' systems directly, not only to find a product at the right price, but to create the right product for the right price. This might be as simple as relying on your rate of energy use to always ensure you're on the right tariff, even if that means switching providers daily. Or it may open up opportunities for collaborative consumption amongst consumers with whom you share similar interests or behaviours.

Of course, most of us won't have the time, skill or resources to do all this for ourselves, anymore than we can build our own cars. So new intermediaries are springing up to store and/or crunch the data for us. I'm not going to name them because I don't want this post to be perceived as some kind of advertisement. These intermediaries have been variously called 'data stores', 'personal data vaults', 'personal information managers' and, most recently for the purposes of the Midata programme, 'midata stores' and 'midata service providers'.

The services provided by these intermediaries vary according to whether they just store, display and/or transmit the data at your request without otherwise processing it; whether they receive data from you or your current supplier; and whether they analyse the data or combine it with other data to produce a result on which you might rely to purchase an alternative product or change your behaviour in some way. 

These intermediaries need contracts with consumers that permit them to access the consumer's transaction data with relevant permissions restrictions, and which agree some form of remuneration. Such contracts would need to go further than the basic service terms of price comparison sites. You might allow them to receive a disclosed commission from a product provider, but at least this would be transparent to you. This marks the line between whether the intermediary is acting for you or the product provider, and there will need to be clarity on this point.


Some people say they don't want suppliers to store any of our transaction data. They want it deleted as soon as it's no longer needed. But it should be clear by now that your transaction history could be very valuable to you, and you should have the option of downloading and storing it and giving it to another provider. Most businesses insist on such 'data portability' when moving from one outsourcing service provider to another, for example, and this can be just as important for consumers and small businesses.

Identity and authentication are also important features of a world in which transaction data is being transmitted. Perviously, I've suggested that proof of identity should be momentary, based on much wider behavioural data than just the static datasets that fraudsters can replicate, and the data used to establish your identity should be discarded straight away, rather than held for re-use. But that is not to say that transaction data itself should be deleted.

The impact of all this on mainstream product providers cannot be underestimated. I've previously drawn the distinction between 'facilitators' who exist primarily to solve customers' problems and 'institutions' who exist primarily to solve their own problems at their customers' expense.  Clearly this new environment will favour product providers who are aligned with consumers' day-to-day activities rather than those who make life awkward because it suits their own profitability. A gulf may well begin to open between product providers as they are tested by this distinction. More confident consumers should be more prepared to spend money with facilitators than those faced with institutions they distrust.

However, I believe that this trend will most likely result in a series of digital platforms on which consumers and suppliers in various markets directly negotiate products and pricing in a transparent way, based on each consumer's transaction data, wherever that is stored. Such platforms have already arrived in so many other consumer markets as part of the Web 2.0 phenomenon, that it's only a matter of time that they arrive in the markets targeted by the Midata programme in any event. And it's only natural that consumers would want to leverage their own transaction data in that context.

Will Midata Turn Institutions Into Facilitators?

The government's warning shot over Midata presents an interesting challenge for some of the UK's institutions. But will it make them focus on solving consumers' problems - transforming them into 'facilitators'? Or will they merely continue to solve their own problems at consumers' expense?

The government wants the suppliers of energy, mobile phones, current accounts and credit cards to provide each of their consumer and small business customers with the records of what they bought, where and for how much. That transaction data must be released in computer-readable format to enable it to be analysed, either by the customer or the customer's authorised service provider. This would help prevent those suppliers from gaining an unfair pricing advantage over consumers, for example, and make it easier for consumers to figure out the product right for them.

Factors the government might consider in deciding whether to expand the programme to other sectors include: 

• the market is not working well for consumers, e.g. consumers find it difficult to make the right choice or their behaviour affects pricing it's difficult to predict that behaviour;

• there's a one-to-one, long-term relationship between the business and the customer, with a stream of ongoing transactions;

• consumer engagement is limited, e.g. low levels of switching or competition; and

• suppliers don't voluntarily provide transaction/consumption data to customers at their request in portable electronic format.

Yet these factors merely hint at the characteristics that an organisation should display if it is to succeed in the future economic environment. In broad terms, the targeted institutions will need to be organised to solve their customers’ problems, operate openly, adapt well to changing circumstances, remain committed to transparency and take responsibility for the impact of their activities on the wider community and society. I've explained these themes in more detail here.

 

The current targets of this programme have a long way to go!

 

I should add that I am involved in the Midata programme, as a member of the Interoperability Board and on the working groups considering issues related to data transmission and law/regulation.

Unload The "Digital Wallet" Before Someone Gets Hurt

And that's not all...

The term "e-wallet" or "digital wallet" has always caused a physical reaction. But what started as a small twitch over my left eye in November 1999 now involves diving under a table. The term has become so loaded with giant concepts like 'identity', 'privacy', 'authentication', 'security', 'payment' and 'funds' that it's simply too dangerous to wave around in meetings.

We need to focus on more of the detail if business presentations are to have any meaning and projects are to deliver anything.

The term 'digital wallet' is impossible to define, anyway. The Oxford English Dictionary has no home for it, and it's wise to ignore suppliers' self-serving, product-specific definitions. Th'internet merely yields a confusing mish-mash: [my emphasis] "a system that securely stores users' payment information and passwords..." (investopedia) and "encryption software that works like a physical wallet during electronic commerce transactions." (webopedia). Unhelpfully, the Free dictionary explains "the wallet data may reside in the user's machine or on the servers of the wallet service. When stored in the client machine, the wallet may use a digital certificate that identifies the authorized card holder." 

Such definitions are confusing because they keep jumping the rails from party to party, feature to feature and function to function, each of which has different implications for transaction flows, data flows and funds flows (to the extent payment is even involved). 

Perhaps the only consistent aspect in the use of the term 'digital wallet' is the sense that it refers to a specific individual, or at least it should be capable of doing so. Otherwise, the term means so many different things that it's useless. FinVentures defined it to mean, "A consumer owned and controlled account that can store any electronic form of what is normally held in a physical wallet, including: payment, ID, coupons, loyalty, access cards, business cards, receipts, keys, passwords, shopping lists, …etc." Indeed, a 'digital wallet' could be a feature within an application or service, or an entire application or service, a database, a set of permissions and so on. It could reside on virtually any digital device, including a smart card or just a microchip. It could enable a specific person to initiate or conclude any kind of transaction, or merely be used in the course of intiating or concluding such a transaction.

So when you next hear the term 'digital wallet', seek cover behind a large, heavy object and try to defuse the situation by asking: 

• which parties are involved;

• which party is agreeing to do what, how do they agree, what actions are taken as a result and by whom;

• where the related data is stored and where it flows; and

• where any related funds are and where they flow.

It could save a lot of time and money.

Image from Tenets in DM.

Rethinking Personal Data

As part of its 'midata' initiative to empower consumers, the department of Business Innovation and Skills has been consulting on a proposal to give the Secretary of State a general power that "might be exercised broadly or in a more targeted way" to compel suppliers to supply transaction data at a consumer’s request. In the interests of transparency, I've summarised my response to the consultation over on The Fine Print. As previously explained, I should disclose that I've been involved in the midata Interoperability Board from its inception in 2011.

On The Futility Of Cookie Consents

It's a month or so since the Cookie Law took effect and already it's an exercise in futility. I haven't clicked on a single cookie consent, yet I know my browser and hard drive are lousy with the things - both the helpful kind that improve my experience of using the web site I'm visiting, and the small proportion that feed information about me to third party advertisers.

There are two reasons for not clicking on cookie consents. 

Firstly, I don't reserve a single minute in my day for reading cookie consents. Life is short. Every second spent not reading cookie consents is a priceless investment in something potentially productive. Sleeping is a better use of time. Not reading cookie consents is in the same category as never watching American celebrity murder trials, or Big Brother or X Factor. Or... well, you get the picture. Reading cookie consents is a true waste of time.

Secondly, the Cookie Law is a one-size-fits-all requirement for user consent before setting all types of cookie - both those that will help you retweet this post and immediately return to read more, as well as those that will lead someone to conclude you have a passion biscuit recipes after you've read this post. I have no problem at all with the first kind, and it seems overkill to ask me to opt-in or out to them being set. I can clear them if I want to. And making me click "I accept" for all types of cookies doesn't even scratch the surface of the very specific, difficult challenges posed by the second kind of cookie: how and why the data about my movements is going to be shared with advertisers, and ensuring it is in fact used appropriately. Those challenges need the pragmatic, holistic attention of a WEF 'tiger team', not the overly zealous intervention of Eurocrats using data protection law as a means of delivering the single market fantasy.

Image from Jefferson Park.

When Will Control Truly Shift To The Consumer?

For those engaged in the process of empowering consumers, 2012 is already a fascinating year. So it was timely that a bunch of us met at Ctrl Shift's "Explorer's Club" to try to map the timeline for when 'customer relationship management' truly inverts and firms finally acknowledge their customers control them. 

The output of the session is being converted into an 'infographic' that will be available as a reference soon. In the meantime, here's an excellent drawing that Joel Cooper produced during the session to reflect the various themes:

Augmented Reality and Linked Data

I enjoyed a far-ranging conversation with @StGilesResident today, and am indebted to him for the reference to the following TED talk by Pranav Mistry. I do not know how I've missed this to date and am looking forward to seeing how this technology evolves with 'midata' initiatives to support our day-to-day activities in future. The folks at Microsoft have also been playing around here (you can follow them here).