Is BigTech Still Battling The Entire Human Race, Or Just Some Of Us?

Readers will be familiar with my view that we consumers tend to be loyal to 'facilitators' who focus on solving our problems, rather than 'institutions' who solve their own problems at our expense. Previously trusted service providers can also lose their facilitator status, and I'd argue that Facebook has already done so (owing to privacy, electoral and extremist content scandals) and Google is firmly headed in that direction (through behaviour incurring massive EU fines). Yet, despite announcements designed to suggest increasing transparency, it seems BigTech is actively resisting independent human oversight and the perceived battle between computers and the human race is far from over...

Part of the problem is that 'BigTech' firms still operate as agents of retailers and other organisations who pay them vast amounts of money for exploiting our personal data targeting advertising at us, rather than as our agents for the purpose of finding what we need or want while shielding us against exploitation. In fact, this is the year when digital advertising spend will exceed spending on the old analogue 'meat space' channels. 

Combine that exploitative role with rogue artificial intelligence (AI) and you have a highly toxic reputational cocktail - particularly because AI based on machine learning is seemingly beyond human investigation and control. 

For instance, Amazon found that an AI programme used for its own recruiting purposes was terribly biased, but could not figure out what was going wrong or how to fix it, so had to simply shut the thing down.  Alarmingly, that suggests other AI programmes that are already notorious for being biased, such as those used for 'predictive policing', are also beyond fixing and should be shut down...

Many BigTech firms are appointing 'ethics boards' to try to avoid their AI programmes heading in inappropriate directions. Trouble is, not only is there doubt about what data scientists might view as inappropriate (which drove the appointment of ethics boards in the first place), but these boards are also generally toothless (only CEOs and main boards can decide the actual course of development), and tend to be populated by industry insiders who sit on each other's ethics boards. 

It is unclear, for example, whether the recommendations of the ethics committee overseeing the West Midlands police 'predictive policing' algorithm will be followed. Meanwhile, 14 other UK police forces are known to be using such AI programmes...

Another worrying trend is for AI firms to prevent investors voting on the company's plans, using "dual class" share structures that leave voting control with the founders rather than shareholders. Lyft is the latest to hit the news, but other offenders include Alphabet (Google), Blue Apron and Facebook, while Snap and Pinterest give shareholders zero control. Those firms might argue that stock prices are a check in themselves. But the stock market and investor greed are notorious for driving short-term decisions aimed at only maximising profits, and even giant regulatory fines are subject to appeal and can take a long time to be reflected in share prices. Voting power, on the other hand, is more qualitative and not simply a function of market forces - and the fact that it is being resisted tells you it's a promising tool for controlling BigTech.

Regulation will also be important, since fines for regulatory breaches are a source of revenue for the public sector that can be used to clean up the industry's mess and to send signals to management, investors, competitors and so on. I'm not suggesting that regulatory initiatives like the UK Brexidiot ToryKIP government's heavily ironic "Online Harms" initiative are right in the detail or approach, but Big Tech certainly cannot keep abdicating responsibility for the consequences and other 'externalities' associated with its services and approach. There has to be legal accountability - and grave consequences - for failing to ensure that AI and the firms themselves are subject to human control.

I guess the real question might be: which humans? 

Humans, Not Big Data, Must Benefit From Opening Up The Banks

One of the 'good news' items in last year's Autumn Statement was the endorsement of an Open Data Institute report on how innovators could make better use of bank data on our behalf (instead of on the banks' behalf). Last week, the Treasury followed up with a call for evidence "on how best to deliver an open standard for application programming interfaces (APIs) in UK banking and... whether more open data in banking could benefit consumers."

This is important, because enabling our own machines to start crunching our financial data on our behalf is key to humans winning the race against Big Data.

'Open data' involves enabling public access to data by connecting data sets using uniform resource identifiers ('Linked Data') and publishing data about them in machine-readable formats. While the government and other public institutions have done a good job of opening up public data sets so far, our private institutions are lagging behind, as discussed here. That's because they benefit from making sure we know less than them about our own requirements - 'information asymmetry'.

The Midata initiative was the first to aim at restoring the balance in favour of consumers, by taking aim at energy utilities, telecoms providers and banks. Now the energy companies face being opened up like so many tin cans by smart metering, while the telecoms market looks like it will go from bad to worse. Banks have also proved a tougher nut to crack. Yes, some people can download their current account data via internet banking. But in addition to consumers, there are 4.5 million small businesses out there that need a hell of a lot more than that. So far, it's taken regulatory action just to start the process of improving access to small business credit data and making a market in rejected small business loan applications. Now the Treasury is trying to force the pace on the technology necessary to support all that.

The ODI's recommendations for opening up banking are: 

• banks should agree an open API standard to support third party access to bank data - basically firms that can help you make sense of the data (but Big Data firms will try to persuade you to share it with them too);
• independent guidance should be provided on technology, security and data protection standards that banks can adopt to ensure data sharing meets all legal requirements; 
• an industry wide approach should be established to vet third party software applications and publish a list of vetted applications as open data - this would allow visibility of firms that are acting on consumers' behalf and those who are not;
• standard data on Personal Current Account terms and conditions should be published by banks as open data; and 
• credit data should be made available as open data.

My main concern is that requiring agreement on 'standards' as a precondition for opening up banking will enable the banks to delay the whole process by a decade - as they did with Faster Payments. As was recommended by the security working group in the Midata initiative, I would prefer to see banks required to immediately make their data available to each consumer/SME in whatever open format the banks choose, while adhering to common data security protocols, and leave it to the open data community to figure out how to re-format and display whatever rubbish they dish up.

Apart from complaints by the banks, Treasury officials expect to hear positive contributions from consumer groups, other financial services providers, financial technology firms and app and software designers.

Let's not disappoint them. This is a good opportunity to ensure the government clears the way for innovation that puts you and me at the heart of financial services, without mistakenly creating further barriers in the process.

Six Years On And Pragmatism Has A New Frontier

I see this blog has reached the ripe old age of six, so I felt compelled to squeeze in at least one post to celebrate.  

It's fitting that the reason for my absence has been the need to get to grips with the FCA's proposals to regulate P2P lending and investment-based crowdfunding - not to mention the revelations concerning the Chairman of the Co-op Bank. After all, this blog set out to chart the rise of facilitators who help us wrest personal control of our day-to-day lives from the one-size-fits-all experience imposed on us by our institutions. Rumbling the 'Crystal Methodist' marks the continuing plunge of faith in those same institutions, while the decision to finally let the 'crowd' into the regulated financial markets shows that even Parliament recognises you and I are better off dealing with each other directly than simply entrusting our life's savings to the banks and investment funds.

Of course, these are just a few examples of the punishment being doled out to our financial institutions. And they aren't the only ones under pressure from the trends sweeping society, as we struggle to figure out a more sustainable form of capitalism. All our institutions, from the BBC to the Police to the Church, unions, political parties, government departments and so on, face the choice of becoming facilitators or withering away. 

So is there anything 'new' to write about? 

Six years on we are still seeing the dawn of where these trends will take us. But to get a sense of the future, I've been following the rise of 'open data' - or open access to data in machine-readable form. This marks a new frontline between institutions and facilitators. Big Data vs You. Not only has it already created new facilitators, in the form of "personal data stores" or "personal information managers", but it may also redefine some of today's facilitators as the institutions of tomorrow... 

As a taste of things to come, last week a senior advertising executive insisted to me that "Big Data can accurately predict human behaviour." To be fair I made him repeat the assertion in case it had slipped out by accident. No one else at the table seemed to find that truly weird, and it wasn't until the end of the week, when I met up with some people working at the sharp end of data gathering, that I was able to fully enjoy the hilarity of that statement.

This is going to be fun.

Image from Data.gov.uk

The Future Is Not Behind Us, It Lives Locally

For every old saying there's an opposite. Today's conflict lies in the adage that "those who don't know history are destined to repeat it." Yet "our history is not our destiny." This leaves a fine line between useful historical insight into how the world works and steering solely by what we see in the rear-view mirror. The distinction becomes critical as we lurch ever more quickly from one financial crisis to the next.

So what lessons from recent history will help us move foward, and which should go in the scrapbook? Here are 5 that I think are worth taking forward, in no particular order:

1. Clearly, the Internet is not a fad. Consumers are the winners, aided by facilitators in their battle against our creaking institutions. Yet e-commerce is still only 10% of all retail. And we are still in the 'primordial soup' phase in the evolution of our tools for extracting meaning from the great, chaotic swirl of data. So this trend has a lot more mileage in it yet and will sweep across more service sectors - if you can buy it online and have it delivered it won't be sold in high volumes on the high street. In fact, it might even be made locally...

2. In addition to democratising services, the Internet is also returning the means of production to local communities through e-enabled machines. Remote/home-working is replacing central workplaces and 'factories' are getting closer to customers, requiring a rethink of corporate systems, processes and supply chains. Huge production plants and sole-occupancy office towers will gradually become a thing of the past. High streets may well regenerate to support this trend, or as a result of it.

3. Cities with at least 3 private workers for every public sector worker see the most growth. Give that some thought if you live in a city near the bottom of this chart. This chimes with findings that economies (regional and national) whose public spending exceeds 35% of their GDP struggle to grow. You have to feel sorry for Northern Ireland...

4. The public sector is very inefficient and needs to act locally - public institutions are already being relentlessly affected by lessons 1 and 3 above, and 2 should increase  the pressure. Civil servants really have no alternative but to spend public money more wisely. Meanwhile, we'll help drive down the cost of government by dealing with the government online. Big government offices must surely go eventually. Fortunately, the moves to devolve more power to local government coincide here, but I don't think we should be fooled into thinking that's part of any real plan to future-proof the UK...

5. The UK's financial system is seriously inefficient at allocating money to people and businesses. The fact that we all rely heavily on a few major banks for whom lending to small businesses is not a core activity is part of the problem, but innovation and competition are constrained by our outdated and rigid regulatory framework and the related incentives. Crowdfunding, or peer-to-peer finance, platforms are springing up all over the country, and are increasingly focused on specific sectors, activities and locations.

In short, if I were a civil servant (see 4) based outside the south east of England (3), I would start an Internet-based service (1) that efficiently provides low cost finance (5) to help localise the means of producing stuff using e-enabled machines - or buy my own 3D printer and start renting it out (2). Now. 

Image from KC Anderson.

Open Data Spiders?

Since 2009 I've hoped that the semantic web - that is my computer dealing with suppliers' computers - would replace the need for price comparison sites. Following a discussion last night at the CtrlShift Explorers' Club, I'm confident that we don't have much longer to wait.

If suppliers publish their product data in computer-readable format, I could then programme an application or 'spider' to search the provders' open systems to find the product that's right for me - ideally a bespoke product assembled from a menu of optional components. This spider would use my personal data to conduct its search without disclosing that data to any product providers, at least until the time of purchase (and disclosure might not even be necessary then). It could also collect, say, public sector Open Data related to my desired activity, and analyse it in the context of my relevant personal transaction history. This could vastly improve my choice of car, holiday or home improvement and how it's financed. Or it could save me money by keeping me on the right energy or mobile phone tariff.

This is not about 'intent-casting' or 'demand-casting' in order to encourage suppliers to send me thousands of offers. My spider would not announce to the world that it's looking for anything. It would simply run around the web looking at openly available product codes and report its findings to me. Ideally, the product provider will have no idea that it's actually me who's looking until I make a purchase, if ever.

And I would not need to read any screens or physically enter any data until my spider reported its findings - or it could save me the trouble by calling my mobile.

In a machine-to-machine world, the marketing challenge is to ensure that anyone's 'spider' can always find your product data, and that data is accurate and up to date. Perhaps it could be somehow 'spider optimised', but it seems to me it's the job of the spider developers to make sure the spiders are good at finding product data, even when it's in a sorry state.

My sense is that an Open Data approach to the market takes such a different corporate mindset that it is unlikely to sit comfortably within traditional suppliers, where "Big Data" is the latest buzzphrase. In the Open Data world the challenge is to enable your products to be directly embedded in the ecosystem, helping to solve problems as customers encounter them and their machines or 'spiders' look for an answer. The traditional product approach is not 'connected' in that way, or at all. And, as I suggested recently, "Big Data" approach to behavioural targeting of advertisting seems fundamentally hamstrung by the fact that personal behavioural data is highly contextual and not really 'predictive' from one scenario to the next. Why spend all that money on what is ultimately a shot in the dark?

Those who ignore the Open Data option could well be spending their way rapidly into oblivion. 


Image from Data.gov.uk

Linked Data: An Energy Credit Rating?

I received a love letter from EON recently, telling me they love me so much they're going to increase my direct debit. That way, I will "build up a credit over summer" and enable EON to spread its bill over the year.

Sadly, this is a case of unrequited love. I do not want to leave any surplus cash with EON, especially when they haven't mentioned paying me any interest on the credit - or even putting the interest toward my bill.

But on the upside, EON's letter did prompt me to call @Oikonomics, who is fairly heavily involved in an alternative energy proposition, amongst many other things. We got talking about Green Deal and feed-in-tariffs and the challenges of comparing the official option for financing your personal alternative energy generating/saving with alternative means of financing. One idea is to enable you to generate an "Energy Credit Rating" to use as a base for comparing the overall benefit to an alternative energy option. It could combine your MPAN billing number, linked to some data from the Met Office about average sunlight/wind in your area, local energy prices for what you buy and what you generate, the Feed-in-Tariff scheme, and your own credit rating.

Linking some of this data may also help reduce the complexity/cost associated with making energy price changes.

Oh, and Bruce also told me Ovo credits your bill with the equivalent of 3% interest on any credit balance (see under "Earn from Energy").

Image from Energy Saving Trust.

Mydata And Consumer Empowerment

On Thursday, work began in earnest on the 'mydata' project featured in the government's "Consumer Empowerment" strategy (see Better Choices: Better Deals, on which I posted earlier).

As with the overall consumer empowerment policy, the primary goal of the ‘mydata’ project is “to put consumers in charge so that they are better able to get the best deals for themselves, individually and collectively.” Achieving that involves enabling consumers to access information about their purchases, analyse it according to their own preferences and use that information to make better purchasing decisions.

As a long-time critic of the European Commission approach to facilitating e-commerce, I'm overjoyed the government is convinced that new legislation is not the best way to achieve consumer empowerment. Instead, it's relying on "a wide range of new programmes that have been developed in partnership with businesses, consumer groups and regulators" against a background of normal regulatory enforcement.

The mydata development work is being fostered through a series of 'boards', chaired by Professor Nigel Shadbolt. There's a Strategy board made up of private sector retail businesses, consumer bodies and government representatives; four Sector boards comprising representatives of suppliers in the energy, financial services, telecoms and retail sectors; and an Interoperability board of private and public sector representatives. The focus of the Interoperability board is on maximising synergies among the sector groups' work; addressing barriers common to all groups; maintaining a balance amongst key issues of practical utility, privacy, security and data portability; and offering suggestions on where ideas and solutions in one sector might link up (or mashup) with others to better reflect consumers' day-to-day activities.

I know what you're thinking, but these boards are not designed to be exclusive talking shops or symbolic meetings of 'the great and the good'. The idea is to be as pragmatic as possible, using the boards to draw in as many interested parties, ideas and resources as possible to achieve rapid progress. It's a fascinating challenge, and I'm thrilled to be helping out on the interoperability front.

There'll be plenty of project communications, of course, which I'll be doing my best to retweet etc, and send to people I know who may be interested or able to help. I also plan to share material that I happen across outside the project. For starters, I've included below the links to Sir Tim Berners-Lee's TED talk on the semantic web (or Linked Data) and a few of the items he mentions. I'm very interested to receive any comments, referrals, ideas etc you may have.

Sir Tim cites some key examples of Linked Data and its uses. DBpedia is the fascinating "community effort to extract structured information from Wikipedia and to make this information available on the Web." And here is the TED talk by Hans Rosling to which he refers:

Image from 1Million1Shot.

A Plan for Small Business Growth?

Hat-tip to @JamesGreenThing for the link to the last Business Insider list of up-and-coming entrepreneurs.

These lists are always much more interesting 6 months on. In this case, the bubble in online coupons is more obvious, as is the fact that the fashion market is getting the Web 2.0 treatment. Yet BankSimple still hasn't launched.

Online invoice discounting has been on my radar since early 2008. So it's heartening that The Receivables Exchange is highly rated in the US, and reassuring that MarketInvoice is gaining traction in the UK. Doubtless there'll be more on that front soon.

Partly for that reason, SecondMarket really caught my eye. It's a market for alternative investments, so incorporates a channel for equity in private companies (e.g. CrowdCube in the UK). But the emphasis on gathering "robust market data" signals a growth in the availability of richer information about private companies. Focus on this is timely, given the massive official paywall around corporate data. Recent commitments to 'Open Government' (follow the Open Knowledge Foundation) are promising. There's already a Government commitment to arm consumers with their own data. Perhaps enabling SMEs to use their own data for their own benefit could be added to The Plan for Growth?

Image from AppAssure.

Alternative Power For Geeks

It's fascinating how much data is publicly available. Here, for instance, is a summary of key data that describe the UK electricity market, including demand and generation by fuel type.

So what?

Well, apart from putting various fuel types into perspective, and maybe settling a few arguments, it's worth reflecting that the Hawthorne Effect was named after the electricity plant in which it was first documented. Henry Landsberger found that workers' productivity improved when he measured it to study the impact of light levels on their work, but declined again when his experiments ended. That suggests that when people know you're measuring their activity, it improves.

Alternative energy-generation measurement widget, anyone?

Water Quality/Filter Filter?

A tale of two teas

I confess to having taken drinking water pretty much for granted, until my brother-in-law joined a leading water softener and filter supplier. Now I'm a little obsessed.

Amazingly, UK water companies actually produce a report that shows the water quality in your area. Just plug in your post code and tick the content you want to see. This is designed to tell you what 'contaminants' the water companies have been able to filter out, and allow you to decide what you might need or want to filter out yourself for whatever reason.

Unfortunately, there seems to be no mash-up that enables you to directly compare the remaining contaminants in your area with the capabilities of all the available water filters on the market - a "water quality/filter filter".

Something for the Open Data community, using API's and data from the various utilities and filter providers?