Is BigTech Still Battling The Entire Human Race, Or Just Some Of Us?

Readers will be familiar with my view that we consumers tend to be loyal to 'facilitators' who focus on solving our problems, rather than 'institutions' who solve their own problems at our expense. Previously trusted service providers can also lose their facilitator status, and I'd argue that Facebook has already done so (owing to privacy, electoral and extremist content scandals) and Google is firmly headed in that direction (through behaviour incurring massive EU fines). Yet, despite announcements designed to suggest increasing transparency, it seems BigTech is actively resisting independent human oversight and the perceived battle between computers and the human race is far from over...

Part of the problem is that 'BigTech' firms still operate as agents of retailers and other organisations who pay them vast amounts of money for exploiting our personal data targeting advertising at us, rather than as our agents for the purpose of finding what we need or want while shielding us against exploitation. In fact, this is the year when digital advertising spend will exceed spending on the old analogue 'meat space' channels. 

Combine that exploitative role with rogue artificial intelligence (AI) and you have a highly toxic reputational cocktail - particularly because AI based on machine learning is seemingly beyond human investigation and control. 

For instance, Amazon found that an AI programme used for its own recruiting purposes was terribly biased, but could not figure out what was going wrong or how to fix it, so had to simply shut the thing down.  Alarmingly, that suggests other AI programmes that are already notorious for being biased, such as those used for 'predictive policing', are also beyond fixing and should be shut down...

Many BigTech firms are appointing 'ethics boards' to try to avoid their AI programmes heading in inappropriate directions. Trouble is, not only is there doubt about what data scientists might view as inappropriate (which drove the appointment of ethics boards in the first place), but these boards are also generally toothless (only CEOs and main boards can decide the actual course of development), and tend to be populated by industry insiders who sit on each other's ethics boards. 

It is unclear, for example, whether the recommendations of the ethics committee overseeing the West Midlands police 'predictive policing' algorithm will be followed. Meanwhile, 14 other UK police forces are known to be using such AI programmes...

Another worrying trend is for AI firms to prevent investors voting on the company's plans, using "dual class" share structures that leave voting control with the founders rather than shareholders. Lyft is the latest to hit the news, but other offenders include Alphabet (Google), Blue Apron and Facebook, while Snap and Pinterest give shareholders zero control. Those firms might argue that stock prices are a check in themselves. But the stock market and investor greed are notorious for driving short-term decisions aimed at only maximising profits, and even giant regulatory fines are subject to appeal and can take a long time to be reflected in share prices. Voting power, on the other hand, is more qualitative and not simply a function of market forces - and the fact that it is being resisted tells you it's a promising tool for controlling BigTech.

Regulation will also be important, since fines for regulatory breaches are a source of revenue for the public sector that can be used to clean up the industry's mess and to send signals to management, investors, competitors and so on. I'm not suggesting that regulatory initiatives like the UK Brexidiot ToryKIP government's heavily ironic "Online Harms" initiative are right in the detail or approach, but Big Tech certainly cannot keep abdicating responsibility for the consequences and other 'externalities' associated with its services and approach. There has to be legal accountability - and grave consequences - for failing to ensure that AI and the firms themselves are subject to human control.

I guess the real question might be: which humans? 

If You Need Consent To Process My Personal Data, The Answer Is No

... there are plenty of reasons for businesses and public sector bodies to process the data they hold about you, without needing your consent. These are where the processing is necessary for:

• performing a contract with you, or to take steps at your request before agreeing a contract; 
• complying with their own legal obligation(s); 
• protecting yours or another person's vital interests (to save your life, basically);
• performing a task in the public interest or in the exercise of their official authority; 
• their 'legitimate interests' (or someone else's), except where those interests are overridden by your legitimate interests or your fundamental rights which require protection of personal data. 

The General Data Protection Regulation lists other non-consent grounds apply where your personal data is more sensitive: relating to criminal convictions and offences or related security measures; or where it reveals racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership; or it is genetic or biometric data for the purpose of identifying only you; or data concerning health or your sex life or sexual orientation. National parliaments can add other grounds in local laws.

These non-consent grounds for processing are all pretty reasonable - and fairly broad. So, if you don't have the right to process my personal data on one of those grounds, why would I want you doing so?

This would seem to herald a new era in which the Big Data behavioural profiling/targeting/advertising model begins to decline, in favour of personal Apps (or open data spiders) that act as your agent and go looking for items in retailers' systems as you need it, without giving away your personal data unless or until it is necessary to do so...

Is There Really A Single EU Market?

Some sobering figures from the European Commission for single market fantasists enthusiasts (as if Greece wasn't sobering enough).

EU cross-border services account for 4% of all online services, as opposed to national services within the US (57%) and in each of the EU member states (39%). 

15% of EU consumers bought online from other member states, compared to 44% who bought online nationally, with online content seeing double-digit growth.

Only 7% of SMEs sell online across EU borders - and it costs an average of €9,000 to adapt their processes to local law in order to do so. 

The cost/price of delivery is (obviously) cited as a major problem, as well as differing VAT arrangements. But suggested solutions seem to ignore these and other key barriers to cross-border retail that have been cited in previous market studies, such as lack of marketing strategy, preference for national brands, language barriers and local employment law challenges. Presumably, that's because the Commission can do little to address such fundamental practicalities. Instead, they want to focus on:

• stronger data protection rules;
• broadband/4G roll-out;
• use of 'Big Data' analytics; and
• better digital skills amongst citizens and e-government by default.

The sense of futility that permeates such reports by Eurocrats only emphasises the fact that the law follows commerce; it doesn't catalyse markets.  

Yet, ironically, in areas where commercial and consumer pressure to enable cross-border activity is emerging, such as crowdfunding and crypto-technology, we find European institutions taking an unduly restrictive approach.

When will they simply get out of the way?

A Developer's Guide to Privacy and Fairness?

Over the past few months I've noticed a range of different articles expressing privacy concerns about mobile apps, wearable devices and internet-enabled things, like smart TVs and bathroom scales ("the Internet of Things") on the one hand; and initiatives like 'Midata' to help you create your own 'personal data ecosystem', on the other. But regulation aimed at unfair trading is also relevant in this context, as are the various security requirements being proposed at EU level in relation to payments and 'cybersecurity' more generally. Official guidance in these areas is often broad but not comprehensive, as in the summary of privacy rules given in the context of Midata. It would be great to see a more concerted effort to draw all the guidance together. I have suggested this to the SCL. In the meantime, this overview explains briefly where to find guidance on meeting privacy and fairness requirements when using apps and other devices for consumer marketing purposes.

Note: as a developer, it's worth reading such guidance as if you were a consumer, to understand the regulatory intent. As a consumer it's worth reading guidance aimed at firms, since that gives you a better insight into how things actually work 'behind the scenes'.

The Information Commissioner has plenty of practical guidance on privacy in the context of cookies, mobile applications and data sharing (and a other guidance by sector or activity).

The Advertising Codes are important sources of information on how systems are supposed to behave in a marketing context.

PhonepayPlus has issued guidance on the use of premium rate numbers.

The Office of Fair Trading had plenty of guidance on how to comply with consumer protection regulation, which is now hosted by the Competition and Markets Authority, including principles for online and app-based games.

The OFT's guidance on what's appropriate in a consumer credit context, such as debt collection, is now in the FCA's consumer credit rules, and the FCA also recently consulted on updates to its guidance on financial promotions in the social media.

Firms seeking FCA authorisation often have to provide a lot of detail on their IT systems and governance in the process. The proposed new EU directive on payment services will broaden the range of regulated services and go into considerable detail on data security. In fact, security standards will be produced by the European Banking Authority, just to add to the confusion.

Knowing where consumers can complain is a guide to other regulators who may be interested in how your application works. There is an overview of UK consumer complaints channels here. There are specific complaints bodies for sectors, such as energy, financial services and telecoms, as well as for activities, like advertising and processing personal data.

However, it's you should be aware that the Data Protection Act gives businesses separate rights to process your personal data in the following circumstances:

• for the performance of a contract to which you are a party, or for taking of steps at your request with a view to entering into a contract;
• for compliance with any legal obligation, other than an obligation imposed by contract;
• in order to protect your vital interests;
• either for the exercise of a function conferred on a business by law or for the exercise of any other functions of a public nature exercised in the public interest;
• for the purposes of legitimate interests pursued by a business or by someone else to whom the data are disclosed, except where that processing is unwarranted by reason of prejudice to your rights and freedoms or legitimate interests.

Public sector bodies also have certain rights to use your data which I haven't covered here. However, it's important to mention the ID Assurance Programme run by the Government Digital Service team, which has issued useful guidance on ID assurance. And the Connected Digital Economy Catapult that builds platforms for SMEs is due to develop a code of practice on consumer protection.

Eurocrats Need A Reality Check

The Society for Computers and Law was recently entertained on the topic of trust in Big Data and the Cloud, by Paul Nemitz, European Commission Director of Fundamental Rights and Union Citizenship (in the Directorate-General for Justice). Both immigration and data protection feature among the main responsibilities of his Directorate, so you can imagine Paul is a very busy man right now, and it was very kind of him to take the time to speak.

Right, so that's the polite bit out of the way ;-)

Paul was keen to challenge the Brits in the audience to be more pragmatic in their attitude to the European Union. He believes the UK is among those who engage with the EU irresponsibly on the basis that "everything that comes out of Brussels is shite". Instead, he says British officials, lawyers and academics should be focused pragmatically on how to engage positively to achieve better European policy and regulation.

Of course, it's an old rhetorical trick to characterise your opponent's views as overly simplistic, boorish and stupid. Paul knows that the UK's opposition to red tape is based on more serious and fundamental differences than simply declaring everthing from Brussels as 'shite', as discussed below. But as a Commission official, he's not able to enter into debates over the fundamental principles of the EU. It's his job to be a 'Believer' and get on with building the vision. He must take it on faith that the European Union is a single market, rather than a loose collection of disparate nations held together by red tape and political ambition. 

It suits some EU member states to accept that same article of faith, but not all, and the people in the streets certainly don't think that way - consumers have been worryingly slow to purchase across borders, for example. And the recent election results revealed that a huge proportion of the electorate remain to be convinced that EU governance is wholly worthwhile. 

In these circumstances, the UK's rather sceptical view of what comes out of Brussels is quite broadly representative, and the attempt to draw a line in the sand over the imposition of a fervent unionist as head of the Commission was completely understandable. It's also pragmatic. If the EUrophiles were humble enough to accept that the single market is still an ambition, they too would realise it's unwise to be seen to force the issue. People have to be brought along on the journey, and maybe the UK is a good indicator of how far they are being left behind.

To back his claim that the UK's attitude is simply boorish, Paul points to a 'typical' lack of empirical evidence for resisting provisions in the General Data Protection Regulation requring large firms to appoint a data protection officer and to facilitate fee-free 'data subject access requests'. He says these things work well in other EU member states already, and haven't driven anyone out of business. And against the UK's charge that the European Commission is needlessly committed to ever-increasing levels of privacy regulation, Paul points to surveys that show ever-increasing levels of concern amongst EU citizens about commercial and governmental intrusion into their private lives; as well as recent judgments from the European Court of Justice and the US Supreme Court curbing commercial and governmental intrusion into these areas (ironic, given that one of the ECJ's decisions was to declare Europe's own Data Retention Directive invalid).

Again, he's missing a sensible, pragmatic point. The UK's reaction is telling him is that when huge swathes of the population questionn the very existence of the EU, it's wiser to stick to the essential foundations and building blocks, rather than snowing people with confetti about day-to-day compliance issues.

However, I'm glad to say that Paul was able to explain how the European Commission is working on some important foundations, such as getting standing for foreigners to take action to protect themselves in the US courts; and preventing indiscriminate mass collection of the personal data of EU citizens by any government or corporation, inside or outside the EU. Those two things are very important to building trust in governments, as well as Big Data, and are the sort of fundamental constitutional changes that citizens would find extremely difficult to achieve solely through the democratic process - though the European Commission has climbed on the bandwagon of public opinion (or Merkel's personal outrage), rather than initiated pressure to achieve these outcomes in its own right.

I also think Paul is right to point out that businesses are wrong in the view that personal data is 'the currency of the future' or 'oil in the wheels of commerce'.  Money is fungible - we view one note as the same as another - and, similarly, oil is just a commodity. So the data related to money and oil are hardly very sensitive and can be dealt with through economic regulation. But people, and the data about them and their personal affairs, come with more fundamental rights that can't simply be dealt with in economic terms. It's important that citizens have a right of action against governments and corporations to protect their interests (though I think the Google Spain decision was wrong).

But Paul overstates the 'synergies' between EU regulation, trust and innovation. He is stretching too far when he says that vigorous regulatory protection is essential to the creation of trust between people and their governments and the corporations they deal with. As evidence for this, he claims that the UK's Financial Conduct Authority as doling out the largest fines in the EU for the abuse of people's personal data, and asserts that this has built trust in the UK financial services market. From there, Paul leaps to the conclusion that similarly vigorous regulatory attention is somehow one of the necessary pre-conditions to the creation of commercial trust generally. He then leaps again to the notion that commercial trust driven by regulation is a pre-condition for innovation because, "There is no trust in start-ups," he says.

This is all nonsense.

Here Paul seems to be looking at the world through the lens of his own area of responsibility rather than from a consumer standpoint. Very few of the FCA's fines have anything to do with abuse of customer data, and its fines are puny compared to US regulators in any event. And in survey after survey, we've also seen that the providers of retail financial services are generally among the least trusted retail organisations in the UK and Europe. Enforcement processes also tend to be slow, resulting in fines for activity that ceased years before, and depriving consumers of the opportunity to cease dealing with firms at the time of wrongdoing. So, relative to consumers' perception of other industries, complex financial regulation and allegedly vigorous enforcement action has been no help at all.

It's also strange for Paul to suggest that "there is no trust in start-ups" without the backing of regulation, given the vast number of start-ups that have achieved mass consumer adoption absent effective regulation - certainly across borders. Unless, of course, Paul still considers Google, Facebook, Twitter etc to be 'start-ups', which would be weird. This ignores the fact that, love 'em or hate 'em, such businesses have been far more responsive to consumer/citizen pressure in changing their terms and policies than the European Commission or national legislators have been in altering their own laws etc. Indeed such businesses have even been relied upon by governments to enforce their consumer agreements to shutdown activities that national governments have been powerless to stop.

Paul's view of start-ups appears to reflect the continental civil law notion that citizens cannot undertake an activity unless the law permits it; while in the common law world 'the law follows commerce' - in the UK and Ireland (and the US, Canada, Australia etc) we can act unless the law prevents it. The havoc that arises from these opposing viewpoints - and the differing approaches to interpreting legislation - cannot be underestimated. In fairness, the UK needlessly creates a rod for its own citizens by 'gold-plating' EU laws (transposing them more or less verbatim). The national version is then interpreted literally. We would be far better off adopting the purposive interpretation of EU laws and implementing them according to their intended effect. This may mean a bit more friction with the Commission on the detail of implementation, but the French don't seem to mind frequent trips to the European Court where the Commission objects, and meanwhile their citizens don't labour under unduly restrictive interpretation of EU laws.

None of this is to say that I disagree with Paul's claim that strong individual rights and regulation to protect them are not inconsistent with making money and healthy innovation. But I reach this conclusion by a different route, starting from the premise that retail goods and services must ultimately solve consumers' problems, rather than be designed to solve suppliers' problems at consumers' expense. Strong individual rights are only one feature of a consumer's legitimate day-to-day requirements, not all of which can be legislated for. Co-regulation, self-regulation and responsible, adaptable terms of service are all part of the mix.

Of course, regulation can be helpful in preserving or boosting trust where it is already present - as can be seen in the development of privacy law amidst the rise of social media services (and in the context of peer-to-peer lending and crowd-investment, for example). But regulation can't create trust from scratch, any more than Parliament can start businesses.

If only the Eurocrats would recognise these realities and limit their attention to areas where government action is essential, I'm sure they would find more favour with pragmatists everywhere.

Google Spain Case Raises More Questions Than It Answers

I'm an enthusiastic supporter of greater control over your data. But I'm really struggling with the European Court of Justice ruling that you can stop a search engine linking to something lawfully published about you in your local newspaper's online archive.

The case in question concerned the appearance of someone's name in a local Spanish newspaper announcement for a real-estate auction connected with proceedings to recover social security debts 16 years ago. The individual concerned (openly named in the judgment, ironically) claimed that the proceedings had been "fully resolved for a number of years and that reference to them was now entirely irrelevant." He failed to obtain an order banning the newspaper from carrying the item in its online archive, but succeeded in getting Google Spain to remove any links to it.

But surely if it was lawful for the local newspaper to have published the item of data - and it remains okay for it to publish the data via its website - then it should be okay to allow someone to find it?

I mean, why stop at gagging Google's local site? Why not make local libraries cut tiny holes in their microfiche records?

On this point, the ECJ cited problems where multiple jurisdictions were involved, even though this was purely Spanish scenario:

"Given the ease with which information published on a website can be replicated on other sites and the fact that the persons responsible for its publication are not always subject to European Union legislation, effective and complete protection of data users [subjects?] could not be achieved if the latter had to obtain first or in parallel the erasure of the information relating to them from the publishers of websites."

But how could removing links to an item from a national search engine achieve "effective and complete protection" of the data subject when the same items are lawfully available via a national newspaper's online archive anyway? Surely a national problem such as this has to be dealt with at source, or not at all?

Another key issue is that the ECJ didn't seem to weigh up all the possible public interests against the particular individual's rights to 'respect for private life' and 'protection of personal data'. 

Surely, for example, there was some public interest in the publication of the notices of auction complained about, such as achieving a fair price for property being sold to pay a debt to the state? Perhaps if that requirement had been abolished you could make a case for requiring the deletion of public notices relating to them. But, absent their abolition, I'm not sure you can say it's "entirely irrelevant" that someone was mentioned in such a notice, even if that were years ago.

And is there not a public interest in being able to more readily find published material via search engines? Consider the huge variety of research processes that must now rely on search engines, from journalistic research, to employment checks, to official background checks. What holes will now emerge in such research processes? Will records be kept of all the links that search engines were told to remove? If so, where will those records be kept? Who will be allowed to access them? Aren't researchers now on notice that they should check individual newspaper archives for data that search engines aren't allowed to let you find? How many won't bother when they really should?

The problems with the judgment don't end there, as is demonstrated by the tortuous path the ECJ took to reach its result (explained here). 

All of this underlines the need for careful policy thought and regulatory clarity around these issues, rather than the celebratory gunfire heard in some quarters. This judgment raises more questions than it answers.

 

Six Years On And Pragmatism Has A New Frontier

I see this blog has reached the ripe old age of six, so I felt compelled to squeeze in at least one post to celebrate.  

It's fitting that the reason for my absence has been the need to get to grips with the FCA's proposals to regulate P2P lending and investment-based crowdfunding - not to mention the revelations concerning the Chairman of the Co-op Bank. After all, this blog set out to chart the rise of facilitators who help us wrest personal control of our day-to-day lives from the one-size-fits-all experience imposed on us by our institutions. Rumbling the 'Crystal Methodist' marks the continuing plunge of faith in those same institutions, while the decision to finally let the 'crowd' into the regulated financial markets shows that even Parliament recognises you and I are better off dealing with each other directly than simply entrusting our life's savings to the banks and investment funds.

Of course, these are just a few examples of the punishment being doled out to our financial institutions. And they aren't the only ones under pressure from the trends sweeping society, as we struggle to figure out a more sustainable form of capitalism. All our institutions, from the BBC to the Police to the Church, unions, political parties, government departments and so on, face the choice of becoming facilitators or withering away. 

So is there anything 'new' to write about? 

Six years on we are still seeing the dawn of where these trends will take us. But to get a sense of the future, I've been following the rise of 'open data' - or open access to data in machine-readable form. This marks a new frontline between institutions and facilitators. Big Data vs You. Not only has it already created new facilitators, in the form of "personal data stores" or "personal information managers", but it may also redefine some of today's facilitators as the institutions of tomorrow... 

As a taste of things to come, last week a senior advertising executive insisted to me that "Big Data can accurately predict human behaviour." To be fair I made him repeat the assertion in case it had slipped out by accident. No one else at the table seemed to find that truly weird, and it wasn't until the end of the week, when I met up with some people working at the sharp end of data gathering, that I was able to fully enjoy the hilarity of that statement.

This is going to be fun.

Image from Data.gov.uk

Open Data Spiders?

Since 2009 I've hoped that the semantic web - that is my computer dealing with suppliers' computers - would replace the need for price comparison sites. Following a discussion last night at the CtrlShift Explorers' Club, I'm confident that we don't have much longer to wait.

If suppliers publish their product data in computer-readable format, I could then programme an application or 'spider' to search the provders' open systems to find the product that's right for me - ideally a bespoke product assembled from a menu of optional components. This spider would use my personal data to conduct its search without disclosing that data to any product providers, at least until the time of purchase (and disclosure might not even be necessary then). It could also collect, say, public sector Open Data related to my desired activity, and analyse it in the context of my relevant personal transaction history. This could vastly improve my choice of car, holiday or home improvement and how it's financed. Or it could save me money by keeping me on the right energy or mobile phone tariff.

This is not about 'intent-casting' or 'demand-casting' in order to encourage suppliers to send me thousands of offers. My spider would not announce to the world that it's looking for anything. It would simply run around the web looking at openly available product codes and report its findings to me. Ideally, the product provider will have no idea that it's actually me who's looking until I make a purchase, if ever.

And I would not need to read any screens or physically enter any data until my spider reported its findings - or it could save me the trouble by calling my mobile.

In a machine-to-machine world, the marketing challenge is to ensure that anyone's 'spider' can always find your product data, and that data is accurate and up to date. Perhaps it could be somehow 'spider optimised', but it seems to me it's the job of the spider developers to make sure the spiders are good at finding product data, even when it's in a sorry state.

My sense is that an Open Data approach to the market takes such a different corporate mindset that it is unlikely to sit comfortably within traditional suppliers, where "Big Data" is the latest buzzphrase. In the Open Data world the challenge is to enable your products to be directly embedded in the ecosystem, helping to solve problems as customers encounter them and their machines or 'spiders' look for an answer. The traditional product approach is not 'connected' in that way, or at all. And, as I suggested recently, "Big Data" approach to behavioural targeting of advertisting seems fundamentally hamstrung by the fact that personal behavioural data is highly contextual and not really 'predictive' from one scenario to the next. Why spend all that money on what is ultimately a shot in the dark?

Those who ignore the Open Data option could well be spending their way rapidly into oblivion. 


Image from Data.gov.uk

Big Data: Is Reputation Really Portable?

At the recent London New Finance session on Big Data in Finance, Mark Hookey of Demyst.data suggested that a more accurate profile of a person is obtained by observing the breadth of the person's behaviour, rather than the depth of their history in any one area. The challenge is knowing which types of data from each area of the person's behaviour are representative (and having permission to use that data). He conceded that the profile is probabilistic rather than predictive.

Rachel Botsman has also talked about the concept of 'reputation capital', which is a product of all who have trusted you, when and why. She says it's only a matter of time before we are able to aggregate, monitor and use our ratings on the many sites on which we interact, so that we extract more value from the total of our "reputation capital". Rachel suggests this capital will be more powerful than our credit score. Rachel also suggests we'll be able to intentionally 'shape' our reputation, and so build-up our reputation capital (or reduce it). Two challenges she suggests are:

• knowing which data should be included in the data set that comprises your total reputation -  the same challenge facing Demyst.data and others Rachel mentions; and

• how to enable 'digital ghosts' to leverage their reputation capital (subject to privacy and data protection), since they don't interact online and therefore do not personally generate their own reputational data. 

But even if you do manage to identify the limited set of data that best represents a person's behaviour in a given context: 

• how relevant is that behaviour in any other context?

• what more does 'total reputation' tell you about a person in a given context than what you can see of their behaviour in that context?

As we observed in the programme on Rethinking Personal Data, the significance and value of personal data can't be captured in a single dollar amount, or a 'yes'/'no' answer to whether it can be used. Instead, the value and utility of personal data is a hugely complex dynamic that varies by: 

• the context or the activity we are engaged in;
• which persona we are using at that moment;
• the actual data being used or provided;
• the permissions given;
• the rights that flow from those permissions; and 
• the various parties involved.

It follows that a reputation derived from a specific activity is also purely contextual, and attempts to rely on a 'good' reputation in one context as suggesting good behaviour in another are flawed. At best, as Mark Hookey conceded, the total profile or reputation data might indicate probable behaviour in another context to a greater or lesser degree, but it won't be predictive. And the person relying on the reputational data still has to know or discover the reliability of making the association.

Of course, we already know how unreliable a reputation from one context can be in a different context. Brands are key reputational badges, and while sticking a trusted brand from one industry on a new product in another market or industry might work from time to time, generally it's not a sure-fire thing. If the brand is extended to enough products that fail, the brand eventually becomes diluted, or less trusted, as the failures outweigh the power derived from success in the original context.

Indeed, I believe that internet technology is liberating us from the tyranny of a single reputation, such as a credit score.

The highly contextual nature of both identity and the behavioural data generated suggests that if you want a good reputation for doing something, then you simply need to do it and do it well. Other people will only rate you highly if you do things they find helpful (assuming you can't simply buy ratings). In other words, the vast array of reputational data available on the internet is enabling us to distinguish the facilitators, who solve other people's problems in a specific context or market, from the 'institutions' who merely claim they're here to help, but actually exist to solve their own problems at other people's expense.

So, no, reputation is not really portable. And the idea that disparate reputations can be unified or expressed as a total amount of 'reputation capital' that can be reliably leveraged over time, regardless of context, is similarly flawed.

Image from MasCanc.

Will Consumer Transaction Data Drive New Online Marketplaces?

I should begin this post by explaining that I'm involved in the Interoperability aspects of the midata programme. I was invited to participate on a voluntary basis and have no client in that process. I donate my time. I'm independent of the dozens of other participants. I didn't shoot JFK. And I don't even own a pet, let alone one I believe to be a reincarnation of Elvis Presley. 

The only conspiracy in which I might be accused of involvement is the mass collaboration by consumers known as Web 2.0, which has evolved into Web 3.0 using "linked data" that computers can read (aka "the semantic web"). Lots of information - including government data and data from online bank accounts - is now available in this format because it makes analysis so much easier. Analysing the data is necessary to convert it from being merely information into useful knowledge. This is why the government is keen that banks, telecoms providers and big energy start making all your transaction details available to you in machine-readable form - if you want it. Gaining insight into your finances, communications and energy use will enable you to make better spending decisions and even negotiate new, bespoke products - if you want to do so.

In an explanatory post on the Which? website, Consumer Affairs Minister Jo Swinson sparked a number of comments by people who do not want to be empowered or have their consumer experience made more efficient. They are happy to make their own product searches and to use price comparison services. Evidently, they are not concerned at the number of problems detected by the Office of Fair Trading in its work on the price comparison sector. They also appear to believe that information security and privacy safeguards around existing transaction databases are adequate. I am not amongst them.

It's perhaps unfortunate that this debate seems to be centring on price comparison services, when it's really the mainstream product providers who are to 'blame' for the lack of consumer bargaining power in key markets. In fairness these services have evolved into a prime marketing channel through being more nimble, internet savvy and committed to transparency than the mainstream product providers themselves. This has rendered the services useful to consumers up to a point, but they are limited by their deals with the product providers as to how far they can really empower the consumer. In this sense they occupy the battleground created by consumer rebellion and institutional resistance.

I have expressed my own frustration with the current model of price comparison service for many years. That so many engage in extensive television advertising campaigns tells you that being a price comparison service provider is a really great business to be in. But consumers are paying for those big advertising campaigns in the same way they're paying for vacuous bank advertising - through the price of the products they buy, which in turn generates commission and/or ad revenue for the comparison service providers. So, while these services are intended as a tool for consumers to use, it would be naive to assume that price comparison service providers are acting on the consumers' behalf. Product providers are paying good money to ensure these marketing channel ultimately work for them, not you. As a result, these services are not so much about 'price comparison' as simply 'comparative advertising'.

Critically, however, the current crop of price comparison sites also only rely on the entry and/or display of personal and product information in human readable form. They have you by the eyeballs, at least for as long as you're able to keep your eyes on the screen. Yet the product providers are able to rely on their computers mining and analysing a wealth of yours and other customers' transaction data to work out the most profitable product to offer you. True, in some scenarios - like insurance - this 'information asymmetry' appears to favour you, but product providers have giant data sets that can overcome any advantage you think you might have. And when in doubt they simply charge you more. In The Undercover Economist, Tim Harford shows how information asymmetry works to our detriment in buying a car - when the dealer has all the information - as well as in the market for health insurance, because we have the information and not the supplier.

So while a price comparison service might enable you to buy a cheaper traditional product from one provider versus another, you are by no means able to negotiate from a position of real knowledge about the product or price that's right for you.

This makes a human-readable interface an enormous waste of your personal time for the tedious yet important task of ensuring you spend your money wisely. Instead, our own computers should be analysing our transaction data and interrogating product providers' systems directly, not only to find a product at the right price, but to create the right product for the right price. This might be as simple as relying on your rate of energy use to always ensure you're on the right tariff, even if that means switching providers daily. Or it may open up opportunities for collaborative consumption amongst consumers with whom you share similar interests or behaviours.

Of course, most of us won't have the time, skill or resources to do all this for ourselves, anymore than we can build our own cars. So new intermediaries are springing up to store and/or crunch the data for us. I'm not going to name them because I don't want this post to be perceived as some kind of advertisement. These intermediaries have been variously called 'data stores', 'personal data vaults', 'personal information managers' and, most recently for the purposes of the Midata programme, 'midata stores' and 'midata service providers'.

The services provided by these intermediaries vary according to whether they just store, display and/or transmit the data at your request without otherwise processing it; whether they receive data from you or your current supplier; and whether they analyse the data or combine it with other data to produce a result on which you might rely to purchase an alternative product or change your behaviour in some way. 

These intermediaries need contracts with consumers that permit them to access the consumer's transaction data with relevant permissions restrictions, and which agree some form of remuneration. Such contracts would need to go further than the basic service terms of price comparison sites. You might allow them to receive a disclosed commission from a product provider, but at least this would be transparent to you. This marks the line between whether the intermediary is acting for you or the product provider, and there will need to be clarity on this point.


Some people say they don't want suppliers to store any of our transaction data. They want it deleted as soon as it's no longer needed. But it should be clear by now that your transaction history could be very valuable to you, and you should have the option of downloading and storing it and giving it to another provider. Most businesses insist on such 'data portability' when moving from one outsourcing service provider to another, for example, and this can be just as important for consumers and small businesses.

Identity and authentication are also important features of a world in which transaction data is being transmitted. Perviously, I've suggested that proof of identity should be momentary, based on much wider behavioural data than just the static datasets that fraudsters can replicate, and the data used to establish your identity should be discarded straight away, rather than held for re-use. But that is not to say that transaction data itself should be deleted.

The impact of all this on mainstream product providers cannot be underestimated. I've previously drawn the distinction between 'facilitators' who exist primarily to solve customers' problems and 'institutions' who exist primarily to solve their own problems at their customers' expense.  Clearly this new environment will favour product providers who are aligned with consumers' day-to-day activities rather than those who make life awkward because it suits their own profitability. A gulf may well begin to open between product providers as they are tested by this distinction. More confident consumers should be more prepared to spend money with facilitators than those faced with institutions they distrust.

However, I believe that this trend will most likely result in a series of digital platforms on which consumers and suppliers in various markets directly negotiate products and pricing in a transparent way, based on each consumer's transaction data, wherever that is stored. Such platforms have already arrived in so many other consumer markets as part of the Web 2.0 phenomenon, that it's only a matter of time that they arrive in the markets targeted by the Midata programme in any event. And it's only natural that consumers would want to leverage their own transaction data in that context.

Will Midata Turn Institutions Into Facilitators?

The government's warning shot over Midata presents an interesting challenge for some of the UK's institutions. But will it make them focus on solving consumers' problems - transforming them into 'facilitators'? Or will they merely continue to solve their own problems at consumers' expense?

The government wants the suppliers of energy, mobile phones, current accounts and credit cards to provide each of their consumer and small business customers with the records of what they bought, where and for how much. That transaction data must be released in computer-readable format to enable it to be analysed, either by the customer or the customer's authorised service provider. This would help prevent those suppliers from gaining an unfair pricing advantage over consumers, for example, and make it easier for consumers to figure out the product right for them.

Factors the government might consider in deciding whether to expand the programme to other sectors include: 

• the market is not working well for consumers, e.g. consumers find it difficult to make the right choice or their behaviour affects pricing it's difficult to predict that behaviour;

• there's a one-to-one, long-term relationship between the business and the customer, with a stream of ongoing transactions;

• consumer engagement is limited, e.g. low levels of switching or competition; and

• suppliers don't voluntarily provide transaction/consumption data to customers at their request in portable electronic format.

Yet these factors merely hint at the characteristics that an organisation should display if it is to succeed in the future economic environment. In broad terms, the targeted institutions will need to be organised to solve their customers’ problems, operate openly, adapt well to changing circumstances, remain committed to transparency and take responsibility for the impact of their activities on the wider community and society. I've explained these themes in more detail here.

 

The current targets of this programme have a long way to go!

 

I should add that I am involved in the Midata programme, as a member of the Interoperability Board and on the working groups considering issues related to data transmission and law/regulation.

Unload The "Digital Wallet" Before Someone Gets Hurt

And that's not all...

The term "e-wallet" or "digital wallet" has always caused a physical reaction. But what started as a small twitch over my left eye in November 1999 now involves diving under a table. The term has become so loaded with giant concepts like 'identity', 'privacy', 'authentication', 'security', 'payment' and 'funds' that it's simply too dangerous to wave around in meetings.

We need to focus on more of the detail if business presentations are to have any meaning and projects are to deliver anything.

The term 'digital wallet' is impossible to define, anyway. The Oxford English Dictionary has no home for it, and it's wise to ignore suppliers' self-serving, product-specific definitions. Th'internet merely yields a confusing mish-mash: [my emphasis] "a system that securely stores users' payment information and passwords..." (investopedia) and "encryption software that works like a physical wallet during electronic commerce transactions." (webopedia). Unhelpfully, the Free dictionary explains "the wallet data may reside in the user's machine or on the servers of the wallet service. When stored in the client machine, the wallet may use a digital certificate that identifies the authorized card holder." 

Such definitions are confusing because they keep jumping the rails from party to party, feature to feature and function to function, each of which has different implications for transaction flows, data flows and funds flows (to the extent payment is even involved). 

Perhaps the only consistent aspect in the use of the term 'digital wallet' is the sense that it refers to a specific individual, or at least it should be capable of doing so. Otherwise, the term means so many different things that it's useless. FinVentures defined it to mean, "A consumer owned and controlled account that can store any electronic form of what is normally held in a physical wallet, including: payment, ID, coupons, loyalty, access cards, business cards, receipts, keys, passwords, shopping lists, …etc." Indeed, a 'digital wallet' could be a feature within an application or service, or an entire application or service, a database, a set of permissions and so on. It could reside on virtually any digital device, including a smart card or just a microchip. It could enable a specific person to initiate or conclude any kind of transaction, or merely be used in the course of intiating or concluding such a transaction.

So when you next hear the term 'digital wallet', seek cover behind a large, heavy object and try to defuse the situation by asking: 

• which parties are involved;

• which party is agreeing to do what, how do they agree, what actions are taken as a result and by whom;

• where the related data is stored and where it flows; and

• where any related funds are and where they flow.

It could save a lot of time and money.

Image from Tenets in DM.

Rethinking Personal Data

As part of its 'midata' initiative to empower consumers, the department of Business Innovation and Skills has been consulting on a proposal to give the Secretary of State a general power that "might be exercised broadly or in a more targeted way" to compel suppliers to supply transaction data at a consumer’s request. In the interests of transparency, I've summarised my response to the consultation over on The Fine Print. As previously explained, I should disclose that I've been involved in the midata Interoperability Board from its inception in 2011.

On The Futility Of Cookie Consents

It's a month or so since the Cookie Law took effect and already it's an exercise in futility. I haven't clicked on a single cookie consent, yet I know my browser and hard drive are lousy with the things - both the helpful kind that improve my experience of using the web site I'm visiting, and the small proportion that feed information about me to third party advertisers.

There are two reasons for not clicking on cookie consents. 

Firstly, I don't reserve a single minute in my day for reading cookie consents. Life is short. Every second spent not reading cookie consents is a priceless investment in something potentially productive. Sleeping is a better use of time. Not reading cookie consents is in the same category as never watching American celebrity murder trials, or Big Brother or X Factor. Or... well, you get the picture. Reading cookie consents is a true waste of time.

Secondly, the Cookie Law is a one-size-fits-all requirement for user consent before setting all types of cookie - both those that will help you retweet this post and immediately return to read more, as well as those that will lead someone to conclude you have a passion biscuit recipes after you've read this post. I have no problem at all with the first kind, and it seems overkill to ask me to opt-in or out to them being set. I can clear them if I want to. And making me click "I accept" for all types of cookies doesn't even scratch the surface of the very specific, difficult challenges posed by the second kind of cookie: how and why the data about my movements is going to be shared with advertisers, and ensuring it is in fact used appropriately. Those challenges need the pragmatic, holistic attention of a WEF 'tiger team', not the overly zealous intervention of Eurocrats using data protection law as a means of delivering the single market fantasy.

Image from Jefferson Park.

4891: Orwell Had It Backwards

Thanks to George Orwell's Nineteen Eighty Four, and the film adaptation, most of us over 20 have grown up with the threat of an omniscient, totalitarian Big Brother looming over us.

While this is a tragic reality for the residents of a few countries, for most it is not.

Yet many of us are obsessed with our own privacy, imagining it as a defence to control by organised crime lords, governments, a "New World Order" or Facebook. Others relish the illusory voyeurism in the melodramatic Big Brother television series, and the phoney 'privacy battles' conducted between celebrities and the tabloid media by agents and public relations advisors for commercial gain.

But it is actually the overwhelming dislosure of information about ourselves that defies control by any single institution (as does the inherent unpredictability of human behaviour). The Chinese government, in particular, seems to understand this. Sharing our preferences, desires, fears and concerns (if not our birth dates and passwords) via social, retail, political and other facilitators enables us to gain greater personal control of our own lives. That process results in services adapted to our own actual or desired behaviour rather than a service provider's bottom line or a political party's dogmatic manifesto. There are literally millions of examples of this dynamic at work. But consider how:

• The UK recently opted for a coalition government amidst scaremongering by 'big media' and the major political parties.
• Newspapers complain about our 'promiscuity' when it comes to their content, and our refusal to pay for that content online.
• Sharing more personal data often results in better markets, e.g. in consumer credit.
Our networks are always changing, both within platforms and between platforms - we create new networks and still newer ones, and leave old networks very quickly.

Of course, George Orwell was writing a cautionary tale rather than necessarily predicting the future, so we at least have him to thank for a vivid image of how society must not be allowed to develop. In the meantime, we should go on sharing information about ourselves, even if only as a last defence to totalitarian control.

Image from Online Social Networking

Phorm Town Meeting

By the end of Phorm's "2nd Town Hall Meeting" it became obvious that the company is still trying to launch a product with both hands tied behind its back.

It's structure means that Phorm's online behavioural advertising service will only be successful if internet service providers implement it, then successfully market it to individual users, advertisers and web site owners. At that point, the company says, advertisers will experience less wastage in advertising spend, content owners will find it easier to monetise content, web site owners can charge more for space, and end-users will see more relevant ads as they browse.

Exactly what this means in commercial terms is naturally unclear. And Phorm rightly points out that it would be wrong for it to release the details of ISPs' trials or take-up incentives likely to be offered to ISPs' customers, at least until the ISPs are good.. and... ready..... to...... launch....... After 7 years of development, Phorm says it has learned to be patient - a revolution in the internet space.

It seems fairly pointless to have public meetings to talk about offering "choice" when you have no product in the market and the meat of your proposition is under wraps for commercial or regulatory reasons. Nevertheless, Phorm chose the opportunity to engage in further damage limitation on the privacy front and to set the commercial context for its service with a rundown on the online advertisting market.

All the legal points have been made on the privacy front, and don't bear repeating here - though I'll summarise them at the SCL's Information Governance conference. Phorm seems to think they've all gone away, or will be made to go away by launch. Network opt-out was mentioned. Network opt-in is preferred, as is a way to block the service altogether, so that I don't need to store either their opt-in or opt-out cookies. Having to choose whether to store Phorm's opt-in or opt-out cookies is only a choice about how you use Phorm's service, not a choice between using its service and not. Phorm says the current cookie practices are less transparent than its own service will be. From a user standpoint this doesn't deal with the point that I can choose not to go to certain sites, and to clear their cookies selectively, but I can't as readily avoid Phorm's service - or choose to use it on some sites and not others - if it's being run at the ISP level. That "choice" doesn't feel very personalised at all, and personalisation is at the heart of how the web is developing. Phorm asks why the likes of [Google and Facebook] don't have "town meetings" to explain their privacy policies and settings, but I can't think of a venue big enough - and of course they do constantly explain and respond to privacy queries from their massive, global communities in a very public way, online, where everyone can participate.

Phorm also appears to be creating some kind of moral panic by saying that it is part of the solution to preserving the humble newspaper - not to mention journalistic integrity. Shock, horror: journalists are apparently being asked to insert certain keywords in their stories to help attract the right traffic to their newspaper's online ads. Apparently, if Phorm were implemented and used by [everybody] content publishers would not [have to] do this. But the newspapers I read from time to time don't seem all that averse to coupling themes and stories with advertising in their offline manifestations, so it's hardly the end of the world as we know it. And I don't see how newspapers can escape people's desire to see their content unbundled any more than the record companies could. Their challenge is to keep innovating, as Eric Schmidt told US newspapers yesterday. Phorm suggests that the major ad service operators (Google, Facebook et al) aren't entitled to their current or growing flows of advertising revenues. The market will no doubt decide, but this suggestion ignores how those companies finance their own core businesses, which millions and millions of people clearly find very compelling - apparently more so than limited bundles of "news". It also ignores the importance of search and online communities for newspapers' content, not to mention ad deals.

Ulimately, comparisons with Google and Facebook highlight the fact that Phorm is not a bottom-up phenomenon. It's something that will only happen if big telecoms providers say so, and that collides with the Web 2.0 ethos. This, coupled with the Orwellian privacy issues - whether real or perceived - makes Phorm's marketing job very much harder.

How to Disable Phorm

I looked at Phorm in February and again in June, but not really wearing my consumer hat. Now I have an unwelcome opportunity to do just that.

You see, I'm a BT broadband subscriber with multiple users at home, some of whom may not be all that, ahem, technologically inclined. So I'm a bit paranoid that, while I'm not aware of having been asked or consented to using Phorm (branded "WebWise"), other users may have inadvertently switched it on in the course of a BT trial.

Why I am paranoid? Well the service is basically designed to track the browsing habits of all users of the broadband-connected PC or laptop and use this to send more targeted advertising, so that BT and Phorm can make money out of you. But I don't just "browse", I research stuff, work and look after my financial affairs. Other users in the house from time to time will do the same. I don't want this stuff tracked, scanned or whatever else Phorm or BT plan to do with it. And I don't want to be pestered by ads, especially ones that may have nothing do with my real interests. I don't consider that I have a relationship with BT when I use my broadband to access the internet. I permission or de-permission cookies or accept marketing bumph from each of the site I'm happy to deal with. And so on.

I've now done what any good consumer should do. I've looked at the BT WebWise site and even the audit report from Ernst & Young (the mere fact that an audit report is felt necessary chills me to the bone). While these purport to tell me what Phorm is or isn't doing, it doesn't explain BT's role or the data it has access to and retains, or what BT is getting out of using Phorm. The BT terms and conditions (clause 18) aren't exactly encouraging on this point. In fact they are so lacking in material information that they deserve further consideration in light of the Consumer Protection from Unfair Trading Regulations 2008 (which I perhaps rather hastily lampooned - but hey, if they're there, use them). The killer is that the mere presence of this unwelcome "service" casts on me an obligation to constantly police my own computer and all its users to ensure that we're opted-out and remain opted-out. It would be too much to hope that the anti-virus software providers will create a Phorm-killer.

Let's be clear. BT needs to persuade me, as its customer, to opt-in to taking this additional "service". It's not for BT to use my broadband connection to build relationships with people who aren't the accountholder, and get me to police their opt-in/opt-out. It must be BT's problem to ensure that if I don't opt-in (or if I do, but opt-out later) that the effective opt-out works for everybody on my connection all the time.

And to have any chance of persuading me to opt-in, BT must specify in more detail the nature of the data that will be obtained, all the proposed uses of that data, what I am going to receive in return (and don't say targeted ads - show me the reduction in the price of broadband to reflect your opportunity to gain ad revenue), and how I can opt-out and have that data deleted. From a personal standpoint, the "WebWise" service doesn't go far enough in this regard for me to trust it. Nor should the current level of disclosure be enought for BT to be able to claim they have my consent to thing under the Data Protection Act - I simply don't consent, anyway.

So, not trusting BT on the particular issue of how to stay opted out, I did a quick Google search hoping to learn how you would really know that you were not signed up, and how to switch it off completely. No luck.

The Register, which has done a lot of digging on Phorm in the past, and got a very concerning post from Chris Williams on 3 October. According to Chris' discussions with BT, they seem to track your usage whether you're opted in or out... so they can record whether you have opted in or out. You then simply have to trust that they won't sell or otherwise use your data to get extra ad revenue, fall victim to organised criminals, or allow the authorities to mash it with the Communications Database (you'll recall that the UK government has been particularly supportive of Phorm).

All the technical detail is in Richard Clayton's excellent piece on Phorm. His research suggests that you can add the Fraud Act, Computer Misuse Act and the Regulation of Investigatory Powers Act to your reading list before deciding whether or not to sign up to WebWise. And even intellectual property rights owners have a serious set of bones to pick, as Nicholas Bohm and Joel Harrison have fulsomely discussed in their excellent September article for the Society for Computers and Law. But none of that is going to occur to the average consumer, so why is the government not taking their corner instead of Phorm's..?

Who knows. For my money, it's time to switch broadband providers.

Speaking of which, I see that Orange is attempting to make a virtue out of not using Phorm.

UK Govt Backs Phorm PR Effort

It's one thing for the UK Government to support Phorm's challenge to personal privacy at EU level while defending its position in the face of European Commission concerns.

But it's quite another to be seen to selectively release to the media the portions of its letter to the European Commission that list the ways in which officials believe Phorm to be a good thing.

Bad Phorm, in fact.

It would seem that the authorities may have something to lose if Phorm isn't a success...

Bad Phorm?

Back in February, I commented on the Open Internet Exchange initiative being planned by Phorm, whereby and major ISP partners BT, Virgin Media and Talk Talk will be paid for allowing all the web browsing by their customers to be trawled for advertising purposes.

Not a lot was known about the initiative at the time, but negative news has been snowballing since, and opponents are taking to the streets. The Register is maintaining a dossier, known as "The Phorm files", and a "No Deep Packet Inspection" street demonstration is timed for BT's AGM on 16 July 2008. See also the Facebook Group "Save UK internet privace - reject ISPs that use Phorm".

Incidentally, you might wish to be more wary than usual of the Wikipedia entry on this subject.

The concerns raised are similar to those related to Facebook's "Beacon" initiative that led FB to significantly alter the functionality (though you might wish to be somewhat sceptical of that Wikipedia entry too!). The chief one being that there seems no reliable way to ensure that you are really opted-out. However, the Phorm scenario is worse than with Beacon, because the inspection, storage and use of data is at the ISP layer, making it much harder in practical terms to avoid the service than if it was operated, say, on a site-by-site basis. In other words, you can't decide simply not to visit certain sites if you doubt that the opt-out would actually prevent the abuse of your personal data. Instead, you would need to switch ISPs. However, you may not actually be able to avoid using one of the "problem" ISPs (e.g. at a friend's place, work, or via an internet cafe). And what if all the ISPs join the initiative?

Further, as the Guardian has noted, the challenge for Phorm is to reconcile two apparently contradictory statements:

"Advertisers are told that it will be able to profile the surfers, based on where they have visited, and target them through that uniquely numbered cookie. But users are told they will not be identifiable. It's the apparent contradiction in those statements that has infuriated so many."

If you are remotely concerned, now is the time to make your feelings known to your ISP, your MP, and participating advertisers.