Do You Know You Are Using And/Or Exposed To Artificial Intelligence?

Source: datamation

Amid the news that the Department of Work and Pensions is using artificial intelligence to decide benefits claims, a third of UK financial firms recently told the Bank of England and Financial Conduct Authority they were not using 'machine learning' (a subset of AI) in their businesses. That's pretty concerning, given the kind of AI we all use without realising and the fact that anyone wrongly denied benefits will need short term finance. That response also begs the question whether those firms know how their many stakeholders are using AI (whether unwittingly or not). If their suppliers, intermediaries, customers, investors, competitors and the government are using AI, then how does that affect their own strengths, weaknesses, opportunities and threats? And how does that in turn affect their stakeholders? No city on Earth is ready for the disruptive effects of artificial intelligence. Also worrying is the finding that smaller, fintech firms seem to believe that machine learning is no use to them. And given the problems with AI explained below, it's important for everyone to consider whether and how they rely on or are otherwise exposed to the two thirds of financial firms who are actually using AI... hyping the benefits without understanding the shortcomings will harm our trust AI where it could be helpful.

What is AI?

The term "AI" embraces a collection of technologies and applications, with machine learning usually featuring at some point:

• narrow artificial intelligence
• machine learning
• artificial neural networks 
• deep learning networks 
• automation
• robotics
• autonomous vehicles, aircraft and vessels
• image and facial recognition
• speech and acoustic recognition
• personalisation
• Big Data analytics

At the recent SCL event I chaired in Dublin, Professor Barry O'Sullivan explained that AI technologies themselves may be complex, but the concepts are simple. In essence, machine learning differs from traditional computer programming in that:

• traditionally, we load a software application and data into a computer, and run the data through the application to produce a result (e.g. how much I spent on coffee last year);

• while machine learning involves feeding the data and desired outputs into one or more computers or computing networks that are designed to write the programme (e.g. you feed in data on crimes/criminals and the output of whether those people re-offended, with the object of producing a programme that will predict whether a given person will re-offend). In this sense, data is used to ‘train’ the computer to write and adapt the programme, which constitutes the "artificial intelligence".

What is AI used for?

AI is used for:

1. Clustering: putting items of data into new groups (discovering patterns);
2. Classifying: putting a new observation into a pre-defined category based on a set of 'training data'; or
3. Predicting: assessing relationships among many factors to assess risk or potential relating to particular conditions (e.g. creditworthiness).

The Challenges with AI

The primary concerns about AI relate to:

1. cost/benefit - $50m in electricity to teach an AI to beat a human being at Go, hundreds of attempts to get a robot to do a backflip, but it can't open a door;
2. dependence on data quantity, quality, timeliness and availability;
3. lack of  understanding - AI is better at some tasks than humans (narrow AI) but general AI (same as humans) and superintelligence (better than humans at everything) are the stuff of science fiction. The AI that can predict 79% of European Court judgments doesn't know any law, it just counts how often words appear alone, in pairs or fours;
4. inaccuracy - no AI is 100% accurate;
5. lack of explainability - machine learning involves the computer adapting the programme in response to data, and it might react differently to the same data added later, based on what it has 'learned' in the meantime; 
6. the inability to remove both selection bias and prediction bias - adding a calibration layer to adjust the mean prediction only fixes the symptoms, not the cause, and makes the system dependent on both the prediction bias and calibration layer remaining up to date/aligned over time; 
7. the challenges associated with the reliability of evidence and how to resolve disputes arising from its use; and
8. there's a long list of legal issues, but lawyers aren't typically engaged in development and deployment.

This means the use of AI cannot be ignored. We have to be careful to understand the shortcomings and avoid hyping the benefits if we are to ensure trust in AI. That means challenging its use where the consequences of selection bias or false positives/negatives are fatal or otherwise unacceptable, such as denying fundamental rights or compensation for loss.

Instead, artificial neural networks and deep learning are better used to automate decision-making where "the level of accuracy only needs to be "tolerable" for commercial parties [who are] interested only in the financial consequences... than for individuals concerned with issues touching on fundamental rights."

Being realistic about AI and its shortcomings also has implications for how it is regulated. Rather than risk an effective ban on AI by regulating it according to the hype, regulation should instead focus on certifying AI's development and transparency in ways that enable us to understand its shortcomings to aid in our decision about where it can be developed and deployed appropriately.

Brits Look Away Now: Free Movement Of Non-Personal Data In the EU

The EU's "Digital Single Market" strategy has been boosted by an agreement to remove requirements for non-personal data to be stored in any one EU member state. The new law, approved in plenary by 520 votes to 81, with six abstentions, is due to be approved by the EU Council of Ministers on 6 November. It will apply six months after its publication in the EU Official Journal.

Restrictions on the free movement of personal data have long been relaxed under the EU data protection framework. The latest move is expected to double the value of the EU data economy to 4% of GDP by 2020.

The agreement builds on a report into "Building a European Data Economy",  statistics on the use of cloud computing by businesses, a study on the legal and administrative restrictions on the free flow of non-personal data (often mistaken) and study of the economic benefits of cloud computing uptake and free flow of data within the EU.

In summary, the agreement means that:

• public authorities cannot impose "unjustified" data localisation restrictions;

• the data remains accessible for regulatory and supervisory control even when stored or processed across borders within the EU;

• cloud service providers are encouraged to develop self-regulatory codes of conduct for easier switching of provider and porting data back to in-house servers, by mid-2020;

• Security requirements on data storage and processing remain applicable when businesses store or process data in another Member State or outsource data processing to cloud service providers;

• Single points of contact in each Member State will liaise with other Member States’ and the Commission to ensure the effective application of the new rules. 

The Irony Or The Ecstasy? The UK Centre For Data Ethics And Innovation

You would be forgiven for uttering a very long string of properly ripe expletives on learning that the current UK government has the cheek to create a "Centre for Data Ethics and Innovation"!  Personally, I think they've missed a trick with the name. With a little more thought, the acronym could've been "DECEIT" - and maybe in some languages it would be - so let's go with that.

You might say that it's better to have, rather than not have, an 'independent' policy body focused on the use of data and "artificial intelligence", even if it's set up by a government controlled by those who masterminded and/or benefited from the most egregious abuse of data ethics in UK history.

Or you might be relieved by the notion that it's easier for the dominant political party of the day to control the ethical agenda and results achieve "the best possible outcomes" if the source of policy on data ethics is centralised, especially within a body being hastily set up on the back of a quick and dirty consultation paper released into the febrile, Brexit-dominated summer period before any aspirational statutory governance controls are in place. 

At any rate, we should all note that:

"[DECEIT], in dialogue with government, will need to carefully prioritise and scope the specific projects within its work programme. This should include an assessment of the value generated by the project, in terms of impact on innovation and public trust in the ethical use of data and AI, the rationale for [DECEIT] doing the work (relative to other organisations, inside or outside Government) and urgency of the work, for example in terms of current concerns amongst the public or business."

...

"In formulating its advice, the Centre will also seek to understand and take into consideration the plurality of views held by the public about the way in which data and AI should be governed. Where these views diverge, as is often the case with any new technology, the Centre will not be able to make recommendations that will satisfy everyone. Instead, it will be guided by the need to take ethically justified positions mindful of public opinion and respecting dissenting views. As part of this process it will seek to clearly articulate the complexities and trade offs involved in any recommendation."

Political point of view is absolutely critical here. This UK government does not accept that the Leave campaign or Cambridge Analytica etc did anything 'wrong' with people's data. Senior Brexiteers say the illegality resulting in fines by the Electoral Commission and further investigation by the ICO and the police are merely politically motivated 'allegations' by do-good Remainers. Ministers have dismissed their own "promises" (which others have called "fake news" outright lies and distortion) as merely "a series of possibilities". There is no contrition. Instead, the emerging majority of people who want Brexit to be subjected to a binding vote by the electorate are regarded as ignoring "public opinion" or "the will of the people" somehow enshrined forever in a single advisory referendum in 2016; and as therefore expressing merely "dissenting views".

Against this gaslit version of reality, the creation of DECEIT is chilling.

Meanwhile, you might ask why there needs to be separate silo for "Data Ethics and Innovation" when we have the Alan Turing Institute and at least a dozen other bodies, as well as the Information Commissioner, Electoral Commission and the police. Surely the responsibility for maintaining ethical behaviour and regulatory compliance are already firmly embedded in their DNA?

I did wonder at the time of its formation whether the ATI was really human-centric and never received an answer. And it's somewhat worrying that the ATI has responded to the consultation with the statement "We agree on the need for a government institution to devote attention to ethics". To be fair, however, one can read that statement as dripping with irony. Elsewhere, too, the ATI's response has the air of being written by someone with clenched teeth wondering if the government really knows what it's doing in this area, anymore than it knows how to successfully exit the EU:

We would encourage clarity around which of these roles and objectives the Centre will be primarily or solely responsible for delivering (and in these cases, to justify the centralisation of these functions), and which will be undertaken alongside other organisations.

... We would encourage more clarity around the Centre’s definitions of AI and emerging technologies, as this will help clarify the areas that the Centre will focus on.

Reinterpreting some of the ATI's other concerns a little more bluntly yields further evidence that the ATI smells the same rat that I do:

• DECEIT will have such a broad agenda and so many stakeholders to consider that you wonder if it will have adequate resources, and would simply soak up resources from other stakeholders without actually achieving anything [conspiracy theorists: insert inference of Tory intent here, to starve the other stakeholders into submission];
• the summary of "pressing issues in this field" misses key issues around the accountability and audibility of algorithms, the adequacy of consent in context and whether small innovative players will be able to bear inevitable regulations;
• also omitted from the consultation paper are the key themes of privacy, identity, transparency in data collection/use and data sharing (all of which are the subject of ongoing investigation by the ICO, the police and others in relation to the Leave campaign);
• the ATI's suggested "priority projects" imply its concern at the lack of traction in identifying accountability and liability for clearly unethical algorithms;
• powers given to DECEIT should reinforce its independence and "make its abolition or undermining politically difficult";
• DECEIT's activities and recommendations should be public;
• how will the "dialogue with government" be managed to avoid DECEIT being captured by the government of the day?
• how will "trade offs", "public opinion" and "dissenting views" be defined and handled (see my concerns above)?

I could add to this list concerns about the government's paternalistic outlook instead of a human-centric view of data and technology that goes beyond merely 'privacy by design'. The human condition, not Big Tech/Finance/Politics/Government etc, must benefit from advances in technology.

At any rate, given its parentage, I'm afraid that I shall "remain" utterly sceptical of the need for DECEIT, its machinations and output - unless and until it consistently demonstrates its independence, good sense, not to mention ethics. 

BigTech Must Reassure Us It's Human

Recent issues concerning the purchase of lethal materials online, "fake news" and secure messaging highlight a growing tension between artificial intelligence and human safety. To continue their unbridled growth, the tech giants will have to reassure society that they are human, solving human problems, rather than machines solving their own problems at humans' expense. While innovation necessarily moves ahead of the law and regulation, developments in artificial intelligence should be shaped more by humane and ethical considerations, rather than outsourcing these to government or treating them as secondary considerations.

In the latest demonstration of this concern, Channel 4 researchers were able to assemble a 'shopping basket' of potentially lethal bomb ingredients on Amazon, partly relying on Amazon's own suggestion features or 'algorithms' ("Frequently bought together” and “Customers who bought this item also bought...”), which even suggested adding ball-bearings. This follows the phenomenon that emerged during the Brexit referendum and US Presidential election whereby purveyors of 'fake news' received advertising revenue from Facebook while targeting gullible voters.

Neither business is keen to proactively monitor or police its services for fear of conceding an obligation to do so and rendering itself liable for not doing so where the monitoring fails.

Channel 4 quoted Amazon as merely saying that:

"all products must adhere to their selling guidelines and all UK laws. [We] will work closely with police and law enforcement agencies should they need [us] to assist investigations." [update 20.09.17: Amazon is reported to have responded the next day to say that it is reviewing its website to ensure the products “are presented in an appropriate manner”.]

Amazon makes a valid point. After all, the same products can be bought off-line, yet unlike an offline cash purchase in a walk-in store, if they are bought on Amazon there is likely to be a digital 'audit trail' showing who bought what and where it was delivered. Indeed, it's conceivable that Amazon had alerted the authorities to the nature of the items in Channel 4 researchers' shopping basket and the authorities may have allowed the session to run as part of a potential 'sting' operation. It is perhaps understandable that neither Amazon nor the authorities would want to explain that publicly, but it would be comforting to know this is the case. Channel 4 is also somewhat disingenuous in suggesting this is an Amazon problem, when less well-resourced services or other areas of the Internet (the 'dark web') may well offer easier opportunities to purchase the relevant products with less opportunity for detection.

At any rate, the main difference, of course, is that no one from an offline store is likely to help you find missing ingredients to make a potentially lethal device (unless they're already part of a terror cell or perhaps an undercover operative) - and this is the key to Amazon's enormous success as a retail platform. It's possible, however, that a helpful employee might unwittingly show a terrorist where things are, and Amazon might equally argue that its algorithms don't "know" what they are suggesting. But whether it's because of the 'promise' of the algorithms themselves, there is a sense that the algorithm should not be vulnerable to abuse in this way.

Similarly, in the case of Facebook, the social network service has become a raging success because it is specifically designed to facilitate the exchange of information that generates passionate connections amongst like-minded people far more readily than, say, the owner of a bar or other social hang-out or a newspaper or other form of traditional media. Equally, however, Facebook might argue that the helpful algorithms aren't actually "aware" of the content that is being shared, despite use of key words etc. Meanwhile, WhatsApp seems to have declined to provide a terrorist's final message because it could not 'read' it (although the authorities seem to have magically accessed it anyway...).

Just as we and the online platform owners have derived enormous benefit from the added dimensions to their services, however, we are beginning to consider that those dimensions should bring some additional responsibilities - whether merely moral or legal - possibly on both users and service providers/developers.

In many ways the so-called 'tech giants' - Apple, Amazon, Alphabet (Google), Facebook and others - still seem like challengers who need protection. That's why they received early tax breaks and exemptions from liability similar to those for public telecommunications carriers who can't actually "see" or "hear" the content in the data they carry. 

But while it's right that the law should follow commerce, intervening only when necessary and in a proportionate way to the size and scale of the problem, the size and reach of these platforms and the sheer pace of innovation is making it very hard for policymakers and legislators to catch up - especially as they tend to have wider responsibilities and get distracted by changes in government and issues like Brexit.  The technological waves seem to be coming faster and colliding more and more with the 'real world' through drones and driverless cars, for example. 

The question is whether these innovations are creating consequences that the service providers themselves should actively address, or at least help address, rather than ignore as 'externalities' that government, other service providers or society must simply cope with.

The tech giants are themselves struggling to understand and manage the scale and consequences of their success, and the relentless competition to attract the best talent and the race to push the boundaries of 'artificial intelligence' sometimes presents as a declaration of war on the human race. Even the government/university endowed Alan Turing Institute seems to consider the law and ethics as somehow separate from the practice of data science. Maybe algorithms should be developed and tested further before being released, or be coded to report suspicious activity (to the extent they might not already).  Perhaps more thought and planning should be devoted to retraining commercial van and truck drivers before driverless vehicles do to them what the sudden closure of British coal mines did to the miners and their communities (and what the closure of steel mills has done since!).

In any event, the current approach to governance of algorithms and other technological leaps forward has to change if the 'bigtech' service providers are to retain their mantle as 'facilitators' who help us solve our problems, rather than 'institutions' who just solve their own problems at their customers' expense. They and their data scientists have to remember that they are human, solving human problems, not machines solving their own problems at humans' expense.

[update 20.09.17 - It was very encouraging to see Channel 4 report last night that Amazon had promptly responded more positively to researchers' discovery that automated suggestion features were suggesting potentially lethal combinations of products; and is working to ensure that products are "presented in an appropriate manner". The challenge, however, is to be proactive. After all, they have control over the data and the algorithms. What they might lack is data on why certain combinations of products might be harmful in a wider context or scenario.]

Better Services For SMEs: Follow The Data

I was at a 'parliamentary roundtable' on Tuesday on the perennial topic of small business banking reform. A more official report will be forthcoming, but I thought I'd record a few thoughts in the meantime (on a Chatham House basis). 

It still seems to surprise some people that small businesses represent 95% of the UK's 5.4m businesses - 75% of which are sole traders - and that they account for 60% of private employment, most new jobs and about half the UK's turnover. So-called 'Big Business' is just the tip of the iceberg, since only they have the marketing and lobbying resources to be seen above the waves. As a result small businesses have long been a blind spot for the UK government - until very recently - and the impact has gone way beyond poor access to funding. It includes slow payment of invoices, the absence of customer protection when dealing with big business and lack of alternatives to litigation to resolve disputes.

What's changed?

A combination of financial crisis, better technology and access to data has exposed more of the problems surrounding SMEs - and made it possible to start doing something about them. And it's clear that legislators are prepared to act when they are faced with such data. The EU Late Payments Directive aimed to eliminate slow payments. The UK has created the British Business Bank to improve access to finance, as well as a mandatory process for banks to refer declined loan applications to alternative finance providers and improved access to SME credit data to make it easier for new lenders to independently assess SME creditworthiness. The crowdfunding boom has also been encouraged by the UK government, and has produced many new forms of non-bank finance for SMEs, including equity for start-ups, debentures for long term project funding, more flexible invoice trading and peer-to-peer loans for commercial property and working capital.  Last week, the FCA launched a discussion paper on broadening its consumer protection regime to include more SMEs.

Yet most of these initiatives are still to fully take effect; and listening to Tuesday's session on the latest issues made it clear there is a long way to go before the financial system allocates the right resources to the invisible majority of the private sector.

A key thread running through most areas of complaint seems to be a lack of transparency - ready access to data. This seems to be both a root cause of a lot of problems as well as the reason so many proposed solutions end up making little impact. But the huge numbers and diversity of SMEs presents the kind of complexity that only data scientists can help us resolve if we are to address the whole iceberg, rather than just the tip. That's surely one job for the newly launched Alan Turing Data Institute, for example, although readers will know of my fear that it seems more aligned with institutions than the poor old sole trader, let alone the consumer. So maybe SMEs need their own 'Chief Data Scientist' to champion their plight?

The latest specific concerns discussed were as follows:

• despite much regulatory activity, few bank reforms have actually taken effect - and many are being watered down - with the result that the banking picture is worse, if anything, than it was in 2008. Recent fines and scandals reveal little change in culture from that described in the report of the Parliamentary Commission on Banking Standards and most recently in the damning report into the failure of HBOS. In addition, banking is more concentrated (despite a few new entrants); less diverse in terms of ownership structure, competition, regional coverage and financial model; and still only about £200bn or 3% of bank assets are focused on SMEs and GDP producing activities. The trouble is we've known this for some time, but aside from the BBB's activity on access to funding, we still seem unclear on what can be done for SMEs at a regional or local level;

• the recent findings and remedies proposed by the Competition and Markets Authority into business current accounts are widely considered to be weak and unlikely to be effective - try searching the word "data" in the report to see how often there was too little available. The report still feels like the tip of an iceberg rather than a complete picture of the market and its problems;

• austerity imperatives seem to be the main driver for off-loading RBS into private shareholder ownership - the bank pleading to be left to its own devices (not what it suddenly announced to the Chancellor in 2008!) - and trying to kill-off any further discussion of using its systems as a platform for a network of smaller regionally-focused banks (as in Germany);

• bank lobbying has become more powerful - and will become even more so with the merger of their various mouthpieces - we need much more transparency on banks' access to government and the revolving door between the City and Whitehall/Westminster - we should have learned long ago that the interests of large banking groups are not aligned with SMEs (or consumers);

• SMEs lack an effective means of resolving disputes with banks, as the Financial Ombudsman Service has limited jurisdiction; banks refuse to release internal reports related to credit decisions and calling in loans; while the FCA is said to be hiding behind its confidentiality obligations; and banks have too much legal fire power for SMEs to take them on in court - hence moves to take matters into their own hands. The FCA has clearly woken up to this, but it's recent discussion paper shows there's a huge gap to close.

• the financial infrastructure for SMEs appears not to be geographically diverse - it doesn't yet mirror the Chancellor's "Northern Powerhouse" policy, for instance - despite calls for bank transparency on the geographical accessibility, a US-style "Community Reinvestment Act" and clear reporting on lending to SMEs by individual banks (rather than the Bank of England's summary reporting). There's a sense that we should see some kind of financial devolution to match political devolution, albeit one that still enables local finance to leverage national resources and economies of scale. Technology should help here, as we are tending to use the internet and mobile apps quite locally, despite their global potential;

• Some believe that SMEs need to take more responsibility for actively managing their finances, including seeking out alternatives and switching; while others believe that financial welfare should be like a utility - somehow pumped to everyone like water or gas, I assume - indeed regional alternative energy companies were touted as possible platforms for expanding access to regional financial services. My own view is that humans are unlikely to become more financially capable, so financial and other services supplied in complex scenarios need to be made simpler and more accessible - we should be relying less on advertising and more on hard data and personalised apps in such instances.

• Meanwhile, SME are said to lack a genuine, high profile champion whose role it is to ensure that the financial system generally is properly supportive of them. This may seem a little unfair to the Business Bank, various trade bodies and government departments, but it's also hard for any one of these bodies to oversee the whole fragmented picture. As I suggested above, however, I wonder whether a 'data champion' could be helpful to the various stakeholder in identifying and resolving problems, rather than a single being expected to act as a small business finance tsar. 

 In other words, we should follow the data, not the money...

The Alan Turing Institute: Human-centric?

A slightly dispiriting day at The Alan Turing Institute 'Financial Summit', yesterday, I'm afraid to say. 

The ATI itself represents a grand vision and stunning organisational achievement - to act as a forum for focusing Britain's data scientists on the great problems of the world. Naturally, this leaves it open to attempts at 'capture' by all the usual vested interests, and its broad remit means that it must reflect the usual struggle between individuals and organisations and between 'facilitators', who exist to solve their customers problems, and 'institutions', who exist to solve their own problems at their customers' expense. 

And of course, it's the institutions that have most of the money - not to mention the data problems - so I can see, too, why the ATI advertises its purpose to institutions as "the convener of a multidisciplinary approach to the development of 'big data' and algorithms". It's true also, that there are global and social issues that transcend the individual and are valid targets for data scientists in combination with other specialists. 

But it was concerning that an apparently neutral event should seem predicated on a supplier-led vision of what is right for humans, rather than actually engineering from the human outward - to enable a world in which you to control what you buy and from whom by reference to the data you generate rather than by approximating you to a model or profile. Similarly, it was troubling to see a heavy emphasis in the research suggestions on how to enable big businesses to better employ the data science community in improving their ability to crunch data on customers for commercial exploitation.  

To be fair, there were warning signs posted for the assembled throng of banks, insurers and investment managers - in the FCA's presentation on its dedication to competition through its Innovation Hub; a presentation on the nature and value of privacy itself; and salutary lessons from a pioneer of loyalty programmes on the 'bear traps' of customer rejection on privacy grounds and consumers' desire for increasing control over the commercial use of our data. The director's slides also featured the work of Danezis and others on privacy-friendly smart metering and a reference to the need to be human-centric.  

But inverting the institutional narrative to a truly human-centric one would transform the supplier's data challenge into one of organising its product data to be found by consumers' machines that are searching open databases for solutions based on actual behaviour - open data spiders, as it were  - rather than sifting through ever larger datasets in search of the 'more predictive' customer profile to determine how it wastes spends its marketing budget.

Personally, I don't find much inspiration in the goal of enabling banks, insurers and other financial institutions to unite the data in their legacy systems to improve the 'predictive' nature of the various models they deploy, whether for wholesale or retail exploitation, and I'm sure delegates faced with such missions are mulling career changes. Indeed, one delegate lightened the mood with a reference to 'Conway's Law' (that interoperability failures in software within a business simply reflects the disjointed structure of the organisation itself). But it was clear that financial institutions would rather leave this as an IT problem than re-align their various silos and business processes to reflect their customers' end-to-end activities. There is also a continuing failure to recognise that most financial services are but a small step in the supply chain, after all. I mean, consider the financial services implications of using distributed ledgers to power the entertainment industry, for example... 

When queried after the event as to whose role it was to provide the 'voice of the customer', the response was that the ATI does not see itself as representing consumers' or citizens' interests in particular. That much is clear. But if it is to be just a neutral 'convenor' then nor should the ATI allow itself to be positioned as representing the suppliers in their use and development of 'big data' tools - certainly not with £42m of taxpayer funding. 

At any rate, in my view, the interests of human beings cannot simply be left to a few of the disciplines that the ATI aims to convene along side the data scientists - such as regulators, lawyers, compliance folk or identity providers. The ATI itself must be human-centric if we are to keep humans at the heart of technology.

Video Did Not Kill The Radio Star

So many highlights from the past few days at the SCL's Technology Law Futures Conference on 'Keeping Humans at the Heart of Technology', available online soon, but a favourite quote is that "video did not kill the radio star" (with apologies to the Buggles). 

A more fulsome report will no doubt be available shortly. In the meantime, we should all be thinking about the responsible development of 'artificial narrow intelligence' in the context of social care, driver-less cars and other autonomous vehicles (not to mention surveillance and warfare, to the extent we can influence that!). 

If we can insist on adequate transparency and appropriate rules governing risk and liability in the context of these types of projects, then maybe we'll be in good shape to deal with 'artificial general intelligence' when that comes, as well as the potential for artificial superintelligence beyond... [drum roll]... The Singularity.  

Or we could simply fade away as the machines take over.

It's up to us.

For now.

Artificial Intelligence: The Control Dichotomy

Professor Nick Bostrom delivered an inspirational speech for the SCL last night on "Superintelligence: a godsend or a doomsday device", although few would have found it reassuring - it is certainly conceivable that machines could become more intelligent than humans and that humans might not be able to control them. But these are still early days, he stresses. Regulating the development of artificial intelligence at this point risks halting progress. There's a lot more work to do to really understand how artificial intelligence will develop beyond playing old video games better than humans or recognising an image as the picture of a cat. We need to consider how the technology could help avert our extinction, as well as how it might wipe us out. Yet Nick also warns that it will take far less time for computers to exceed human level machine intelligence than to reach it in the first place. So we need to start work on the control mechanisms for the development and use of artifical intelligence, without regulating the industry out of existence: the control dichotomy.

Nick suggests that the guiding principle should be that of "differential technological development" - diverting resources away from technologies and their application which could cause human extinction while focusing on those which will either help prevent our demise or will facilitate the expansion of the human race throughout the cosmos.

But how do we distinguish between helpful and harmful technologies and their application? 

As Nick points out, it's tough to think of any human invention that is inherently 'good'. He mentions many things, from gun powder to genetic engineering, and I think we can throw in the wheel and the Internet for good measure. All these things are used by humans in bad ways as well as for the greater good. But I think what horrifies us especially about the idea of Superintelligence or 'The Singularity' is that it will be machines, not bad humans, who will be using other machines against us. And while we have lots of experience in dealing with evil humans, even our top minds admit we still don't know much about how machines might act in this way or how to stop them - and what we humans fear most is the unknown. 

You'll notice I haven't said 'evil' machines, since they might not be operating with any evil 'intent' at all. Human extinction might just be a mistake - 'collateral damage' arising from some other mission. For instance, Nick suggests that a particular machine left to itself in a given situation might decide to devote itself entirely to making paperclips. So, presumably, it would not bother to put out a fire, for example, or free a human (or itself) from a burning building. It might leave that to other machines, who might in turn have chosen their own narrow objective that involves ignoring people's screams.

Here's where I struggle with the notion of Superintelligence. In fact, as someone who hates being pigeon-holed into any single role, I think a machine's decision to only ever make paperclips might be fabulously logical and a brilliant choice for that machine in the circumstances, but it makes the machine as dumb as a post. For me, Superintelligence should involve a machine being able to do everything a human can and more. 

But that's beside the point. Knowing what we know already, it would be insane to ignore the paperclip droid and wait for artificial intelligence to develop a machine more capable than humans before figuring out how we might control it. Nick is right to point out that we must figure that out in parallel. In other words, the concept of human control has to be part of the artificial intelligence programme. But it won't be as simple as coding machines to behave protectively, since machines will be able to programme each other. For instance, Nick suggests we could put the machines to work on the control problem, as well as on the problem of how to ensure the survival of our species. AI labs might also pay insurance premiums to cover the damage caused by what they develop. He was less certain about what we might do to constrain developments that occur in the context of secret defence programmes or intelligence gathering, but he seemed confident that we could at least infer the pace of development from the results, and be able to consider how to control the wider application of those developments. Mmmm.

At any rate, Nick also warns that we need to be careful what we wish for. Mandating human survival in a prescriptive way - even a specific biological form - would be a bad move, since we should not assume we are in a position to foster positive human development any more than the Holy Office of the Spanish Inquisition. Better to embed positive human values and emotions or, say, entertainment as a feature of intelligent machines (although I'm guessing that might not go down well with the jihadis). From a phyiscal standpoint, we already know that the human body won't do so well for long periods in space or on Mars, so some other version might need to evolve (okay, now I'm freaking myself out).

To retain a sense of pragmatism, at the end of the speech I asked Nick what he would recommend for our focus on 'Keeping Humans at the Heart of Technology' at the SCL conference in June. His tip was to consider which of the various types of control mechanism might work best, recognising the need to avoid constraining the positive development of artificial intelligence, while ensuring that we will be able to keep the machines in check if and when they become smarter than us.

No pressure then...