Open Agentic AI And True Personalisation

Sixteen years on from my own initial posts on the subject of a personal assistant that can privately find and buy stuff for you on the web, and we have 'open agentic AI'. But are you really any closer to the automated selection and purchase of your own personalised products without needlessly surrendering your privacy or otherwise becoming the victim? Should this latest iteration of open generative AI be autonomously making and executing decisions on your behalf? 

What is Agentic AI?

An 'agentic' AI is an evolution of generative AI beyond a chatbot. It receives your data and relies on pattern matching to generate, select and execute one of a number of potential pre-programmed actions without human guidance, then 'learns' from the result (as NVIDIA, the leading AI chip maker, explains). 

A 'virtual assistant' that can find, buy and play music, for example, is a form of agentic AI (since it uses AI in its processes), but the ambition involves a wider range of tasks and more process automation and autonomy (if not end-to-end). 

You'll see a sleight-of-hand in the marketing language (like NVIDA's) as developers start projecting 'perception', 'understanding' and 'reasoning' on their agentic AIs, but computers don't actually do any of those human things. 

It's certainly a compelling idea to apply this to automating various highly complex, tedious consumer 'workflows' that have lots of different parameters - like buying a car, perhaps (or booking a bloody rail ticket in the UK!). 

Wearing my legal hat, I also see myriad interesting challenges (which I'd be delighted to discuss, of course!), some of which are mentioned here, but not all...

Some challenges

The main problem with using an 'agentic AI' in a consumer context is the involvement of a large language model and generative AI where there is a significant (e.g. economic, medical and/or legal) consequence for the user (as opposed to a chatbot or information-only scenario (though that can also be problematic). Currently, the household or device based virtual assistants are carrying out fairly mundane tasks, and you could probably get a refund if it bought you the wrong song, for example, if that really bothered you. Buying the wrong car would likely be a different matter.

There may also be confusion about the concept of 'agency' here. The word 'agentic' is used to mean that the AI has 'agency' in the sense it can operate without human guidance. That AI is not necessarily anyone's legal 'agent' (more below) and is trained on generic training data (subject to privacy, copyright consents/licensing), which these days is itself synthetic - generated by an AI. So, agentic AIs are not hosted exclusively by or on behalf of the specific consumer and do not specifically cater to a single end-customer's personalised needs in terms of the data it holds/processes and how it deals with suppliers. It does not 'know' you or 'understand' anyone, let alone you.  

Of course, that is consistent with how consumer markets work: products have generally been developed to suit the supplier's requirements in terms of profitability and so on, rather than any individual customer's needs. Having assembled what the supplier believes to be a profitable product by reference to an ideal customer profile in a given context, the supplier's systems and marketing/advertising arrangements seek out customers for the product who are 'scored' on the extent to which they fit that 'profile' and context. This also preserves 'information asymmetry' in favour of the supplier, who knows far more about its product and customers than you know about the supplier or the product. In an insurance context, for example, that will mean an ideal customer will pay a high premium but find it unnecessary, too hard or impossible to make a claim on the policy. For a loan, the lender will be looking for a higher risk customer who will end up paying more in additional interest and default fees than lower risk customers. But all this is only probabilistic, since human physiology may be 'normally distributed' but human behaviour is not.

So using an agentic AI in this context would not improve your position or relationship with your suppliers, particularly if the supplier is the owner/operator of the agentic AI. The fact that Open AI has offered its 'Operator' agentic AI to its pro-customers (who already pay a subscription of $200 a month!) begs the question whether Open AI really intends rocking this boat, or whether it's really a platform for suppliers like Facebook or Google search in the online advertising world. 

It's also an open question - and a matter for contract or regulation - as to whether the AI is anyone's legal 'agent' (which it could be if the AI were deployed by an actual agent or fiduciary of the customer, such as a consumer credit broker). 

Generative AI also has a set of inherent risks. Not only do they fail to 'understand' data, but to a greater or lesser degree they are also inaccurate, biased and randomly hallucinate rubbish (not to mention the enormous costs in energy/water, capital and computing; the opportunity cost of diverting such resources from other service/infrastructure requirements; and other the 'externalities' or socioeconomic consequences that are being ignored and not factored into soaring Big Tech stock prices - a bubble likely to burst soon). It may also not be possible to explain how the AI arrives at its conclusions (or, in the case of an agentic AI, why it selected a particular product, or executed a specific task, rather than another). Simply overlaying a right to human intervention by either customer or supplier would not guarantee a better outcome on theses issues (due to lack of explainability, in particular). A human should be able to explain why and how the AI's decision was reached and be able to re-take the decision. And, unfortunately, we are seeing less and less improvement in each of these inherent risk areas with each version of generative AIs.

All this means that agentic AI should not be used to fully automate decisions or choices that have any significant impact on an individual consumer (such as buying a car or obtaining a loan or a pension product).  

An Alternative... Your Own Personal Agent

What feels like a century ago, in 2009, I wondered whether the 'semantic web' would spell the end of price comparison websites. I was tired of seeing their expensive TV ads - paid for out of the intermediary's huge share of the gross price of the product. I thought: "If suppliers would only publish their product data in semantic format, a 'widget' on my own computer could scan their datafeeds and identify the product that's right for me, based on my personal profile and other parameters I specify". 

By 2013, I was calling that 'widget' an Open Data Spider and attempted to explain it further in an article for SCL on the wider tech themes of Midata, Open Data and Big Data tools (and elsewhere with the concept of 'monetising you'). I thought then - and still think now - that: 

"a combination of Midata, Open Data and Big Data tools seems likely to liberate us from the tyranny of the 'customer profile' and reputational 'scores', and allow us instead to establish direct connections with trusted products and suppliers based on much deeper knowledge of our own circumstances."

Personalised assistants are evolving to some degree, in the form of 'personal [online] data stores' (like MyDex or Solid); as well as 'digital wallets' or payment apps that sit on smartphones and other digital devices and can be used to store transaction data, tickets, boarding passes and other evidence of actual purchases. The former are being integrated in specific scenarios like recruitment and healthcare; while the latter tend to be usable only within checkout processes. None seems to be playing a more extensive role in pre-evaluating your personal requirements, then seeking, selecting and purchasing a suitable product for you from a range of potential suppliers (as opposed to a product that a supplier has created for its version of an 'ideal' customer that you seem to fit to some degree). 

Whether the providers of existing personal data stores and digital wallets will be prepared to extend their functionality to include more process automation for consumers may also depend on the willingness of suppliers to surrender some of their information advantage and adapt their systems (or AIs) to respond to and adapt products according to actual consumer requests/demand.

Equally, the digital 'gatekeepers' such as search providers and social media platforms will want to protect their own advertising revenue and other fees currently paid by suppliers who rely on them for targeting 'ideal' customers. Whether they can 'switch sides' to act for consumers and preserve/grow this revenue flow remains to be seen.

Overall, if I were a betting man, I'd wager that open agentic AI won't really change the fundamental relationship between suppliers, intermediaries and consumers, and that consumers will remain the targets (victims) for whatever suppliers and intermediaries dream up for them next...

I'd love to be corrected!

The Cat Is Out Of The Bag: The EU Bars UK Financial Outsourcing

A key EU financial authority has asked EU regulators to be strict on UK firms seeking to escape the impact of Brexit. The concern is that having lost their EU passporting rights, desperate Brits will try to get authorised in Europe but continue to rely on UK managers and operations. 

"UK-based market participants may seek to relocate entities, activities or functions to the EU27 in order to maintain access to EU financial markets. In this context, these market participants may seek to minimise the transfer of the effective performance of those activities or functions in the EU27, i.e. by relying on the outsourcing or delegation of certain activities or functions to UK-based entities, including affiliates. It is therefore necessary to ensure that the conditions for authorisation as well as for outsourcing and delegation do not generate supervisory arbitrage risks."

ESMA even proposes a Cat o' nine tails set of 9 "principles" to prevent UK firms making the best of Brexit: 

1. No automatic recognition of existing financial firm authorisations;
2. Authorisation processes by the EU27 should be "rigorous and efficient";
3. Regulators must verify the objective reasons for relocation;
4. Regulators should avoid "letterbox" entities in the EU27 - the EU firm must perform substantial activities;
5. Outsourcing and delegation to third countries (like the UK) is only possible under strict conditions;
6. Substantive decision-making must occur in the EU, especially over outsourced activities;
7. There must be sound local governance of EU entities, by resident directors/senior managers;
8. Regulators must have the resources and data to effectively supervise and enforce EU law. 
9. ESMA is watching and will co-ordinate to ensure adequate and consistent supervision. 

Of course, the UK could retaliate with red tape of its own. Brexit is also a challenge for 8,008 EEA firms that hold 23,532 passports (about 3 each) to cover their UK offerings.

Thoughts On The Potential For P2P Insurance

Some interesting discussions these past few weeks about the potential for innovation and 'disruption' in the insurance markets. As ever, there are stark differences between areas that industry players see as ripe for innovation/disruption and the opportunities outsiders see...

A signficant source of this disconnect - and a great source of opportunity for outsiders - is the tendency for established institutions to view the market through the narrow lens of their own existing products and activities, rather than from the customer's standpoint. To really solve a customer's problem, a supplier has to understand the end-to-end activity in which that customer is engaged; and has to consider that it might need to collaborate with other suppliers in the process.

For instance, as a consumer of car insurance, it's important to understand that you don't simply drive you car. You drive it from A to B in the course of some other activity. Is it a one-off journey, or a commute? Does it involve both city streets, motorways and/or rural roads? What time of day is it? Are the road conditions always the same, often wet or sometimes extreme? Why couldn't I switch insurers, policies and/or premiums as these variables change? Could my car be covered by household insurance while parked at home? The answer hardly requires advanced telematics.

Another problem for insurers is their preoccupation with managing short term financial performance within regulatory capital requirements. This favours cost-reduction at the expense of more strategic, long term business development. In fact some insurers may be better off admitting they are simply running-off their existing book. [Update on 26 March: FT coverage of RSA's rights issue underlines this point - it's all about cost-cutting and disposals, to which CEOs have tied some nice incentives].

At any rate, this tells me that insurers will end up reacting to changing demand, rather than reinventing insurance in any substantial way.

The same goes for the insurance industry's attitude to Big Data. While large insurers are quite sophisticated exponents of Big Data, the industry is merely dedicating itself to persuading customers to disclose more and more personal data about themselves for use in marketing extra products, reducing fraud or improving claims-handling.

This ignores the evolution of personal information management services that go in search of products that are right for you personally. Insurers argue that's what happens on price comparison sites already, and the Cheap Energy Club takes that a step further. But we have not yet seen the truly personal 'open data spider' that some of us have been dreaming about. In that machine-readable future, the challenge for insurers won't be to find customers, but to be able to instantly formulate policies in response to customer devices directly peppering their systems with requests for tailored cover.

To be fair, there are also plenty of mistaken assumptions by outsiders about how insurance actually works (or doesn't) today, and which elements of the value/supply chain that are ripe for improvement or disintermediation. For instance, people forget the key role of reinsurers and reinsurance brokers in diffusing the risk of loss across many sources of capital.

So before disrupting today's insurance markets, it's worth pausing briefly to understand the nature of insurance and how the markets operate.

In layman's terms, insurance is a way for you (the 'insured') to transfer to someone else (an 'insurer') the risk of loss, in return for payment (a 'premium'). 

But it's not quite that simple. In legal terms, that 'risk of loss' translates into 'a defined event, the occurrence of which is uncertain and adverse to the interests of the recipient'. The practice of pooling risks also lies at the heart of modern insurance, such that premiums paid for insuring lower risks are used to fund payouts on higher risks. This of course presents a significant moral hazard, and the scandals involving payment protection insurance and so-called 'identity theft' insurance illustrate how the industry has tended to seek out customers who don't actually face a genuine risk that is adverse to their interests and/or would never be able to make a claim (even if they were aware they'd bought the insurance).

Which brings us to the main problem with insurance markets today - they are highly complex and heavily intermediated, often by players who have little or no interest in seeing a genuine risk is insured appropriately.

Modern insurance can be traced to the need to insure property against the risk of fire after the Great Fire of London (and some might say little has changed since then in the way non-retail insurance business is transacted!). The need to spread the exposure to other risks of loss has created markets around certain types of other events, businesses and property. Reinsurance markets have developed to enable insurers to insure themselves against the risks they underwrite. In each of these markets, the distribution, marketing and sale of insurance is heavily intermediated by brokers and others who take their own cut from the gross premium that you pay (the net premium being what the insurer receives in return for underwriting the risk). Insurers also must invest their premiums in order to help fund payouts and ensure they have enough capital to cover their exposures. So there are strong links between global markets for insurance and other financial products, which brings with it hidden costs and fees, the risk of re-concentrating risk in suprising places and exposure to global financial crises...

Rolling all of these issues together, it seems to me that the real purpose of an insurance business is to find people who genuinely face adverse consequences from specific events, the occurrence of which are uncertain, and then to diffuse that risk across as many different sources of capital as possible, as efficiently as possible. 

Some would say that this amounts to concentrating the risk of loss, since those who don't genuinely need insurance would be excluded (but allowed to buy it if they genuinely do just want it for 'peace of mind').  But that only means we should cease pooling risk and find another way to spread it, such as the peer-to-peer marketplace model that is at work in many other industries.

Peer-to-peer insurance would involve the operator of an electronic platform enabling direct insurance contracts between each insured and many investors (whether traditional insurers or not), each of whom would receive a small portion of the overall premium yet only have to pay out small sums in the event of loss. In this way, the risk of loss could be diffused amongst many investors who would only provide insurance as part of a widely diversified portfolio.  In common with the impact of the P2P model in other industries, removing all the middlemen would cut the margin between net and gross premium to a transparent fee for running the platform, leaving the lion's share of the difference with the market participants.

There are some interesting examples that are headed in this direction. Friendsurance, for example, goes part of the way by enabling a crowd of people to fund the excess on each of their insurance policies. I'm also aware of jFloat (yet to launch), which some have suggested is an application of the P2P model. But I understand that it will still involve pooling risk on a kind of mutual basis, whereas I'm talking more about a 'pure' P2P model.

Presumably, this is not what today's insurers, brokers, reinsurers, reinsurance brokers and other established industry participants want to hear. But they too could benefit in the longer term (if they can afford to think that far ahead) by setting up their own platforms or contributing their own capital and expertise.

It's okay, everyone, I'm not holding my breath...

CDSs on Financial Crime?

Insurers emboldened by the bail-out of AIG and a rock-solid culture of greed and stupidity, Schumpeter warns us, are boldly venturing into the directors and officers insurance marketplace with a great little product that will cover “damages established by FDIC due to non-intentional wrongful acts or omissions by the executive or director.”

I love these guys.

Love them!

Talk about the "The Big Short". These guys are short of just about everything.

I mean, how is this a good idea for anyone but criminals and the criminally insane?

WHERE ARE THE REGULATORS???!!!

Well, Schumpeter tells us, "The FDIC, and three financial-industry lobby groups we also approached, declined to comment."

F*cking asleep.

I despair. Really, I do.

Might even rush out and buy a KickOut Bond.

Or Greek debt.

Finish up on a park bench under old PPI policies outside the FSA.

ID Theft Insurance: Bonfire Of The Gullible

I guess it started out as a reasonable idea - provide card customers with comfort that someone else will help if your credit card is stolen and your card issuer let's you down.

But your card issuer isn't allowed to let you down, unless you do that yourself through fraud or negligence - in which case why would an insurer help? Slow as they are, it's very much in the legal and financial interests of card issuers to invest in anti-fraud protection. And they're usually 'first-on-the-scene' in a card fraud scenario, as you'll be aware if you've ever been called to confirm you weren't standing at a point of sale in both Brazil and Bulgaria in the past few hours.

If you're an ID theft policyholder and you don't agree, are you are among the 0.5% who've actually made a claim?

Cue the FSA probe into CPP, purveyor of fine ID theft protection at £80 a year. The investigation was prompted by Which? who have campaigned against this rort for some time. A glance at the fund managers who backed the IPO tells you all you need to know. Those people have no right to complain. Either they knew the drill or they were gullible enough to think this product had a real future.

As for the attitude of the card issuers: the FT cites at least one analyst's view that:

"You could argue that half the companies on the high street shouldn't exist because the things they sell are vulgar, tasteless or tacky... Which may be true, but it's irrelevant. The point is, there's demand there."

Demand or gullibility?

Will The Semantic Web Kill Price Comparison Sites... Please?

Maybe I'm confused, but every time I see a TV ad for one of the multitude of financial services price comparison web sites, I wonder what proportion of the gross product price or insurance premium is commission to cover all those advertising costs.

Yet it's claimed that I'd pay the same price if I went directly to the product provider...

So then I wonder: if there has to be enough fat in the price to cover multiple distributors' TV ad campaigns, why don't product providers start semantic publishing? That way, a widget on my own computer could scan their datafeeds and identify the product that's right for me, based on my personal profile and other parameters I specify?

Surely this would bring down the cost of products and I would also see the whole market, not just those who are prepared to pay to be on the price comparison sites?