Open Agentic AI And True Personalisation

Sixteen years on from my own initial posts on the subject of a personal assistant that can privately find and buy stuff for you on the web, and we have 'open agentic AI'. But are you really any closer to the automated selection and purchase of your own personalised products without needlessly surrendering your privacy or otherwise becoming the victim? Should this latest iteration of open generative AI be autonomously making and executing decisions on your behalf? 

What is Agentic AI?

An 'agentic' AI is an evolution of generative AI beyond a chatbot. It receives your data and relies on pattern matching to generate, select and execute one of a number of potential pre-programmed actions without human guidance, then 'learns' from the result (as NVIDIA, the leading AI chip maker, explains). 

A 'virtual assistant' that can find, buy and play music, for example, is a form of agentic AI (since it uses AI in its processes), but the ambition involves a wider range of tasks and more process automation and autonomy (if not end-to-end). 

You'll see a sleight-of-hand in the marketing language (like NVIDA's) as developers start projecting 'perception', 'understanding' and 'reasoning' on their agentic AIs, but computers don't actually do any of those human things. 

It's certainly a compelling idea to apply this to automating various highly complex, tedious consumer 'workflows' that have lots of different parameters - like buying a car, perhaps (or booking a bloody rail ticket in the UK!). 

Wearing my legal hat, I also see myriad interesting challenges (which I'd be delighted to discuss, of course!), some of which are mentioned here, but not all...

Some challenges

The main problem with using an 'agentic AI' in a consumer context is the involvement of a large language model and generative AI where there is a significant (e.g. economic, medical and/or legal) consequence for the user (as opposed to a chatbot or information-only scenario (though that can also be problematic). Currently, the household or device based virtual assistants are carrying out fairly mundane tasks, and you could probably get a refund if it bought you the wrong song, for example, if that really bothered you. Buying the wrong car would likely be a different matter.

There may also be confusion about the concept of 'agency' here. The word 'agentic' is used to mean that the AI has 'agency' in the sense it can operate without human guidance. That AI is not necessarily anyone's legal 'agent' (more below) and is trained on generic training data (subject to privacy, copyright consents/licensing), which these days is itself synthetic - generated by an AI. So, agentic AIs are not hosted exclusively by or on behalf of the specific consumer and do not specifically cater to a single end-customer's personalised needs in terms of the data it holds/processes and how it deals with suppliers. It does not 'know' you or 'understand' anyone, let alone you.  

Of course, that is consistent with how consumer markets work: products have generally been developed to suit the supplier's requirements in terms of profitability and so on, rather than any individual customer's needs. Having assembled what the supplier believes to be a profitable product by reference to an ideal customer profile in a given context, the supplier's systems and marketing/advertising arrangements seek out customers for the product who are 'scored' on the extent to which they fit that 'profile' and context. This also preserves 'information asymmetry' in favour of the supplier, who knows far more about its product and customers than you know about the supplier or the product. In an insurance context, for example, that will mean an ideal customer will pay a high premium but find it unnecessary, too hard or impossible to make a claim on the policy. For a loan, the lender will be looking for a higher risk customer who will end up paying more in additional interest and default fees than lower risk customers. But all this is only probabilistic, since human physiology may be 'normally distributed' but human behaviour is not.

So using an agentic AI in this context would not improve your position or relationship with your suppliers, particularly if the supplier is the owner/operator of the agentic AI. The fact that Open AI has offered its 'Operator' agentic AI to its pro-customers (who already pay a subscription of $200 a month!) begs the question whether Open AI really intends rocking this boat, or whether it's really a platform for suppliers like Facebook or Google search in the online advertising world. 

It's also an open question - and a matter for contract or regulation - as to whether the AI is anyone's legal 'agent' (which it could be if the AI were deployed by an actual agent or fiduciary of the customer, such as a consumer credit broker). 

Generative AI also has a set of inherent risks. Not only do they fail to 'understand' data, but to a greater or lesser degree they are also inaccurate, biased and randomly hallucinate rubbish (not to mention the enormous costs in energy/water, capital and computing; the opportunity cost of diverting such resources from other service/infrastructure requirements; and other the 'externalities' or socioeconomic consequences that are being ignored and not factored into soaring Big Tech stock prices - a bubble likely to burst soon). It may also not be possible to explain how the AI arrives at its conclusions (or, in the case of an agentic AI, why it selected a particular product, or executed a specific task, rather than another). Simply overlaying a right to human intervention by either customer or supplier would not guarantee a better outcome on theses issues (due to lack of explainability, in particular). A human should be able to explain why and how the AI's decision was reached and be able to re-take the decision. And, unfortunately, we are seeing less and less improvement in each of these inherent risk areas with each version of generative AIs.

All this means that agentic AI should not be used to fully automate decisions or choices that have any significant impact on an individual consumer (such as buying a car or obtaining a loan or a pension product).  

An Alternative... Your Own Personal Agent

What feels like a century ago, in 2009, I wondered whether the 'semantic web' would spell the end of price comparison websites. I was tired of seeing their expensive TV ads - paid for out of the intermediary's huge share of the gross price of the product. I thought: "If suppliers would only publish their product data in semantic format, a 'widget' on my own computer could scan their datafeeds and identify the product that's right for me, based on my personal profile and other parameters I specify". 

By 2013, I was calling that 'widget' an Open Data Spider and attempted to explain it further in an article for SCL on the wider tech themes of Midata, Open Data and Big Data tools (and elsewhere with the concept of 'monetising you'). I thought then - and still think now - that: 

"a combination of Midata, Open Data and Big Data tools seems likely to liberate us from the tyranny of the 'customer profile' and reputational 'scores', and allow us instead to establish direct connections with trusted products and suppliers based on much deeper knowledge of our own circumstances."

Personalised assistants are evolving to some degree, in the form of 'personal [online] data stores' (like MyDex or Solid); as well as 'digital wallets' or payment apps that sit on smartphones and other digital devices and can be used to store transaction data, tickets, boarding passes and other evidence of actual purchases. The former are being integrated in specific scenarios like recruitment and healthcare; while the latter tend to be usable only within checkout processes. None seems to be playing a more extensive role in pre-evaluating your personal requirements, then seeking, selecting and purchasing a suitable product for you from a range of potential suppliers (as opposed to a product that a supplier has created for its version of an 'ideal' customer that you seem to fit to some degree). 

Whether the providers of existing personal data stores and digital wallets will be prepared to extend their functionality to include more process automation for consumers may also depend on the willingness of suppliers to surrender some of their information advantage and adapt their systems (or AIs) to respond to and adapt products according to actual consumer requests/demand.

Equally, the digital 'gatekeepers' such as search providers and social media platforms will want to protect their own advertising revenue and other fees currently paid by suppliers who rely on them for targeting 'ideal' customers. Whether they can 'switch sides' to act for consumers and preserve/grow this revenue flow remains to be seen.

Overall, if I were a betting man, I'd wager that open agentic AI won't really change the fundamental relationship between suppliers, intermediaries and consumers, and that consumers will remain the targets (victims) for whatever suppliers and intermediaries dream up for them next...

I'd love to be corrected!

Before You Invest, Please Realise That 'Web 3' Is Not The Same As 'Web 3.0'

Tech hypsters are colliding in cyperspace over rival claims to being the latest version of 'the Internet'. And the media aren't helping by failing to spot critical distinctions, as in CNN's recent mistake in interviewing a Web 3/'blockchain expert' about the Web 3.0 data privacy business started by Tim Berners-Lee (who has already warned us to ignore Web 3).

Perhaps the confusion stems from that fact that 'decentralisation' is a common theme for both: 

• the "Web 3" world of distributed ledger technology, blockchains, crypto-tokens, cryptocurrencies, cryptoassets and non-fungible tokens (NFTs), in which the community of more zealous participants are referred to as "Crypto"; and

• the evolution of the World Wide Web from Web 1.0 to Web 2.0 to "Web 3.0" under the W3C banner - the space on the Internet (a network of networks, defined by the TPC/IP standards) in which the items of interest ("resources"), are identified by global Uniform Resource Identifiers (URI), the first three specifications of which were URLs, HTTP, and HTML.

While the Ethereum blockchain co-founder Gavin Wood suggests he "coined" the term Web 3.0 in 2014, it was certainly in use before then, as even my own post from January 2013 demonstrates - and I definitely didn't coin it. In fact, the Wikipedia entry for 'semantic web' cites a New York Times quote from 2006 that suggests 'Web 3.0' began being used in connection with linking data resources somewhat earlier than that:

People keep asking what Web 3.0 is. I think maybe when you've got an overlay of scalable vector graphics – everything rippling and folding and looking misty – on Web 2.0 and access to a semantic Web integrated across a huge space of data, you'll have access to an unbelievable data resource … 

— Tim Berners-Lee, 2006

The goal of the Web 3.0 community is to enable you to take control over 'your own data' (all data generated by your activity, not just personal data), enabling you to permit who can access and use it from wherever it is stored, by means of 'application programming interfaces' (APIs). This is encompassed in the concept of "linked data" and "linked datasets", often also referred to as "the semantic web". Midata is a related regulatory trend toward enabling consumers to control their own data, of which Open Banking and Open Finance generally are examples.

On the other hand, the goal of the Web 3/Crypto community is a broader, libertarian utopia that (according to Ethereum) guarantees 'personal sovereignty' in a 'permissionless', 'trustless' cryptographic environment with its own 'native' means of peer-to-peer payment. 

Unfortunately, in my view, the absence of trust in this would-be Crypto idyll is merely an externality that others are yet to satisfactorily address, as demonstrated by the collapse of many crypto exchanges and other 'decentralised finance' (DeFi) participants, the most recent being FTX group... 

All the more reason not to confuse 'crypto' Web 3 with 'semantic' Web 3.0!

Dirty Data

Westiminster recently feigned shock and horror that the UK's coppers cook the crime figures. But Simon Jenkins says we've known for years that the numbers are meaningless and they should be banned as "they spread confusion and fear".

But 'plod' is not alone in mis-classifying, mis-recording, ignoring or otherwise presenting data in a way that suits himself. We've had many financial trading scandals where banks apparently had no idea of the exposures they faced, either because transactions were concealed or perhaps no one was looking hard enough - the global financial crisis was a function of poor due diligence.

A possible root cause of the problem is that humans are involved too early in the data collection and reporting processes. Rarely are we responding to the 'raw' data, as opposed to figures that have been 'gathered' and 'rolled up' through a series of other people's filters, manipulations and interpretations (which are often taken out of context). It's puzzling why regulators' systems don't receive a feed of the actual trades straight from bank trading desks - or from peer-to-peer lending or crowdfunding platforms - rather than relying on periodic reporting of summary data.

Maybe GCHQ can help...

At any rate, we should focus more on 'clean' mechanisms for capturing and presenting raw data rather than someone else's interpretation of it.

Image from TraceyNolte.
 

Six Years On And Pragmatism Has A New Frontier

I see this blog has reached the ripe old age of six, so I felt compelled to squeeze in at least one post to celebrate.  

It's fitting that the reason for my absence has been the need to get to grips with the FCA's proposals to regulate P2P lending and investment-based crowdfunding - not to mention the revelations concerning the Chairman of the Co-op Bank. After all, this blog set out to chart the rise of facilitators who help us wrest personal control of our day-to-day lives from the one-size-fits-all experience imposed on us by our institutions. Rumbling the 'Crystal Methodist' marks the continuing plunge of faith in those same institutions, while the decision to finally let the 'crowd' into the regulated financial markets shows that even Parliament recognises you and I are better off dealing with each other directly than simply entrusting our life's savings to the banks and investment funds.

Of course, these are just a few examples of the punishment being doled out to our financial institutions. And they aren't the only ones under pressure from the trends sweeping society, as we struggle to figure out a more sustainable form of capitalism. All our institutions, from the BBC to the Police to the Church, unions, political parties, government departments and so on, face the choice of becoming facilitators or withering away. 

So is there anything 'new' to write about? 

Six years on we are still seeing the dawn of where these trends will take us. But to get a sense of the future, I've been following the rise of 'open data' - or open access to data in machine-readable form. This marks a new frontline between institutions and facilitators. Big Data vs You. Not only has it already created new facilitators, in the form of "personal data stores" or "personal information managers", but it may also redefine some of today's facilitators as the institutions of tomorrow... 

As a taste of things to come, last week a senior advertising executive insisted to me that "Big Data can accurately predict human behaviour." To be fair I made him repeat the assertion in case it had slipped out by accident. No one else at the table seemed to find that truly weird, and it wasn't until the end of the week, when I met up with some people working at the sharp end of data gathering, that I was able to fully enjoy the hilarity of that statement.

This is going to be fun.

Image from Data.gov.uk

Open Data Spiders?

Since 2009 I've hoped that the semantic web - that is my computer dealing with suppliers' computers - would replace the need for price comparison sites. Following a discussion last night at the CtrlShift Explorers' Club, I'm confident that we don't have much longer to wait.

If suppliers publish their product data in computer-readable format, I could then programme an application or 'spider' to search the provders' open systems to find the product that's right for me - ideally a bespoke product assembled from a menu of optional components. This spider would use my personal data to conduct its search without disclosing that data to any product providers, at least until the time of purchase (and disclosure might not even be necessary then). It could also collect, say, public sector Open Data related to my desired activity, and analyse it in the context of my relevant personal transaction history. This could vastly improve my choice of car, holiday or home improvement and how it's financed. Or it could save me money by keeping me on the right energy or mobile phone tariff.

This is not about 'intent-casting' or 'demand-casting' in order to encourage suppliers to send me thousands of offers. My spider would not announce to the world that it's looking for anything. It would simply run around the web looking at openly available product codes and report its findings to me. Ideally, the product provider will have no idea that it's actually me who's looking until I make a purchase, if ever.

And I would not need to read any screens or physically enter any data until my spider reported its findings - or it could save me the trouble by calling my mobile.

In a machine-to-machine world, the marketing challenge is to ensure that anyone's 'spider' can always find your product data, and that data is accurate and up to date. Perhaps it could be somehow 'spider optimised', but it seems to me it's the job of the spider developers to make sure the spiders are good at finding product data, even when it's in a sorry state.

My sense is that an Open Data approach to the market takes such a different corporate mindset that it is unlikely to sit comfortably within traditional suppliers, where "Big Data" is the latest buzzphrase. In the Open Data world the challenge is to enable your products to be directly embedded in the ecosystem, helping to solve problems as customers encounter them and their machines or 'spiders' look for an answer. The traditional product approach is not 'connected' in that way, or at all. And, as I suggested recently, "Big Data" approach to behavioural targeting of advertisting seems fundamentally hamstrung by the fact that personal behavioural data is highly contextual and not really 'predictive' from one scenario to the next. Why spend all that money on what is ultimately a shot in the dark?

Those who ignore the Open Data option could well be spending their way rapidly into oblivion. 


Image from Data.gov.uk

When Will Control Truly Shift To The Consumer?

For those engaged in the process of empowering consumers, 2012 is already a fascinating year. So it was timely that a bunch of us met at Ctrl Shift's "Explorer's Club" to try to map the timeline for when 'customer relationship management' truly inverts and firms finally acknowledge their customers control them. 

The output of the session is being converted into an 'infographic' that will be available as a reference soon. In the meantime, here's an excellent drawing that Joel Cooper produced during the session to reflect the various themes:

Augmented Reality and Linked Data

I enjoyed a far-ranging conversation with @StGilesResident today, and am indebted to him for the reference to the following TED talk by Pranav Mistry. I do not know how I've missed this to date and am looking forward to seeing how this technology evolves with 'midata' initiatives to support our day-to-day activities in future. The folks at Microsoft have also been playing around here (you can follow them here).

Mydata And Consumer Empowerment

On Thursday, work began in earnest on the 'mydata' project featured in the government's "Consumer Empowerment" strategy (see Better Choices: Better Deals, on which I posted earlier).

As with the overall consumer empowerment policy, the primary goal of the ‘mydata’ project is “to put consumers in charge so that they are better able to get the best deals for themselves, individually and collectively.” Achieving that involves enabling consumers to access information about their purchases, analyse it according to their own preferences and use that information to make better purchasing decisions.

As a long-time critic of the European Commission approach to facilitating e-commerce, I'm overjoyed the government is convinced that new legislation is not the best way to achieve consumer empowerment. Instead, it's relying on "a wide range of new programmes that have been developed in partnership with businesses, consumer groups and regulators" against a background of normal regulatory enforcement.

The mydata development work is being fostered through a series of 'boards', chaired by Professor Nigel Shadbolt. There's a Strategy board made up of private sector retail businesses, consumer bodies and government representatives; four Sector boards comprising representatives of suppliers in the energy, financial services, telecoms and retail sectors; and an Interoperability board of private and public sector representatives. The focus of the Interoperability board is on maximising synergies among the sector groups' work; addressing barriers common to all groups; maintaining a balance amongst key issues of practical utility, privacy, security and data portability; and offering suggestions on where ideas and solutions in one sector might link up (or mashup) with others to better reflect consumers' day-to-day activities.

I know what you're thinking, but these boards are not designed to be exclusive talking shops or symbolic meetings of 'the great and the good'. The idea is to be as pragmatic as possible, using the boards to draw in as many interested parties, ideas and resources as possible to achieve rapid progress. It's a fascinating challenge, and I'm thrilled to be helping out on the interoperability front.

There'll be plenty of project communications, of course, which I'll be doing my best to retweet etc, and send to people I know who may be interested or able to help. I also plan to share material that I happen across outside the project. For starters, I've included below the links to Sir Tim Berners-Lee's TED talk on the semantic web (or Linked Data) and a few of the items he mentions. I'm very interested to receive any comments, referrals, ideas etc you may have.

Sir Tim cites some key examples of Linked Data and its uses. DBpedia is the fascinating "community effort to extract structured information from Wikipedia and to make this information available on the Web." And here is the TED talk by Hans Rosling to which he refers:

Image from 1Million1Shot.

A Plan for Small Business Growth?

Hat-tip to @JamesGreenThing for the link to the last Business Insider list of up-and-coming entrepreneurs.

These lists are always much more interesting 6 months on. In this case, the bubble in online coupons is more obvious, as is the fact that the fashion market is getting the Web 2.0 treatment. Yet BankSimple still hasn't launched.

Online invoice discounting has been on my radar since early 2008. So it's heartening that The Receivables Exchange is highly rated in the US, and reassuring that MarketInvoice is gaining traction in the UK. Doubtless there'll be more on that front soon.

Partly for that reason, SecondMarket really caught my eye. It's a market for alternative investments, so incorporates a channel for equity in private companies (e.g. CrowdCube in the UK). But the emphasis on gathering "robust market data" signals a growth in the availability of richer information about private companies. Focus on this is timely, given the massive official paywall around corporate data. Recent commitments to 'Open Government' (follow the Open Knowledge Foundation) are promising. There's already a Government commitment to arm consumers with their own data. Perhaps enabling SMEs to use their own data for their own benefit could be added to The Plan for Growth?

Image from AppAssure.

Conceptual Search: Putting "Mean" Into Meaning

This evening I caught the SCL's Annual Lecture. This year it was expertly delivered by Dr Mike Lynch OBE on the "Advent of Meaning Based Computing".

I want to say that "meaning based computing" is the enterprise version of the semantic web, if only to keep 'price comparison' web sites in the technological cross-hairs. But I can't really, as it wasn't even mentioned.

According to Mike, the more fundamental Information Technology challenge is that the world is producing far more human-friendly, unstructured data than structure data (compound annual growth rate of 62% vs 21%). Cloud computing might help scale the technology aspect, but that doesn't help you find the right information. Keyword search, PageRank, meta data and so on all help with the data. But if computers are to search more comprehensively, they must 'understand' the meaning, concepts or ideas you're looking for, and how these relate to one another.

Conceptual search is partly constrained by who is searching, and whether they know what they're looking for. Boolean and linguistic search models in particular require 'training' and lots of maintenance by very smart people who know a lot about what's being searched for. But Mike explained that probabilistic systems are independent of language and investigator bias. Instead, they process the whole data set and look for scenarios where words are 'more likely than chance' to appear together. In this way, a more objective set of search results are returned.

So what?

Well, Mike says the main applications for conceptual search currently seem to be amongst spooks, regulators and major corporations. In other words, it's a mean weapon in major games of cat-and-mouse - one that US financial markets investigators seem to have spent five years perfecting. In fact, the most worrying statement of the evening for some must be that "most of Wall Street's messaging is in the cloud".

Specific uses lie in improving document management, retention and electronic discovery, particularly as an aid to early case assessment and crisis management. A killer app, no doubt, and one where Mike reckons law firms can indeed make a killing - possibly providing outsourced data systems for their clients. But pre-programmed, conceptual search systems are also good enough to enforce corporate policies real-time, preventing 'smoking gun' emails from ever being sent, or corporate bribes being accepted.

If only they could help find decent car insurance, utility and mortgage deals, we'd be shot of price comparison sites as well ;-)

Big Media: Do You Really Facilitate What People Want To Do?

It's illuminating to compare Alan Rusbridger's recent presentation of the Guardian's collaborative business model, with Facebook Open Graph.

Both the Guardian and Facebook initiatives seek to capitalise on people's desire to socialise (to summarise many subsidiary activities) by offering the opportunity to socialise on their respective advertising platforms. But we are still seeing a stark difference between an institution seeking to solve its own problem, top-down, and that of a facilitator focused on solving people's problems, bottom-up.

The links in the content on the Guardian's site are almost entirely to Guardian content, rather than to source material or the digital presences of people/organisations named. So, while the Guardian is moving towards a more collaborative model, it patently wants to 'own' that collaboration - promoting Guardian content, centring on the Guardian's favoured topics and on the Guardian's advertising platform. So if you wish to socialise on the Guardian platform you must surrender your 'voice' and identity to the Guardian.

By contrast, Facebook Open Graph facilitates the various activities that comprise 'socialising' without dictating location, content and whether it involves creating links away from Facebook, or even to Facebook.

It seems obvious to me which approach will ultimately attract more people and advertisers. The question is whether the Guardian has the institutional skill and courage required to reinvent itself as a genuine facilitator before everyone moves to a more facilitative platform.

Image from Logic+Emotion.

Improve Financial Capability - Simplify Products

The FSA says, "Financial Capability is about being able to manage money; keeping track of your finances; planning ahead; choosing financial products; and staying informed about financial matters."

The implication is that financial services don't have to change - you do. And the FSA provides a dazzling array of data to bamboozle you along the change curve.

This approach is doomed. Adult learning research has emphasised that the older you are, the less likely you are to learn. Of the 20-24 age group, 61% say they are learning now or have been recently. For the 55-64s, that statistic is 31%, and for the 65-74s, it is 18%. We have an ageing population and a ballooning pensions deficit.

So financial services must change, not you. Products must become simpler and cheaper, and it must be really easy for investors to develop fully diversified portfolios that produce sustainable returns.

Why can't I put suitable financial services in a shopping cart, like I can buy other stuff?

To make financial services simpler and more consumable for more people, providers and intermediaries must do much more to make it easy to find, compare, choose and buy products that contribute towards sustainable returns for the investor rather than scandalous profits for the provider. I'm not talking about the price comparison sites that simply list the same old stuff, by product type, by price. I'm talking about far more automated services that make the detail available to those who want it, but simply deliver diversification without the average person needing to understand more than the concept of not putting "all your eggs in one basket".

To support this, the clear objective of the financial regulatory regime should also be to deliver simple, low cost financial products that are accessible to us all. Currently, regulation funnels investment opportunities and funds into a zone in which relatively few firms are permitted to operate, enabling them to charge excessive fees and related compensation. In other words, regulation designed to protect the consumer is actually underwriting "fat banking". But what we need is a regime that fosters the growth of low cost 'facilitators' such as those who've allowed us to unbundle flights and hotels, music tracks and other one-size-fits-all products to create our own personalised, lower cost alternatives.

Do Price Comparison Sites Increase Premiums?

Hypothesis: the rising cost of car insurance is actually driven by marketing costs.

Gross premiums have increased 14% in the past year, and the car insurance industry would have us believe this is driven by the 'rising cost of personal injury claims and fraud'. But that would suggest net premiums (the proportion of the gross premium that actually covers the insured event) are being priced wrongly, which I find difficult to believe. Actuaries must be fairly good at predicting accident rates, deaths and injuries and so on by now. And these actually appear to be in decline, according to the Office of National Statistics research published in April 2009:

"The total number of deaths in road accidents fell by 7 per cent to 2,946 in 2007 from 3,172 in 2006. However, the number of fatalities has remained fairly constant over the last ten years...

The total number of road casualties of all severities fell by 4 per cent between 2006 and 2007 to approximately 248,000 in Great Britain. This compares with an annual average of approximately 320,000 for the years 1994-98.

The decline in the casualty rate, which takes into account the volume of traffic on the roads, has been much steeper. In 1967 there were 199 casualties per 100 million vehicle kilometres. By 2007 this had declined to 48 per 100 million vehicle kilometres."

So I wonder if there's a variable cost in there that's proving difficult to control. A chief culprit might be marketing. We're certainly being inundated with TV advertisements for insurance price comparison sites, so I wonder if that is in turn being paid for by higher 'costs per click' associated with advertising on those sites? While some insurers are refusing to list their products on the price comparison sites, they may still face competition for key search terms, for example.

Needless to say, I haven't yet been able to find much public analysis on this, but there is a helpful post on analysing the efficiency of cost per click by The Catalsyst. It's easy to over-spend, and the competition may force you to.

If my hypothesis is right, the case is building for semantic web applications to enable people's computers to find deals simply by interrogating insurers' computers - without all the expensive advertising noise.

Conspicuous Thrift - Your 5 New Consumer Tactics

Hat-tip to none other than EU Commissioner Meglena Kuneva, whose recent post alerted me to the report of the McKinsey Global Institute on our highly effective responses to the credit crunch, at least as consumers.

McKinsey reports that a third of the decline in Eurozone GDP in Q4 2008 was the result of reduced consumer spending. That's particularly signicant because consumer spending was the single largest factor fueling GDP growth during 2002-07.

They say our spending is driven by confidence, incomes, wealth, the availability of credit and cost of living. All these currently point downward, though the cost of living is obviously less of a problem as prices fall in line with declining demand.

The McKinsey report suggests our savings are a function the type of items we target for savings, the type of consumers we are (discussed below) and the tactic we choose (replace items only when needed, control spending, do-it-yourself, seek value and shop smarter).

McKinsey also say that a "cheap is chic refrain" has inspired the 'do it yourself' tactic. This is more confirmation that the Age of Conspicuous Thrift has dawned. This may also be shorthand for moral or ethical choices - 'green' sentiment and the 'counter-Veblen' effect ("preferences for goods increasing as their price falls, over and above the traditional supply and demand effect, due to conspicuous thrift amongst some consumers"). Do The Green Thing, for example, lists seven tactics for leading a greener life:

"1. You get from A to B without any C when you Walk The Walk
2. It’s delicious but it causes more CO2 than cars so go Easy On The Meat
3. Resist the urge to buy the latest and Stick With What You Got
4. Turn down the central heating and turn up the Human Heat
5. The art of wasting nothing and using up everything: All-Consuming
6. Instead of jetting your way around the world, Stay Grounded
7. Don’t leave it on or even put it on, Plug Out"

What type of items are we targeting for the most savings?

McKinsey say that of all spending categories, eating out has taken the biggest hit. Interestingly, however, electronic gadgets rank 13th on average - we'd rather an iPhone than a fancy meal. But if incomes drop by 20% gadgets rise to the 6th most likely to be cut. The least likely area to target for savings is insurance - perhaps reflecting higher anxiety about what the future holds.

What type of consumer are you? What are you tactics?

The report finds there are 4 consumer types: "party's over", "domestic downsizers", "food scrooges" and "basic bargainers". Of all categories, most people are "party's over" types and these have the largest impact on all spending (especially eating out, clothes, booze and fags). The other consumer types cut back more selectively using specific tactics. For example, "domestic downsizers" target equipment (cars, electronics and home furnishings) by 'replacing only when needed' and make 45% of their total savings on holidays, by simply controlling their spending. Electronic goods tend to fall prey to people engaged in 'controlled spending' and 'shopping smarter'.

What's the message to businesses?

Retailers will have to think carefully about how their offerings may fall victim to these tactics. McKinsey suggests assisting people to budget, pricing competitively and transparently, highlighting usage costs, incentivising the purchase of replacements, less wasteful packaging and promoting home-use/assembly of products. Those selling consumer equipment and holidays in the UK would do well to diversify into insurance, utilities, education or groceries.

I was in a chain store today, and the cashier left her station to take me to the shelves where I could get a 2-for-one item and add something else to cut my overall bill. I was stunned. But in a good way. I'll go back there.

Internet of Things... A European Prophecy

This just out from the European Commission: "Internet of Things: an action plan for Europe".

My apologies to the authors, but it seems to be a recital of various technological innovations concerned with RFID and the semantic web, attempting to link them to various pet EC initiatives in vague, mystical terms under a Pythonesque EU slogan. It concludes:

"As this document has described, IoT is not yet a tangible reality, but rather a prospective vision of a number of technologies that, combined together, could in the coming 5 to 15 years drastically modify the way our societies function."

Rather reminds me of this:

I guess it's all an attempt to put governments at the centre of the innovation process. What that will do to innovation is anyone's guess. Bizarrely, I see the EC is already requiring an opt-in before your milk can tell your fridge that it's past its use-by date. Lest you be "taken unawares by the new technology."

Please facilitate, don't regulate. You will only slow the pace of innovation.

Mapumental Map-Timetable-House-Price Mash-Up

See the movie, read the blog and apply here for your invitation to the private beta.

WolframAlpha Brings Semantic Web To Life

I see that WolframAlpha has just launched. It's a "computational knowledge engine" that gives an answer to your query by doing computations from its own internal knowledge base, instead of merely searching the web, returning links and leaving you to figure out an answer. That's not to say it's the right answer, any more than the search results you're used to seeing. But it's very likely to be a much better start, or at least worth doing in parallel to a normal search.

The introduction is certainly impressive already, and it's potential is limitless.

At last we are starting to see the fruits of semantic publishing, but even if it only kills price comparison sites, I'll be ecstatic!

Will The Semantic Web Kill Price Comparison Sites... Please?

Maybe I'm confused, but every time I see a TV ad for one of the multitude of financial services price comparison web sites, I wonder what proportion of the gross product price or insurance premium is commission to cover all those advertising costs.

Yet it's claimed that I'd pay the same price if I went directly to the product provider...

So then I wonder: if there has to be enough fat in the price to cover multiple distributors' TV ad campaigns, why don't product providers start semantic publishing? That way, a widget on my own computer could scan their datafeeds and identify the product that's right for me, based on my personal profile and other parameters I specify?

Surely this would bring down the cost of products and I would also see the whole market, not just those who are prepared to pay to be on the price comparison sites?

Two Stones, One Bird?

Convenience.

Because most web sites with anything remotely important on them seem to require log-in codes, I keep many different usernames and passwords in my head. Apparently, the average person uses 12 (Independent Extra, 21.11.07, p.8). That's nothing compared to the many phone numbers that we used to remember before we began relying on the directory in our mobile phones and laptops, or Skype. But it hardly aids freedom of movement around the web.

To ease my passage, so to speak, the (worryingly named) Open ID programme would have me replace all my passcodes with a single ID. It would sit in a database somewhere to be checked when I access each participating web site.

Cue another standards battle, and Round 10 between Google, Microsoft et al.

But the people working on the semantic web would say that I shouldn't have to move around the web at all. Their goal is making information "understandable by computers, so that they can perform more of the tedious work involved in finding, sharing and combining information on the web". As I recall the explanation of Dr Nick Gibbins (School of Electronics and Computer Science at the University of Southampton) at the SCL's Law 2.0 event in September, the key issues are trust and provenance in the information which the computers are being made to understand. Both vary according to the source, time and context in which the information is given, as well as the content itself. You trust Prof Lillian Edwards' view of privacy law, but not her tips on car repair. But rather than drawing on a single ID in a single (hackable) repository somewhere, the computers would rely on a whole range of circumstantial evidence to confirm that the data in question is likely to be true and relevant to you - or in a log-in scenario, that the person whose computer is trying to gain access to a database is you.

Cue another massive battle over standards, but also over ontologies, taxonomies and other elements of the semantic web that are worthy of such top-draw words.

I guess that Open ID may be a stepping stone along the way to the semantic web, in which case we should get on with it. But that does seem like two stones for the one bird. Whereas the semantic web promises convenience without humans having to do all the moving around - so two birds with one stone.

I know which sounds better.