I Thought Prompt Injection Was A Vaccine Thing Until I Discovered AI

Information Week

The generative AI hype bubble definitely deflated significantly during the summer. Gartner calls it the "trough of disillusionment" in the "AI hype-cycle" and points to 'agentic AI' and 'AI-native software engineering' (aka 'vibe coding') as (somewhat) distinct new entrants. Why the disillusionment? Well, on top of earlier risk management warnings, we've heard lots about the fact that inaccuracy, bias and hallucination are features of generative AI, rather than bugs, as explained very well in the 'myth busting' post by The Guardian. But what we're really hearing about now are the security vulnerabilities, which seem even more problematic for agentic AI and vibe coding. In fact, the more applications that sit on top the worse the problem gets.

AI CEOs Get Cold Feet

Having greedily rushed to get their open generative AI services to market as 'minimum viable products' leaving all the shortcomings as 'externalities', the AI bosses spent this summer pretending to care as a way of demonstrating the 'true power' of what they'd foolishly unleashed. 

Altman signposted the fraudster's charter, and later found himself on the receiving end of a tragic wrongful death suit in connection with the death of a teenage user of Chat GPT between September 2024 and January 2025. This appears to have led the CEO of Microsoft AI to begin his own hand-wringing over the illusion that you seem to be having a 'conversation' with an AI, which he couldn't resist 'branding' as 'Seemingly Conscious AI'.

Humans replaced... then rehired

Meanwhile, overly enthusiastic adopters of chatbot functionality found themselves rowing back on their plans to nuke their customer service teams. Klarna performed such a volte 'farce', as did Australia's Commonwealth Bank, making it all the more bizarre that Microsoft should publish some, er, artificial research claiming to be able to 'predict' which jobs will be replaced.

Even if it were possible to run 'agentic AI' processes that do a lot of the mundane work "...before escalating..." more complex issues, to whom would they escalate those issues? Experienced senior managers? When they retire, who will have gained the experience to replace them? 

Advisory AI?

While the state of Illinois became the first to ban the use of AI to provide therapy, the UK government remain undeterred, announcing its decision to enable the unwitting British 'populace' to use agentic AI for everything from employment advice to obtaining driving licences...

Meanwhile, lawyers have had to be warned again about the fact that open generative AI tools produce fake law.

And before you start thinking of simple processes that AIs could fulfill, it's worth pointing out that ChatGPT-5 still fails at such seemingly straightforward tasks as creating an alphabet chart with each letter represented by an animal whose name starts with that letter; maps and decision trees, among other things. But a generative AI will still boast that it - or others - can do such things. For instance, when searching for an example of poor map making, Google 'AI Overview' produced the following slop (my emphasis):

The assertion that "AI can't do a map of Europe" is false, but it highlights the limitations of generative AI in producing accurate, detailed maps, which often contain errors like misplaced cities, incorrect country borders, and inaccurate iconography. While AI has access to a vast amount of data and can create maps that look plausible, it struggles with the precision and reliability required for a complex geographical representation.

AI Insecurity

But by far the worst issues are to do with security, including 'poisoned calendar invites' containing malicious prompt injections. This is a grave issue that 'better prompting' cannot fix, and the open architecture of open generative AI militates against a 'zero trust' approach which is unlikely to be commercially viable in any event. 

Rogue AIs can only be shut down

An 'agentic AI world' would be wide open to malicious prompt injections, hallucination, bias and error. So what, you might say. We could just shut it down. Yet researchers have found that some AIs can resist shutdown and find ways to keep working on their latest tasks. 

And if you've replaced your 'traditional' staff, systems and business processes with an AI, what then? 

Further reading:

For the AI sceptic's view, I follow Professor Barry O'Sullivan, Denis O., Axel C., Simon Au-Young and Georg Zoeller. Your mileage may differ ;-)