Dangers of relying on AI

[myglaren]

Watching the BBC series "The Future With Hannah Fry ", admittedly mostly because it is Hannah Fry, the second in the series has motoring implications that many here may find interesting and disturbing.

https://www.bbc.co.uk/iplayer/episode/m ... -do-i-feel

[CitroJim]

admittedly mostly because it is Hannah Fry

The only reason I'd ever watch it

[MattBLancs]

Had an interesting repeated Ai hiccup recently:

The Google news headlines that pop up on my phone seems to be AI prompts, where if I click on one of interest, it then gets AI to give me a summary of the story. It also includes traffic reports on the M6 that I travel on regularly.

The other day it gave a headline like:

M6 is closed due to overturned lorry

So I clicked the "Interested" button to hear more:

today 15th May, the M6 is closed due to overturned lorry. Motorists are currently experiencing delays if 45 minutes. The accident happened at 9:30 AM

What confused me was mention of the date - the previous day!

So I asked it to confirm if was still closed

M6 open and flowing normally

Err, so when did it reopen?

reopened 15th May, 12:30 pm

Had almost the same another day too. AI hasn't yet got to grips with "currently" or what day it is!

[MattBLancs]

Ah, seems I've got a "history" of these discussions

1 = top line is the "headline" that triggers AI to leap into action - Day 1

2

Second occurrence:

3

^^^^ again, top line is "the prompt"

4

5

6

7

Missed between 7 and 8

8

[myglaren]

If I pose a Google question there is invariably an AI response at the top of the page.
I hurry quickly past that.

Also regarding AI, I forget where but there was news of a 'zero day' malignancy stripping details from mobile phones, including banking details.
Strengthens my reluctance to have banking and most other 'apps' on my phone.

[PaulC5]

In Atlanta an AI glitch has been causing Waymo driverless cars to go round a cul-de-sac https://www.bbc.co.uk/news/videos/czx20g00ly1o

[MattBLancs]

If I pose a Google question there is invariably an AI response at the top of the page.
I hurry quickly past that.

Bit of a faff, but if you type

-ai

At the end of your search term, Google will hide the AI bit at the top, saves a bit of scrolling past it

[myglaren]

Good tip but I am so used to ignoring it I do it without thinking now.

[CitroJim]

It all goes to show AI still has a way to go... I know this sounds ironic but it seems to me that AI needs to be used with a great deal of awareness and intelligence. Take what it says at face value at your peril

[MattBLancs]

Not sure if you'll need Facebook to view this, but a superficially very convincing but "the more you look the worse it gets"

Audi


Highlights include:

Spoiler: show

wheel spats screwed into ...

the tyre!

And a dyslexic rev counter:

Spoiler: show

[mickthemaverick]

"Loading Facebook SDK. You will need to disable any adblocker, privacy extension, or built-in tracking protection."

Won't be happening here then but thanks for the glimpses Matt, just further evidence of the whole exasperating world we seem to be being pushed into!! I shall continue to resist in every way I can!!

[MattBLancs]

"Loading Facebook SDK. You will need to disable any adblocker, privacy extension, or built-in tracking protection."

Ah, typical! Shame it wouldn't just play it!

Won't be happening here then but thanks for the glimpses Matt, just further evidence of the whole exasperating world we seem to be being pushed into!! I shall continue to resist in every way I can!!

Yep fair enough.

Facebook has decided I must have enjoyed the nonsense and keeps showing me various "classic car previews" each a quick series of exterior and interior shots.

It's generally the interiors that generate "hang on a minute, what!?!" So here's a few more for your enjoyment:

Speedo

Tucker odometer dispensing with numbers, Speedo again random smattering of numbers

Empty and full apparently will do for fuel and amps.

Radio a particular treat too! Speaking of Amps:

Hmm, oil or amps, which to choose!?

[myglaren]

Rather a long read but most informative, and concerning.

"A Dutch computer scientist gave one lecture in 1988 arguing that programming is unlike anything humans have ever tried to do before, and the reason most software on earth is broken is that we are still teaching it as if it were a hobby.

His name was Edsger Dijkstra. He won the Turing Award in 1972. He invented the shortest path algorithm that every GPS on earth still runs on.

He wrote the paper that killed the goto statement in modern programming languages.

He spent 50 years quietly being one of the most consequential thinkers in the entire history of computer science, and he was in a very bad mood by the time he stood up at the ACM Computer Science Conference in 1988 to deliver the lecture that almost nobody at the conference wanted to hear.

The lecture was called On the Cruelty of Really Teaching Computer Science.

It is now one of the most cited papers in the entire history of computing education. It was filed in his archive as EWD1036, handwritten in his careful fountain-pen calligraphy because he refused to use a typewriter and famously refused to use email for the rest of his life.

The argument was simple and uncomfortable.

Programming, Dijkstra said, is a radical novelty. Not a new tool. Not a new skill. Not a faster version of something humans already knew how to do. A genuinely new category of intellectual activity that has no real precedent in the entire history of the human species, and our brains have not been built to handle it.

Here is what he meant by that.

When a programmer writes a line of high-level code and presses run, that single line might trigger a billion operations at the level of the silicon.

The ratio between the abstraction you are working in and the physical events you are actually causing is roughly one billion to one. No engineer in history before computing ever had to reason about a system spanning that kind of ratio inside their own head.

A bridge builder reasons about steel beams and the physics of weight. A surgeon reasons about organs and the physics of tissue. A chemist reasons about molecules and the physics of bonds.

All of them are working inside ratios of physical scale where the largest and smallest things they need to think about are within a few orders of magnitude of each other.


A programmer routinely writes one line that orchestrates a billion physical events on a chip, and is expected to predict the behavior of all of them in advance.

Dijkstra argued that the human brain was simply not built for this. Every intuition we have evolved over hundreds of thousands of years comes from a world of medium-sized objects behaving in continuous ways. Computing is the opposite. It is discrete, not continuous.

A program that runs perfectly a billion times can crash on the billion-and-first iteration because of a single bit. A single character missing from a line of code can take down a power grid. There is no margin. There is no graceful degradation. The system either works or does not, and the only way to know is to actually run it.

This was the part of the lecture where Dijkstra made everyone in the room uncomfortable.

He said the way computer science was being taught in universities was a quiet disaster. Professors were teaching programming the way carpenters teach woodworking. With examples. With metaphors. With analogies to things students already understood. Files are like folders. Memory is like a desk. A function is like a recipe.

Dijkstra said this was actively making it harder for students to think clearly. The whole point of a radical novelty is that there is nothing in your past experience to compare it to

The moment you start reaching for metaphors, you are smuggling in old intuitions that do not apply, and those intuitions will betray you the first time you try to reason about a system the metaphor was not built to describe.

His exact line was this: the usual way in which we plan today for tomorrow is in yesterday's vocabulary. And yesterday's vocabulary, he argued, was killing the field.

The reason most software is broken is downstream of this single misunderstanding. Programmers are taught to think of code as a craft. Something you get a feel for

Something you pick up through practice. Something where intuition gets sharper with experience.

Dijkstra said this is exactly backwards. Programming is not a craft. It is closer to mathematics than to carpentry, and the moment you treat it as a craft, you guarantee that the software you produce will be full of the kind of bugs that craftsmanship cannot catch.

The fix, in his view, was to teach programming the way mathematics is taught. You should be able to prove your program correct before you run it.

You should reason about your code formally, the way a mathematician reasons about a theorem, not the way a carpenter feels their way through a joint. The students who learned this way, he said, would walk out of their classes with a kind of confidence that no amount of typing practice could produce.

The lecture was published in Communications of the ACM in 1989. The field did not listen. Universities kept teaching programming the same way.

Software kept getting bigger. Bugs kept compounding. By 2026, almost every piece of software on earth has known security vulnerabilities, undefined behaviors, and edge cases that nobody has ever proven safe. The doom that Dijkstra warned about in 1988 is now the default condition of the digital world we have built.

The deeper lesson is the one most readers miss the first time through.

Dijkstra was not just talking about software. He was making a much bigger point about how humans learn anything that is genuinely new. The instinct to translate the unfamiliar into the familiar is the most natural thing in the world.

It is also the single biggest obstacle to actually understanding something that has no precedent. If you keep reaching for analogies, you will never see the new thing clearly. You will only see your old framework projected onto it.

This is happening right now with AI. The same instinct that made people learn programming through metaphors of files and folders is making people understand large language models through metaphors of brains and people.

Almost every framework being used to describe AI in 2026 is borrowed from a previous domain. None of them quite fit. The few people who are actually building useful intuitions about how these systems work are the ones who have done what Dijkstra recommended forty years ago.

They have set down the old vocabulary. They have looked at the new thing on its own terms. They have accepted that the radical novelty is radical for a reason.

You are not slow. You were taught a discipline as if it were a hobby. The cruelty is real.

The fix is still available".