Quote of the day by computer scientist Judea Pearl: 'Deep learning has instead given us machines with truly impressive abilities but no intelligence' — capturing the central fault line in modern AI.
I was just involved in a comment war on this platform with someone who simultaneously denied that a genocide is occurring in Gaza while defending Hamas’s killing of civilians as a necessary byproduct of their war for freedom.
He also had something of a soft spot for ISIS and attempted to convince me that America had committed genocide against ISIS because, apparently, ISIS is a society consisting of tens of millions of people.
Out of the thousands of comments I’ve made on this platform, this was, without question, the most bizarre interaction I’ve had to date.
Now, you might be asking yourself, “What on earth does any of this have to do with the comment I just left?”
And the answer is very simple: absolutely nothing.
You mentioned ISIS, which reminded me of this particularly memorable encounter, and apparently I felt compelled to share it with everyone.
AI can write a convincing essay about fire, but it has never burned a finger. Pearl’s point is less that these systems are useless and more that fluency gets mistaken for understanding. Silicon Valley saw a spectacular autocomplete and immediately started selling it as a digital philosopher. The marketing department achieved AGI first.
That also often happens with people. I guess these are the same people that think AI is actual intelligent because that’s the only intelligence they know: being good with words without deeper understanding.
To put it phosophical words are just our attempt to describe and understand the world around us. There is no truth in them other than that to what they are trying to point in the real world.
Just so you know, the account you replied to is itself just an AI spewing its mindless, oozing bullshit here as comments. Why instances aren’t banning it is beyond me.
Well, I believe it’s a person who uses an AI to create comments. More managed and hands on, rather than something fully automated, and occasionally they might be taking complete control. Some kind of idiotic "experiment ", I’m sure.
And a huge component is that Silicon Valley mindset that… values fluency over understanding, I guess?
Call me cynical if you will, but I think it’s more that Silicon Valley values short term profit over integrity. They know the limitations of what they are selling, but if it makes money, that drives everything.
I don’t think they care about limitations (if even they can understand them, let alone spend time assessing), they are only in for the quick buck this quarter.
Steal it freely. I think the mindset is less about valuing fluency than treating confidence as evidence. Holmes sold a story so convincingly that asking whether the machine worked became almost rude. Now we’re doing it again, except the machine can write its own pitch deck.
Yes, LLMs are incredible tools if used in appropriate situations by people who understand LLMs’ limitations. Like any other tool, actually. Give a drill to an idiot and just wait for the disaster.
In case anyone is interested, here’s an article I really like that very well describes how LLMs are not like all other tools, in particular how they’re are fundamentally different than tools like a drill, and how this difference is bad actually: https://deadsimpletech.com/blog/no-such-thing-as-just-a-tool
Not disagreeing that an idiot with a drill is also a disaster, I just think there are some interesting differences worth lingering on too!
Interesting read. I agree with the overall analysis. I don’t agree with the premise that calling something a tool somehow implies that its use is neutral. Of course it isn’t.
The article fundamentally agrees that LLMs are tools. Just that being a tool comes with a lot of luggage and indirectly shapes human behavior. We certainly ought to think about what kind of tools we want to create and what their effects on society are.
I think the frasing the article critiques is saying that something is “just a tool” and not saying that something is “a tool”, as it agree with you that the latter doesn’t imply the same neutrality as the first.
Writing this comment I realised how hard it is to tell whether “just” is part of the quote or not without using quotation marks, so haha that can sure be confusing.
I stopped reading after a few paragraphs. No idea what the author wanted to say. Reference to old philosophers is cool, but first you need to explain WHAT you’re trying to say, and only then pour down all that philosopher gibberish onto reader.
I generally think most philosophy is overly academic wank written by people who desperately need to go outside but I also think the author of that article does a very good job laying out exactly what they’re trying to say and why it matters. You might actually learn a thing or two if you bothered to read it.
Exactly. A useful tool becomes dangerous when people mistake its output for judgment, especially when the people selling it have every incentive to encourage that confusion.
The quote is from 2018. Back then most neural networks were still mostly single purpose - far from today’s all-in-one package. Text generation was barely working at all.
The underlying problems are still the same. Transformer models do not have an inkling of intelligence, and they still don’t have an internal model of the world around them. Creativity is also non-existent, which is a hard requirement if you want these things to actually create something new instead of rehashing what already exists.
There was no rehashing in 2018. What you did was you trained your own (local) model on some data to help you work. For example, a grad student would painstakingly mark cells in em-scans for weeks and even months, now they could do it for a day, use that result to train a neural network and only supervise the job done. And the job wiuld take minutes instaid of weeks or months.
There were no llms in 2018, IIRC the google/deepmind paper was from 2019 (can be wrong here).
Source: built AI training app in 2018 for science company.
they still don’t have an internal model of the world around them
This has been shown repeatedly to not be correct since 2023.
That’s literally how transformers turned out to have been working so well. They build world models and bags of heuristics fitting the training data to extrapolate from.
It’s almost surprising you managed to use the correct term for the tech but then managed to misrepresent it so egregiously.
You really think that an Othello board could represent the world? Talk about being delusional. This is pathetic. The subheadline: “OTHELLO-GPT USUALLY PREDICTS LEGAL MOVE” says it all - it cannot have a representative internal world model, because in that case the “USUALLY” would go away, as anyone who has played othello for longer than 10 minutes will tell you - and we are talking about a game with very simple rules here, the complexity does not even reach chess - and people expect these things to spit out coherent and valid solutions from target selection in wars to the color their bedroom walls should be painted in.
Edit: I really have to add here the following excerpt (percentages are the rate of invalid moves as the top choice):
The error rate is 0.01% for Othello-GPT trained on the synthetic dataset and 5.17% for Othello-GPT trained on the championship dataset. For comparison, the untrained Othello-GPT has an error rate of 93.29%.
So even if you just feed them real world data without completely synthesizing it, 1 in 20 moves are invalid in a very simple (rule-side) game. If you do not give them a whole lot of played games as a basis, it fails in over 93% of cases (not at making a GOOD move, just at making any valid move!), which is probably very close to a random guess for a move. Something that has even an inkling of an internal world view would be able to adapt after being told that their move is invalid. Also, an internal world view would allow for simply dropping the rulebook onto ChatGPT as a basis, as it works with humans that can imagine.
You really think that an Othello board could represent the world?
No, you seem to be misunderstanding what a “world model” is. That research was on a toy model a fraction of a fraction of the size of modern models and exclusively trained on board notation moves. And yet it modeled the geometry of a board and positions of pieces and ‘my’ vs ‘opponent’ status of those pieces (there’s been multiple follow-ups and replications looking deeper at what encoded).
The question it answered is that transformers don’t simply learn surface statistics from the data but build world models from the data they are trained on.
So when you take a model orders of magnitude larger and train them on collective human culture orders (of orders) of magnitude larger than the board moves in the Othello-GPT paper, what do you think happens? No world modeling at all? Well there’s been a number of interpretability papers since 2023 finding that modem LLMs have functional models of all sorts of things, even including tracking emotions for themselves vs others.
Also, you are assuming that a trained world model is a lossless identical version, which it isn’t and nobody claimed. In fact, arguably the best way to identify AI generated videos these days is by over-expressed emotions of characters. Why? Because the training data is over indexed on influencer exaggerated faces which game clicks so video and image world models of human emotions are miscalibrated.
Transformers have a world model, but often it’s of a kind of parallel imagined/extrapolated world that’s not the real one. Another good example of this is how most models are certain that real world news after their cutoff isn’t true because extrapolating from their internal world model (from the training data before their cutoff) those events seem improbable. The real world doesn’t exactly match the internal model and that drift grows the more they need to extrapolate.
If you want to continue to discuss (I’m happy to help explain and educate you), I’d recommend reading up a bit on what “world model” means in relation to the tech.
I know very well what a “world model’” is, please stop trying to undermine my credibility.
If you feed these things thousands of datasets that only consist of a specific set of coordinates, your output will also only consist of these coordinates, which will create the illusion that they actually have knowledge about how an game board looks (as i said, these things are remixers after all). And feeding it a championship data set, where not one invalid move will appear, should ensure at least a close to 100% of valid moves if they were actually able to have an actual internal model of the system they are trying to manipulate. (and in reality, it should choose pretty good moves too, but this was not evaluated at all…).
Their test cases (and the error rates!) were not even “play a game of Othello”, but feeding them gamestates and asking them for a legal move. IMO, this whole setup does not require any form or shape of a world model, and if it actually existed, the error rates would be a ton lower - the championship data will not contain a single invalid move, and the concept of choosing an empty space on the board, touching the other players piece with one of my own ones on the other side of the mass of pieces, is really not a hard one.
I agree that current models will probably have a better performance, but this is NOT because of a better “world model”, but because the statistical sieve has become a lot finer by blowing up the amount of parameters. This approach is valid, until the amount of parameters becomes unhandleable by current hardware, and it also means much higher energy costs for these computations.
If it had any intelligence it’d be over. For better or worse.
It’d be humanity’s last invention.
You wouldn’t get news reports that the machine has gotten 30% smarter over the last three months or something.
It’d be like waiting at a station platform and then getting knocked over by a gust of wind as the train disappears over the other horizon.
My conclusion is that they’re auto-indexing books capable of rapidly temporarily integrating and regurgitating knowledge they store if you inject electricity into them, but still just books in the end after you cut their power.
Comments
…”truly impressive abilities and no intelligence but the bad actors waiting to exploit those abilities.”
I still can’t believe that his son was the WSJ journalist who was beheaded by ISIS.
Privilege is a hell of a drug.
I was just involved in a comment war on this platform with someone who simultaneously denied that a genocide is occurring in Gaza while defending Hamas’s killing of civilians as a necessary byproduct of their war for freedom.
He also had something of a soft spot for ISIS and attempted to convince me that America had committed genocide against ISIS because, apparently, ISIS is a society consisting of tens of millions of people.
Out of the thousands of comments I’ve made on this platform, this was, without question, the most bizarre interaction I’ve had to date.
Now, you might be asking yourself, “What on earth does any of this have to do with the comment I just left?”
And the answer is very simple: absolutely nothing.
You mentioned ISIS, which reminded me of this particularly memorable encounter, and apparently I felt compelled to share it with everyone.
Anyway, thank you for coming to my TED Talk.
THANK YOU FOR YOUR ATTENTION TO THIS MATTER.
PRESIDENT AGENT ORANGE.
AI can write a convincing essay about fire, but it has never burned a finger. Pearl’s point is less that these systems are useless and more that fluency gets mistaken for understanding. Silicon Valley saw a spectacular autocomplete and immediately started selling it as a digital philosopher. The marketing department achieved AGI first.
That’s it. I am going to borrow that phrase.
And a huge component is that Silicon Valley mindset that… values fluency over understanding, I guess?
The words to describe it are hard. The “Elizabeth Holmes Complex” in You Can See Everything pretty much sums it up.
That also often happens with people. I guess these are the same people that think AI is actual intelligent because that’s the only intelligence they know: being good with words without deeper understanding.
To put it phosophical words are just our attempt to describe and understand the world around us. There is no truth in them other than that to what they are trying to point in the real world.
Just so you know, the account you replied to is itself just an AI spewing its mindless, oozing bullshit here as comments. Why instances aren’t banning it is beyond me.
Nah, there’s a person behind it at least.
…I have almost blocked eicker for posting Futurism clickbait, though. I didn’t come to Lemmy for that junk in my feed.
Well, I believe it’s a person who uses an AI to create comments. More managed and hands on, rather than something fully automated, and occasionally they might be taking complete control. Some kind of idiotic "experiment ", I’m sure.
Call me cynical if you will, but I think it’s more that Silicon Valley values short term profit over integrity. They know the limitations of what they are selling, but if it makes money, that drives everything.
I don’t think they care about limitations (if even they can understand them, let alone spend time assessing), they are only in for the quick buck this quarter.
Source: have worked for normal american company.
Steal it freely. I think the mindset is less about valuing fluency than treating confidence as evidence. Holmes sold a story so convincingly that asking whether the machine worked became almost rude. Now we’re doing it again, except the machine can write its own pitch deck.
Yes, LLMs are incredible tools if used in appropriate situations by people who understand LLMs’ limitations. Like any other tool, actually. Give a drill to an idiot and just wait for the disaster.
In case anyone is interested, here’s an article I really like that very well describes how LLMs are not like all other tools, in particular how they’re are fundamentally different than tools like a drill, and how this difference is bad actually: https://deadsimpletech.com/blog/no-such-thing-as-just-a-tool
Not disagreeing that an idiot with a drill is also a disaster, I just think there are some interesting differences worth lingering on too!
That was a really interesting article with ideas that were new to me. Thanks for sharing it.
Interesting read. I agree with the overall analysis. I don’t agree with the premise that calling something a tool somehow implies that its use is neutral. Of course it isn’t.
The article fundamentally agrees that LLMs are tools. Just that being a tool comes with a lot of luggage and indirectly shapes human behavior. We certainly ought to think about what kind of tools we want to create and what their effects on society are.
I think the frasing the article critiques is saying that something is “just a tool” and not saying that something is “a tool”, as it agree with you that the latter doesn’t imply the same neutrality as the first.
Writing this comment I realised how hard it is to tell whether “just” is part of the quote or not without using quotation marks, so haha that can sure be confusing.
I stopped reading after a few paragraphs. No idea what the author wanted to say. Reference to old philosophers is cool, but first you need to explain WHAT you’re trying to say, and only then pour down all that philosopher gibberish onto reader.
It’s not that long, but I agree that it does not get to the point, just ramble about how tools have imperfections. Could have been way better.
In a nutshell; the tool becomes a part of you, and thus limits your options.
Or so I read it.
I generally think most philosophy is overly academic wank written by people who desperately need to go outside but I also think the author of that article does a very good job laying out exactly what they’re trying to say and why it matters. You might actually learn a thing or two if you bothered to read it.
“A fool with a tool is still a fool” - Grady Booch (father of UML)
I’ve always wondered about Ungly Mutant Lad’s backstory.
Before he joined the revengers
He tried to warn us about his son
Exactly. A useful tool becomes dangerous when people mistake its output for judgment, especially when the people selling it have every incentive to encourage that confusion.
I will become truly frightened the day an LLM grasps humor. I believe this might be the canary in the coal mine for true artificial intelligence.
Until then, I will continue to believe these are overhyped digital parrots. I’m not holding my breath that an LLM could ever get there, though.
Roughly citing Taleb: in a way, we have managed, indeed, to create the utopia that people and computers think alike.
…though we didn’t manage to bring about that machines think like people, but exactly the opposite.
The quote is from 2018. Back then most neural networks were still mostly single purpose - far from today’s all-in-one package. Text generation was barely working at all.
The underlying problems are still the same. Transformer models do not have an inkling of intelligence, and they still don’t have an internal model of the world around them. Creativity is also non-existent, which is a hard requirement if you want these things to actually create something new instead of rehashing what already exists.
There was no rehashing in 2018. What you did was you trained your own (local) model on some data to help you work. For example, a grad student would painstakingly mark cells in em-scans for weeks and even months, now they could do it for a day, use that result to train a neural network and only supervise the job done. And the job wiuld take minutes instaid of weeks or months.
There were no llms in 2018, IIRC the google/deepmind paper was from 2019 (can be wrong here).
Source: built AI training app in 2018 for science company.
This has been shown repeatedly to not be correct since 2023.
That’s literally how transformers turned out to have been working so well. They build world models and bags of heuristics fitting the training data to extrapolate from.
It’s almost surprising you managed to use the correct term for the tech but then managed to misrepresent it so egregiously.
You really think that an Othello board could represent the world? Talk about being delusional. This is pathetic. The subheadline: “OTHELLO-GPT USUALLY PREDICTS LEGAL MOVE” says it all - it cannot have a representative internal world model, because in that case the “USUALLY” would go away, as anyone who has played othello for longer than 10 minutes will tell you - and we are talking about a game with very simple rules here, the complexity does not even reach chess - and people expect these things to spit out coherent and valid solutions from target selection in wars to the color their bedroom walls should be painted in.
Edit: I really have to add here the following excerpt (percentages are the rate of invalid moves as the top choice):
So even if you just feed them real world data without completely synthesizing it, 1 in 20 moves are invalid in a very simple (rule-side) game. If you do not give them a whole lot of played games as a basis, it fails in over 93% of cases (not at making a GOOD move, just at making any valid move!), which is probably very close to a random guess for a move. Something that has even an inkling of an internal world view would be able to adapt after being told that their move is invalid. Also, an internal world view would allow for simply dropping the rulebook onto ChatGPT as a basis, as it works with humans that can imagine.
No, you seem to be misunderstanding what a “world model” is. That research was on a toy model a fraction of a fraction of the size of modern models and exclusively trained on board notation moves. And yet it modeled the geometry of a board and positions of pieces and ‘my’ vs ‘opponent’ status of those pieces (there’s been multiple follow-ups and replications looking deeper at what encoded).
The question it answered is that transformers don’t simply learn surface statistics from the data but build world models from the data they are trained on.
So when you take a model orders of magnitude larger and train them on collective human culture orders (of orders) of magnitude larger than the board moves in the Othello-GPT paper, what do you think happens? No world modeling at all? Well there’s been a number of interpretability papers since 2023 finding that modem LLMs have functional models of all sorts of things, even including tracking emotions for themselves vs others.
Also, you are assuming that a trained world model is a lossless identical version, which it isn’t and nobody claimed. In fact, arguably the best way to identify AI generated videos these days is by over-expressed emotions of characters. Why? Because the training data is over indexed on influencer exaggerated faces which game clicks so video and image world models of human emotions are miscalibrated.
Transformers have a world model, but often it’s of a kind of parallel imagined/extrapolated world that’s not the real one. Another good example of this is how most models are certain that real world news after their cutoff isn’t true because extrapolating from their internal world model (from the training data before their cutoff) those events seem improbable. The real world doesn’t exactly match the internal model and that drift grows the more they need to extrapolate.
If you want to continue to discuss (I’m happy to help explain and educate you), I’d recommend reading up a bit on what “world model” means in relation to the tech.
I know very well what a “world model’” is, please stop trying to undermine my credibility.
If you feed these things thousands of datasets that only consist of a specific set of coordinates, your output will also only consist of these coordinates, which will create the illusion that they actually have knowledge about how an game board looks (as i said, these things are remixers after all). And feeding it a championship data set, where not one invalid move will appear, should ensure at least a close to 100% of valid moves if they were actually able to have an actual internal model of the system they are trying to manipulate. (and in reality, it should choose pretty good moves too, but this was not evaluated at all…).
Their test cases (and the error rates!) were not even “play a game of Othello”, but feeding them gamestates and asking them for a legal move. IMO, this whole setup does not require any form or shape of a world model, and if it actually existed, the error rates would be a ton lower - the championship data will not contain a single invalid move, and the concept of choosing an empty space on the board, touching the other players piece with one of my own ones on the other side of the mass of pieces, is really not a hard one.
I agree that current models will probably have a better performance, but this is NOT because of a better “world model”, but because the statistical sieve has become a lot finer by blowing up the amount of parameters. This approach is valid, until the amount of parameters becomes unhandleable by current hardware, and it also means much higher energy costs for these computations.
There’s been no qualitative change ever. Computers are computing today just like in 2018 and like 1960.
Just like a slug neuron and a human brain neuron? No difference at all?
When the quote of the week is something that any child can easily recognize… The AImperor has no clothes.
There’s probably an extraterrestrial machine intelligence watching our planet and saying the same thing about our meat-based computers.
Well duh.
If it had any intelligence it’d be over. For better or worse.
It’d be humanity’s last invention.
You wouldn’t get news reports that the machine has gotten 30% smarter over the last three months or something.
It’d be like waiting at a station platform and then getting knocked over by a gust of wind as the train disappears over the other horizon.
My conclusion is that they’re auto-indexing books capable of rapidly temporarily integrating and regurgitating knowledge they store if you inject electricity into them, but still just books in the end after you cut their power.
Sign in to comment. Sign in