Comment on Quote of the day by computer scientist Judea Pearl: 'Deep learning has instead given us machines with truly impressive abilities but no intelligence' — capturing the central fault line in modern AI.
You are viewing one comment and its replies. View all comments · View the comment above
Comments
By “required information for the other proofs”, what do you mean exactly?
The proof, more or less in the same structure that the AI output, existed in the literature? This would’ve been published already and would be very easy to prove was stolen from the input corpus. We know this isn’t true.
The theorems and techniques used in the generated proof existed, but were not yet assembled into a final proof? Trained mathematicians have looked at the proof and seem to think it contains novel techniques, or at least novel applications of techniques from different areas of math. Even if that weren’t true, this would still be novel work; a lot of math breakthroughs are made this way, and it’s a nontrivial feat. Although if this was as far as it could go, we would run into limitations at some point.
The starting point of the proof existed, but the entire proof was built from the ground up by AI, using no existing methods or theorems? This would be very impressive, and as noted above the consensus seems to be that it did create some new techniques, but I’m not experienced or knowledgeable enough to verify that claim. Either way it definitely used at least some previous work for nontrivial reasons.
It looks like it’s somewhere between options 2 and 3 in reality; it used a lot of existing techniques and at least invented some new applications of existing techniques if not created new ones entirely (the line can be blurry). This is also what all mathematicians do! Remixing is coming up with new stuff. No thoughts, or generated output from LLMs, exist without prior context.
In the case i mentioned, it is very much number 2 IMO - the coincidence is just too much. In other cases, we are probably looking mostly at number 1: techniques that already exist, applied to other situations. These things are very good at iterating through rather uncommon combinations and have a large corpus of techniques in their training data, letting them walk across the “plane” of combinations quickly.
That’s not how humans work, and that’s probably the reason why there was a bunch of mathematical problems solved in a short time - the ingredients were there, but noone had yet tried that specific approach.
I have not yet seen any evidence of number 3, mainly because we do not know what these models are being trained on - transparency would give us the possibility to check, but we will never get it from OpenAI.
Late Edit: This is the same situation as we currently have with the security incidents popping up left and right. Existing techniques are being applied across a large corpus of code, surfacing security issues. Just like with the coding problems, the maths well will run dry after the obvious (for an LLM) stuff has been fixed or solved, leaving us humans alone again at the front of expansion of knowledge. New discoveries will help apply them across the rest of pretty much everything (which these models are really good at!), but the breakthrough itself is still human intelligence at work.
Sign in to comment. Sign in