970x125
It is undeniable that modern-day AI machines have achieved remarkable fluency with language. They seem to understand what we tell them, regardless of the words we choose to express ourselves. This enables the same conversational fluidity that we have with humans. However, we shouldn’t lose sight of the fact that LLMs are not designed to be truthful, but to ensure that the narrative “makes sense” in any context. Given a context, LLMs are trained to generate what should come next in the developing narrative. Confabulations—plausible- sounding distortions or fabrications—are part of its repertoire, regardless of whether they correspond to truth or facts in our world.
One of the primary functions of language is to imagine and enable the creation of ideas that have never been expressed before. LLMs do that easily, even when the context has nothing in common with the data on which they were trained. The narrative always makes sense in any context because the machine has learned some general structures of language that transfer to new situations. One of these is “compositionality,” the concept that the meaning of a complex expression is determined by the meanings of its parts and the way they are combined. The AI has learned several such useful regularities
On my most recent podcast, machine learning researcher Léon Bottou argues that LLMs are essentially fiction machines that can be remarkably good at conversing about new situations that are far removed from their training data. Indeed, what I find impressive is how often LLMs are truthful and correct, considering that they are not designed to be so. One reason for this could be the intensive reinforcement learning from human feedback (RLHF) from the armies of human validators employed by operators of LLMs that fine-tune their responses to be correct or socially acceptable.
Given its prowess at generating content, could the AI come up with novels? Could it come up with future theories in physics that are unknown to mankind and not represented in the training data?
Arguably, it should be easy for the AI to create new plots and write novels. After all, if LLMs are fiction machines, they should have no problem creating stories, their quality notwithstanding. As Bottou puts it: “Rather than an artificial intelligence with perfect reasoning abilities and encyclopedic knowledge, an ideal language model is best visualized as a machine that prints fiction on a tape. As new words are printed on the tape, the creation follows fresh twists and turns, borrowing facts from the training data and filling in the gaps with plausible confabulations.”
But can the AI discover novel theories?
If we already have a set of identified candidate models and the task is to identify the correct one, that’s an easy lift for the AI. However, if the theory requires new concepts to describe it, this could require assigning new meaning to existing words or creating entirely new concepts, which is a big stretch for the machine. Einstein’s theory of relativity created new meanings for existing words such as time, gravity, and force. Similarly, thermodynamics and quantum mechanics created new concepts that required the introduction of new words such as photon, quark, quantum, and entropy.
Theories usually go a step further. In addition to the symbols and concepts that express them, theories typically require a causal structure and a mathematical formulation. Causality means that the phenomenon must be understandable to humans in terms of the symbols used. This raises a deeper question about whether intelligence can be completely specified in terms of symbols. Are symbols involved in phenomena such as emotion, visual imagery, and motor control? If not, it might be impossible for us to understand a new theory that the machine comes up with if it is unable to explain it to us in terms of symbols that we can understand. This reminds me of Geoff Hinton’s metaphor of AI as an alien, one that thinks very differently from us, in which case we may not always be able to understand each other.
That’s a bizarro brave new world: an intelligent alien, albeit of our own creation, that lives alongside us and that we can’t fully understand. Not until we learn its language.
