Every day a new scientific paper is posted here that is like straight from the desk of captain obvious. It's a language model. It guesses words based on previous words. We know this.
A significant fraction of ML-adjacent people think it can do more.
I'll say that it's more than just words. LLMs can learn patterns, and patterns of patterns, recursively to a degree. They can represent real knowledge about the real world to the degree that this is revealed through the text they train on. This means LLMs can make inferences based on similarities, sometimes similarities at a surprisingly abstract level. And reasoning at the basic logical step by step can of course be done, since that can be reduced to textual pattern matching and string substitution.
But LLMs have no computational space to, for example, read about the description of a novel computation, and then perform the computation without using generated text as a scratchpad, if the computation physically takes more steps than are available in its feedforward stack. It would need to call out to a subsystem in that case. And callable subsystems are ripe for abuse through confused deputy - LLMs are not reliable deputies.
There's a lot of people, text-oriented people, who mistake authorial voice for animus. To me this is like mistaking a CGI animation for a real person behind frosted glass. Text is a low bandwidth medium and it relies on the reader bringing their own mental model to the party. So a machine which produces convincing text has a high leverage tool to seem more capable than it is.
In a sense, LLMs- particularly "conversation-shaped" LLMs like ChatGPT- harvest the benefit of the doubt we are all, as readers, used to providing to text.
For most of our lives, most of the text we have encountered was an intentional communication, self-evident through its own existence. LLMs challenge us with something new: text that has the shape of communication, but no intent.
The proliferation of generative "AI" for text will profoundly alter the human relationship to the printed word, and perhaps ultimately dispell that benefit of the doubt.
I don't think it's obvious. First of all there's the "magic" and often surprisingly good results that leads many to think there's something else there.
Then it's also not entirely clear that nothing else is learnt beyond next word, there was talk of runtime reconfigurable neural nets being embedded within the transformer weights where the models can learn on the fly and such.
And I think some believed that there was some higher-level encoding of human reasoning happening necessary to predict the next word well in some contexts beyond just memorization.
I think research like this is necessary to put some "obvious beliefs" onto solid ground.
So do I when I respond to your comment, or talk to another person while staying on the same topic. Am I better at staying consistent in output quality and at referring to past events? Yes, but I also have more than 70B parameters.
Side note: I personally have trouble speaking fluently sometimes for no reason, and in those situations I have to manually dig for one word after the other while my brain seems to be temporarily unable to translate thoughts to language in realtime. I would prefer if people calling LLMs word guessers would provide reasons for why they think humans are fundamentally different.
Well... sure. But OpenAI and MSFT have gone to a lot of trouble to build up the mystique around GPT-4 by being secretive about its architecture and publishing papers with tantalizing phrases like "sparks of AGI" and so on. I think this type of thing provides a useful counterbalance.
>But OpenAI and MSFT have gone to a lot of trouble to build up the mystique around GPT-4 by being secretive about its architecture and publishing papers with tantalizing phrases like "sparks of AGI" and so on.
An LLM will never be AGI itself. They are word calculators. However, a word calculator is precisely the tool we were missing to be able to create AGI. I believe OpenAI will be left in the dust with this stuff, as federated agents built on open models connect and induce the singularity.
> However, a word calculator is precisely the tool we were missing to be able to create AGI
This seems like the kind of step that the person above you was complaining about.
The "emergent" features of LLMs, or LLMs even being a step in the direction of AGI is entirely unproven so far. They are however powerful enough that they spark the imagination and hypothesis of tons of amateur futurists (and many financial backers of such proyects)
>The "emergent" features of LLMs, or LLMs even being a step in the direction of AGI is entirely unproven so far. They are however powerful enough that they spark the imagination and hypothesis of tons of amateur futurists (and many financial backers of such proyects)
That's exactly what I was saying, that it's a mistake to ever think of LLMs as AI. They are the prefrontal cortex. The I/O mechanism. But we still need the spark of agency. The soul if you will. Point being that we can actually work on that for real now, since the language part has been handled.
Can you support the position that language models are unable to reason?
Secondly, how do you accurately guess the next word without the ability to reason? If reasoning can arise from GPT-4's architecture then we should assume that it will with enough scale. Given we don't even know the architecture of GPT-4 I genuinely have no idea how people make these baseless claims so confidently.
"It's a language model" and "it's just guessing the next token" isn't an argument. You're just a collection of atoms obeying physical laws. Obviously you don't reason. Am I doing this right?
Language modelling is the objective (if we ignore RLHF). That doesn't mean that interesting kinds of reasoning can't emerge. You could just as easily dismiss humans as being 'biological replicators' which just 'reproduce their genomes'.
Wait for the influx of HN comments to disagree with you, citing specific prompts that they found working. It may be obvious to you and me that it’s smoke and mirrors, but a lot of smart people fall for it.
"Stochastic Parrot" is a really tired take and none of the major players, Ilya Sutskever, Andrej Karpathy, etc. believe that's all these models are doing.
"Smart" is a useless concept because it conflates cleverness and wisdom, which are orthogonal dimensions. Clever people who are not wise fall for lots of things.
Why can't this fallacy just die already?
GPT "guesses" just like ZIP guesses random bits to archive and un-archive files.
Except GPT is lossy and IMENSELEY more powerful than the lossless ZIP.
It's not a fallacy. Previous words are very important part of its scratch space. Few-shot learning is based on previous words. Prompt modifiers like "let's think step by step" encourage encoding of reasoning verbosely in words, which then allow simpler induction rules to be pattern matched onto the previous words. Previous words is what gives an otherwise feed-forward network a way to recur.
I feel it is, because it implies it just some statistical trick that's being performed, which is not true at all, imho.
I don't know enough about language models, my machine learning knowledge stops to around 2018, but I know from image recognition/style transfer that there's a lot of high-level self-organization/abstraction in those neural nets, and from the results I get from Chat GPT there's no doubt in my mind it's very well capable of reasoning and generalization.
"Guessing" implies those guesses are being compared for correctness against a reference. That only happens during training; the rest of the time, it's not guessing, it's selecting words. But then, how else would you expect a sentence to be made? First writing out all the vowels, and then filling in the rest of the letters between them?
You say that, but based on the way many people are treating these LLMs, and imagining their consequences -- they are either treating them as an oracle with reasoning powers, or expecting that they'll just naturally become them through (hand waving) "progress."
Decades of Moore's law has given some people the impression that there's a progressive & exponential improvement in almost all things "technology." Which I think is wrong or misleading when talking about this subject.
I'm just finishing the introduction section of the paper. I'm a bit out of my depth, but impressed so far. It is very well written.
I'm not entirely convinced that is all there is to it. I had it write some code and associated unit tests, and then it came up with passing and failing examples. I also prompted for function results based on arbitrary input, and it would perform the calculations.
It has some emergent ability to evaluate code IMO. I do believe this ability has been drastically reduced in the last several months. It no longer executes complex code as reliably as it once did.
I agree that it should be obvious, but a lot of people seem to be under the misapprehension that we've got sapient AI out there. I suppose those people aren't reading a lot of journal preprints, but articles like this might trickle down to them eventually. You never know. And this one has a title that is just short enough that it might sink in.
