I feel like there are two parallel discourses going on here, and it's crazy.
On the one hand, we have LLM, and people arguing that they are simply memorizing the internet and what you're getting is a predictive regurgitation from what actual people have said.
On the other hand, you have AI Art, and people arguing that it's not just copy-pasting the images it's recognized, and it's actually generating novel outputs by learning 'how to draw'.
Do you see a commonality?
It's that people are arguing whatever happens to be convenient for them.
If a model can generate human-like responses, and it has a large input token size that effectively allows it to maintain a 'memory' by sticking the history in as the input rather than being a one-shot text generator...
Really.
What is the difference between that and AGI?
Does your AGI definition mean you have to have demonstrated understanding of the underlying representations that are put in as text?
Does it have to be error free?
What fundamental aspect of probabilistic text generation means that it can't be AGI?
...because, it seems to me that it's incredibly convenient to define AGI as something that can't be represented by a LLM, when all you have really is a probabilistic output generator, and a model that currently doesn't do anything interesting.
...and it doesn't. It's not AGI. Right now; but your comment suggests that because of the technical process that the output is generated by that LLMs are fundamentally unable to produce AGI; and I think that's not correct.
The technical process is not relevant; it's simply that these models are not sophisticated enough to really be considered AGI.
...but a 5000 billion param model with a billion character token size? I dunno. I think it might start looking pretty hard to argue about.
I have the same sentiment. To me, there's two kinds of groups in most recent discussions about GPT: those who don't understand the underlying functionality at all and those who think they deeply understand it down to its bits.
The second group seems to be very stubborn in downplaying GPT et al capabilities. What's curious is that, for the first time in history of AI field, the source of general amazement is coming straight from AI responses, rather than some news or corporate announcement about how the thing works or what it will be able to do for you.
On the one hand, we have LLM, and people arguing that they are simply memorizing the internet and what you're getting is a predictive regurgitation from what actual people have said.
On the other hand, you have AI Art, and people arguing that it's not just copy-pasting the images it's recognized, and it's actually generating novel outputs by learning 'how to draw'.
Do you see a commonality?
It's that people are arguing whatever happens to be convenient for them.
If a model can generate human-like responses, and it has a large input token size that effectively allows it to maintain a 'memory' by sticking the history in as the input rather than being a one-shot text generator...
Really.
What is the difference between that and AGI?
Does your AGI definition mean you have to have demonstrated understanding of the underlying representations that are put in as text?
Does it have to be error free?
What fundamental aspect of probabilistic text generation means that it can't be AGI?
...because, it seems to me that it's incredibly convenient to define AGI as something that can't be represented by a LLM, when all you have really is a probabilistic output generator, and a model that currently doesn't do anything interesting.
...and it doesn't. It's not AGI. Right now; but your comment suggests that because of the technical process that the output is generated by that LLMs are fundamentally unable to produce AGI; and I think that's not correct.
The technical process is not relevant; it's simply that these models are not sophisticated enough to really be considered AGI.
...but a 5000 billion param model with a billion character token size? I dunno. I think it might start looking pretty hard to argue about.