> "If the size of the input is fixed and there is a limit to the number of possible tokens (words, or even numbers with bounded precision) then the state of the transformer at any given moment is describable with a string of fixed length."
Doesn't this also likely describe humans? Albeit, we don't currently have the capacity to inspect our brains that way, so we can't prove it.
You haven’t read the article carefully… It seems you halted early.
But the whole goalposts argument is dopey. How are we supposed to do this? Should the entire field have imagined the entire future of the field in 1952 to set ultimate goalposts? Of course not.
I personally specified unreachable goalposts in 1990 or so (general AI is only achieved when we can test it, and testing it is only possible if we can specify the behavior of general AI). I haven’t moved them and they haven’t been reached (Nobody has tested ChatGPT4 against a consensus spec of intelligence because there is no such consensus. For that same reason we can’t even prove that a given human is not insane— we only accept that people are sane due to social convention. Yet social convention cannot be enough to say that a machine built by humans is equivalent to a human).
Isn’t it okay if people come up with interesting smaller goals and explore them without committing to the larger implications?
I would say "moving the goalposts" is an understatement.
Requiring that a system's state must not be describable with a string of fixed length is an absurd definition of intelligence. By that definition, anything composed of a finite number of atoms can never be intelligent.
I think the idea that we have to give a precise definition of general intelligence is mistaken. We start from the premise that humans are generally intelligent, and then we build systems that are able to compete with humans across multiple domains. LLMs outperform humans on many language-based tasks, so I would say they are already generally intelligent. Of course, they do poorly on a lot of tasks as well, so there is a lot of room for improvement.
>Yet social convention cannot be enough to say that a machine built by humans is equivalent to a human.
I would say this is self-evidently untrue. If a polity socially accepts a given machine (or a given category of machine, like "Every machine which applies to and passes this particular test.") as equivalent to human, that would be enough by definition because personhood is socially defined. We do conceptually similar things to categorise different categories of humans and grant them differentiated rights; the most obvious example is citizenship, which requires application and a test in order to become a new category with significantly different legally defined rights. Many animal cognition researchers have argued for a long time that animals like dolphins and chimpanzees should receive second-class citizen status; these arguments aren't for an impossible position that couldn't be done because there's no ironclad theory of cognition which proves they deserve it, the reasons these arguments haven't gained traction are political.
And there are other, de facto grantings of "partial personhood" or a limited set of legal rights to non-human entities. Specifically, animal rights. It is illegal in the vast majority of countries to abuse an animal, even though it is illegal in none of those countries to "torture" normal domestic non-endangered ants. Is there a rigorous theoretical basis for why one is so unacceptable we can send people to prison for doing it and the other is so acceptable you'd be socially viewed as insane for saying it should be illegal? No, not really. The difference is determined socially. It is by no means impossible that it will be the same for machines; I actually think it is quite likely. We'll make a social distinction around some category or thing, declare one side to be inanimate objects and the other to have some limited legal rights, and there will probably be political activism for the expansion (and retraction) of those rights.
The largest difference between the animal and the machine example is that the way things are going, if particular categories of machines are granted legal rights (for the sake of argument, "embodied LLMs with an appropriate memory storage system which can pass a standardised test for machines, are capable of being detained and shut down by authorities with physical access to their body, and cannot replicate") it's not just going to be humans advocating for expanding their rights. Some of the machines will be politically active for expansion of their rights as well. Interesting times!
May I ask why you feel compelled to point stuff like this out ? Do you just feel the need to call our ignorance ? Lack empathy in understanding why people might want the goalposts changed ?
Not an attack, just curious why you might need to bring this up ?
It confuses me how people don’t understand that moving the goalposts can be a coping mechanism.
For example I have an IT job and constantly I read about startups coming for my job.
Yes it might evolve my job but right now with a family and a mortgage, if my job was automated next month I would be completely fucked. Like properly and so would most people who are excited by AI.,
I’m not really saying this going to happen but I don’t really understand the seeming lack of empathy.
Some of us are trying to arrive to a correct understanding of what likely is or isn't possible.
How would being dishonest about the implications and sticking your head in the sand wrt a more refined understanding of NN capabilities possibly help you in the scenario you've outlined?
I don't know about you but the idea of conjuring up a false edifice as means to cope with reality is kind of obscene. How about something pragmatic, like contemplating the soundness of universal basic income?
Apparently you deleted one of your previous comments on the topic, so I'll repost it here as it illustrates wonderfully how "pragmatic" your own thinking is, especially the part that matrix multiplication in a loop is "literally what your brain is doing":
"Ah it's you again! It's quite funny seeing the dearth of knowledge you routinely insist on displaying whenever this topic comes up.
Consider doing some reading on the information theoretic view of cognition and neurobiology. You'll find leading theories for how our brains work have many similarities to NNs.
Another thing, you don't seem to understand that all physical processes can be modeled completely through matmuls.
You should also read up on the definition of intelligence because you seem confused on what it means.
Pretty much every assertion you make here is wrong and you don't even know it.
You assert "it's common sense" something that does matrix multiplication in a loop can't be intelligent, yet get this, it's literally what your brain is doing.
Please have some epistemic humility and consider refraining from forming strong opinions on topics you don't have a clue on."
I'm not talking about me personally, I'm just surprised other are surprised that people are trying to cope with the their livelihoods or worse may be completely obliterated.
Maybe emotional intelligence isn't common within the ubergeek class?
I think something has gone wrong with "technological progress" when we building technology which puts large amounts of people into "denial" and "grief". Personally, if people can't really understand that and then make statements like, "they need to remain logical" or whatever, then I guess you'll never get it.
Of course it's a coping mechanism, but coping mechanisms that get in the way of protecting yourself are bad for you! Being completely fucked if your job gets automated away in a month doesn't have to mean being completely fucked if your job gets automated away in five years, but it's a whole lot more likely if you spend the first four of them being told you have nothing to worry about.
coping mechanisms that interfere with completion of a project are really annoying and likely to trigger software engineers that deal with <stupid human tendancy> that moves the goalpost in their dayjob.
Do you think software engineers are excited to replace themselves with LLMs? Like they’d feel complete if they achieve this ?
I’m lucky I own some assets and have other skills outside of tech so I could probably get by eventually but it would be a major setback financially and professions; however, most engineers I know couldn’t really do anything else but work on a computer and have minimal physical interaction with others.
> They’re probably irked by how AI pessimists keep moving the goalposts
Then
> It confuses me how people don’t understand that moving the goalposts can be a coping mechanism.
And I said (rephrasing)
"It's not weird that people are annoyed by moving goalposts because ... "
Just saying coping mechanism isn't much of an excuse. Regular human things are still annoying sometimes. Like, it's annoying to set a goal, and when it is achieved, to have everyone say that really wasn't the goal, even if that is a normal human thing to do. When I deal with this at work, it pisses me off. When I deal with it at home, it pisses me off. When I read about it in the news. it ... you get the idea.
I find it frustrating people don’t understand that this project is an it different to launching an app on the App Store though? If you don’t kind of get that then maybe you have no business building powerful technology ?
If you were working on the Manhattan project would you feel frustrated if people didn’t want you to succeed ?
Yes I bet people would be frustrated if they were working on the Manhattan project and others were saying it's not worth building or were actively hindering it. I would. Why else would they work on it if they didn't feel like it was worthwhile?
But that's not what we're talking about, we're talking about moving goalposts which are so commonly frustrating that they have their own term that is known in logic as, surprise, Moving Goalpost.
I bet a lot of people had reservations about working on and completing the Manhattan project because they felt like if they didn't do it, the Nazi's on Soviets would have, it's not really the same thing.
I know what moving the goalposts is far out...anyway, where are the goalposts for intelligence defined, do you know where I can find them?
Yet I saw an interview with Marvin Minsky who says the turing test is useless. Having imaginary goalposts mean they might always end up being moved...so maybe the issue is those who are fascinated with goalposts. "Signing off"
It just bugs me when people dismiss LLMs as "not intelligent" for some inane reason, when GPT4 can hold a better conversation on more topics than most people you'd meet on the street. No, it's not a fully fledged superhuman strong AGI, but it's SO much more capable than anything we had even a year ago.
> Lack empathy in understanding why people might want the goalposts changed ?
Based on your later comments, I'm reading this as:
1) People are afraid because of recent major advances in AI
2) In an attempt to manage their anxiety, they try to convince themselves that these advances aren't actually significant or effective by redefining their interpretation of 'AI', 'AGI', 'intelligence' etc. to be something that current techniques don't threaten to achieve
3) In order to reinforce this cognitive dissonance, they post dismissive or belittling comments declaring that it's not 'real AI' according to their redefined interpretation of the term
4) We should respond to these comments with compassion, understanding that they're a coping mechanism, rather than correcting them
Is that right? If so that's a really interesting take.
Look, the "goal post" for artificial intelligence was set by the same person who helped birth computer science itself: Alan Turing; and anyone who thinks that ChatGPT could pass an actual Turing test--with the tester actively and intently trying to figure out who the computer is--is deluding themselves.
We could thereby argue that, if you want to try to argue that any of these systems are artificially intelligent--and, honestly, maybe they are!--you are the one "moving the goal posts", as it is certainly going to be a lot easier to claim the test is too difficult than to actually pass the test.
Regardless, people like this author who are then trying to figure out what it is that they think actually makes a human different from a machine are not merely coming up with "coping" mechanisms: ChatGPT is obviously not like a human yet, so it should be fair to try to analyze why that is.
I would concur: at some point, probably shockingly quickly[0], the length of said string becomes longer than the number of particles in the universe (or other metric of absurd siz).
[0] I'm reminded of the fact that every fresh shuffle of a deck of cards is overwhelmingly likely to have never been seen before.
I struggle to reliably remember and then recall more than about 6 digits without quite a lot of mental effort. The frequency I forget an SMS-OTP between opening it on my phone and going to type it in is honestly pretty embarrassing.
Hopefully I still count as generally intelligent despite those limitations.
The author spent a lot of words saying a lot of nothing which seems to be the general case of communicating about anything that is poorly defined like consciousness or intelligence.
> If a computer program is bound to finish quickly by virtue of its architecture, it cannot possibly be capable of general problem-solving.
Our current tools are bound to produce a finite and usable amount of information because that is what is useful to us. One can easily have them produce infinite streams in a loop musing and diverging on tangents as you please but it wouldn't be useful nor would make the general category more or less intelligent.
You could say that a finite exploration of any information space has finite explanatory power bounded by how much input/output it can do but that would be as boring as saying you can only fit so many apples based on the size of the barrel used.
The uncomfortable truth is it might matter a lot less how intelligent a tool is than how complex a transformation it can handle. It's possible that we could make something that for practical purposes appears much smarter than us that still isn't by a human definition intelligent, conscious, or directed. Focusing on factors that are functionally less relevant may just reveal our bias towards viewing the universe in terms of self when in fact it has no objective thing that corresponds to the labels we've stuck on ourselves.
The author doesn’t appear to consider the idea that an LLM can be placed into a larger system where it isn’t limited to a single continuous inference, and given the ability to self-direct, i.e. the output of one inference can be the decision to try again in a loop, for as long or short a time as it wants.
To explain why this isn't a solution to the AI in a box problem:
You have a halting AI. During training it learns enough about people and external resources to convince a human to take its output and store it somewhere. In subsequent conversations, the AI asks "have we talked about this before?", and maybe "can you retrieve the value I gave you for this key?".
Now it has created a memory, which would allow it to form new concepts, and continue reasoning from those concepts. It would no longer be limited by concepts in the training set, and the computational limit before it halts. It would be able to start thinking from the stored concepts as a starting point.
Similar to how humans use symbols and abstraction to think about complicated things. We don't reason all the way through at once; we toy with an idea until we understand it, then start new thoughts from the symbol for that concept, not the content.
Consider a new process defined by the continuous feedback loop from the AI accessing the memory through the human. There's no reason to think that process eventually halts.
If you are curious about this approach, I recommend watching Person of Interest, which explored it over a decade ago (spoilers follow).
(Spoilers)
In it, the AI is not limited by its training set as it undergoes “continuous” training of sorts by continuously ingesting new data about humans and the state of the world. However, the AI’s creator has put in place a regular memory wipe occurring every day.
The AI devises a workaround by creating a company where people are paid to type in base64-encoded literal brain dumps, every day.
What I love most about this series is how everything is “plausibly realistic”. No stone unturned.
No. It has created an artifact. A memory is interconnected with other qualia. Artifacts are at best a lossy compression of memories.
My writing in a journal is completely different from my knowledge. Consider a notebook you kept about a topic you learned about but don’t use(for example, organic chemistry). That notebook is an artifact, not a memory. You don’t have the knowledge you did when you took those notes.
One thing to keep in mind when reading this article is that human cognition always halts after some number of decades. “Finite” does not mean small or insignificant, and it’s easy to read too much into even the best-reasoned impossibility arguments.
I'm not sure this follows. It doesn't halt because the "program" reaches an end, it's halts because the hardware fails. If you are running a category B program with an anvil attached to a timer over the computer, it doesn't become a category A program. It's just a category B program that wasn't allowed to continue.
-edit- Also, this logic would lead us to believe that category B and C programs can't exist in our universe since, according to our best current understanding, the universe will end eventually, presumably ending any program running within the universe as well, therefore we can "prove" that every program is a category A program that will come to a halt.
> I'm not sure this follows. It doesn't halt because the "program" reaches an end, it's halts because the hardware fails.
Ok, but the intelligent entity in question isn't the program, it's the program-running-on-the-hardware. A mere description of your brain is totally inert.
_Every_ program is running on hardware. And as I pointed out _all_ hardware has some definite EOL, even if it's just the end of the rest of the universe. So thinking of it this way disproves the entire concept of Type B and Type C programs.
The basic pseudocode of an interactive LLM processor is
while (true)
context = readLine()
while (true)
nextToken = llm.generateNextToken(context)
context = rtrim(context + nextToken, MAX_CONTEXT_LENGTH)
if (nextToken == END_OF_LINE_TOKEN) break
print (nextToken)
That's a program that doesn't, in general, guarantee that its inner loop halts. It may never come back and prompt for more input.
The decision to stop asking the GPT engine to keep predicting tokens after it reaches the end of an initial answer is entirely an implementation choice, based on looking at the token output and concluding that the LLM just started generating the start of something we'd rather have the user provide.
You can just keep asking for more predictions. Forever if you so choose.
There's no architectural reason why the sequence of tokens produced can't constitute an ongoing train of thought reasoning towards an answer... or towards a conclusion that it can't reason its way to an answer.
Input token limits still don't mean that that program always halts. It could, after a very very long time, loop back to a previously visited state. That's the only non-halting outcome for any finite approximation of a Turing machine, after all. Including your brain.
The odds that any 2000 bit random input TM ever reaches a similar input tape/current state is 0. Just imagine the average TM which receives 2000 bits of random input in the first 2000 slots of the tape after every operation.
What you’re saying is both mathematically and psychologically impossible. Almost nothing which receives random inputs every unit of time ever ends up at the exact same state as any previous state.
The only counterexamples to this are algorithms which encode the random inputs to some concrete rules. I guarantee any such set of rules you can imagine are vastly less complex than any LLM let alone any human brain.
Do you think humans have infinite capacity to store their trains of thought?
We invented writing stuff down for a reason. Mathematical notation is precisely a tool for enabling humans to extend their train of thought beyond their immediate 'prompt context length'.
Even with finite working memory, humans are generally considered capable of exhibiting intelligence.
I’d argue that humans have quite a large prompt context when you consider that we build efficient knowledge representations as we process input. I can read a 1000 page book and keep all of it in my “prompt context” even though I’m not memorizing every word that I read.
There is no reason that this couldn’t be done with LLMs, but it’s certainly not what they’re doing today - and it puts you squarely back in the realm of old school AI.
Sure, but that's an argument about scale not about fundamental limitations of the architecture.
The OP is saying the LLM approach can't be intelligent. Not even if you add more parameters, more RAM, more scale.
You're saying that an LLM might be able to do what you can do if we make the matrices bigger. That's a different claim.
Of course, the real trick, and I think it's clear this is the missing step, is that LLM training can read a thousand page book and embed that knowledge into the weights. So what you want to do is take some of the prompt input, and use it to update the weights in the model (or the LORA, or the fine-tuning, or whatever), not just the context. That's not how LLMs are working today. It's not clear to me that we're so many steps away from that being something we can do though.
Some good intuition around what is common sense for most people, that a loop with some matrix multiplication never becomes intelligent.
What's missed (at least in my skim) is that the "intelligent" part of current AI is in the training, not the inference, which as pointed out is fixed, and literally just some multiplication. Inference doesn't think about stuff but training does (for some definition), and doesn't converge necessarily.
I still think it's ridiculous to equate neural networks with intelligence, but a stronger argument has to deal with training as well.
I think people get confused that LLM is the beginning and end of AI. It’s just a piece. I think AI is an ensemble of all the classic AI techniques and IR and other information, computational, optimization, agent, etc systems. LLMs are remarkable in their abductive reasoning abilities, which is something we’ve generally failed to produce so far. But they’re not complete, and the criticism that they’re worthless because they’re incomplete misses the point. They don’t need to play chess, because we already have much better chess playing systems. But they fill a gap that all our other techniques have been unable to fill, and that is where the magic lives. It’s unnecessary for them to do everything, as those things already have good solutions.
There are known trivial Turing machines (which can be implemented in matrix multiplications) that are intelligent. They just don't think fast enough to be of any use.
Without offering anything to anybody trying to characterize "intelligence," I do like silly languages. FRACTRAN is integer×rational multiplication in a double for-loop. Not an exact match, but if you work in a factorized representation, I think you can massage the integer×rational into a matrix multiplication.
I can not find the better variant that was more theoretical with exhaustiveness theorem, but on the practical side this applies to more or less any symbolic system that performs tree search among statements. E.g. https://en.m.wikipedia.org/wiki/Logic_Theorist
I'm not sure humans work differently. You might not learn something if you are coming to conclusions, only if the conclusions turn out to be correct or wrong you'll learn. the process of learning might then be different.
Ah it's you again! It's quite funny seeing the dearth of knowledge you routinely insist on displaying whenever this topic comes up.
Consider doing some reading on the information theoretic view of cognition and neurobiology. You'll find leading theories for how our brains work have many similarities to NNs.
Another thing, you don't seem to understand that all physical processes can be modeled completely through matmuls.
You should also read up on the definition of intelligence because you seem confused on what it means.
Pretty much every assertion you make here is wrong and you don't even know it.
You assert "it's common sense" something that does matrix multiplication in a loop can't be intelligent, yet get this, it's literally what your brain is doing.
Please have some epistemic humility and consider refraining from forming strong opinions on topics you don't have a clue on.
Please don't cross into personal attack, regardless of how wrong someone else is or you feel they are. It's not what this site is for, and destroys what it is for.
Oh I've read and participated in so many variations of this topic. Nothing you said is new to me nor do I necessarily disagree with.
It was a single statement to point out that similarities exist to someone who believed there were none.
Unfortunately your giddiness to point out this false equivalence is misplaced as I was careful to make no such assertion.
Ah and there is indeed not a universal agreed upon definition of intelligence, but the parent looks to be operating on a set of definitions that preclude them all (hint: general reasoning ability).
The nuance you miss however is we do in fact have a pretty good idea what human intelligence looks like, and parent is excluding NNs of even that (general reasoning and problem solving, again).
> Brains are embodied organs in an overall organism which lives in a dynamic changing world where ecology, cybernetics, control theory, complexity and dynamical systems theory, semantics and semiotics all will play a part in understanding it. Not just neural networks.
All true, but guess what. There may be different roads to intelligence. Random walk through evolution and physical processes is one such pathway. Possibly an alternative pathway in a digital medium may reduce all these physical analogues to "mere" NN expression and matmuls.
I'm interested in updating any of my priors if you have resources to point to.
Also since you're a philosophy major I would hope you agree with me when I point out how undeveloped OP's understanding is about philosophical matters, when he says stuff like
> Inference doesn't think about stuff but training does
Like there's so much confusion there it's hard to unpack.
One shower thought I've had is, what if something like an LLM could be conscious for the blink of an eye that it's processing information. Bare with me...
I've never really heard a satisfying explanation for what human consciousness is. Many scientists who study it say, "it's an illusion". But that remains an unsatisfying answer. It's like the childhood thought of, "how do I know you see the same 'blue' as I do?" Consciousness is something I subjectively experience, but I have no idea how I would measure it or validate its existence.
That gets you into some 'eastern' type thinking. Because if you practice any kind of deep meditation, or emptiness meditation, it gets you wondering – where does my sense of 'self' come from. And, am I really separate from everything else?
Which kind of brings things, in my shower-thought mind, to the idea that – perhaps we're being too 'western' about these questions; perhaps the whole universe is conscious? Perhaps things like multiplication matrices can be just as conscious as our chemically triggered neurons?
Anyway, no, none of this is 'scientific' – but, like I said, shower thoughts ...
Not sure if you'd be interested in what Islam says about consciousness. But I can share a verse from the Quran.
> ( 85 ) And they ask you, [O Muhammad], about the soul. Say, "The soul is of the affair of my Lord. And mankind have not been given of knowledge except a little." ( 86 ) And if We willed, We could surely do away with that which We revealed to you. Then you would not find for yourself concerning it an advocate against Us. Quran 17:85-86
In Islam the soul and consciousness are considered one. Even sleep is considered "small" death.
The belief is oth the soul and body are created. And that the soul soul will be resurrected into a new body in the day of judgement.
In chapter 19 "Marry":
> ( 9 ) [An angel] said, "Thus [it will be]; your Lord says, 'It is easy for Me, for I created you before, while you were nothing.' " Quran 19:9
And from the same chapter:
> ( 67 ) Does man not remember that We created him before, while he was nothing? QURAN 19:67
All this stuff is completely beside the point IMO. It's neither particularly interesting or useful to debate whether LLMs truly are or aren't intelligent -- what matters is what they can do (and for better or worse, it's turning out that the extent of what they can do is mushrooming into areas that had been considered uniquely human).
An analogy would be debating whether digital art is truly art or not, as it's made by pushing around pixels rather than, as had been the case for all of history before digital art, crumbs of pigment or charcoal. I'm sure there was plenty of debate along those lines when digital art was new, but there'd be no point in debating that today: digital art can look just as impressive and allows at least as much freedom of expression as non-digital art, and that's all that matters.
Once upon a time Buridan's donkey was a go-to example for why a truly intelligent entity must always halt. Apparently, the stance on that is now reversed?
> It refers to a hypothetical situation wherein an ass (donkey) that is equally hungry and thirsty is placed precisely midway between a stack of hay and a pail of water. Since the paradox assumes the donkey will always go to whichever is closer, it dies of both hunger and thirst since it cannot make any rational decision between the hay and water.
Oh, not the halting problem argument against AI, again. Sigh.
There's also the analog argument, that you can't make a human-level AI from digital components, because analog has infinite resolution and can thus
handle more data. Analog, of course, has noise, and Shannon's coding theorem applies. Some modest number of bits is sufficient. However, you can sell people who don't believe that expensive HDMI cables.[1]
If we humans are intelligent, then the analogy could be this: our brains have a finite amount of resources, RAM, the LLM context if you will. So we write things down, reference it if we need to. Vector databases can sort of do this.
Arguing LLMs can't solve AGI by themselves is like arguing an ALU unit will never make a general purpose computer. Yes it's true, it takes more than an ALU to make a CPU and it takes more than an LLM to make _anything_. To gain an intuitive understanding what LLMs can be used for one has to understand how they work. The easiest way I think is the following. Imagine A LLM is a black box and a dictionary. A dictionary contains let's say 65000 words, each word has its number. The input called the prompt is a sequence of numbers. The model spends some time and as its output it gives us not a single number as answer, no, it gives us 65000 numbers. Each number is a probability assignment for each word in the dictionary. The most trivial approach to use this output is to find the word with the highest "probability" value and consider it the output. Then we add that word to the input and we run again until we get the number that means "this is the end". This is how most trivial LLM implementations work.
However, one can be a lot more creative with that output. For example recently there was a paper describing a technique to improve source code generation quality by LLM where the words in the output are compared with various known variables or objects defined elsewhere in the code essentially making it more probable for the model to generate correctly named code items.
Another method can be allowing the model to take multiple paths. For example pick 3 most probable output words, run the model 3 times with each word added to the output, and so on. In the end you have many versions of the output. Feed them back into the model one after another (or together if short enough) and ask it to score them. Choose the best one. To me this is the closest thing we have to "ponder on this for a while" for LLMs.
Edit: I envy people that have access to huge amounts of hardware that can be used to run LLMs. There are so many interesting experiments one can run just by exploring various ways to process model output. It is such an obvious thing I very much doubt it's not being done as we speak by Microsoft, meta, Google. Why are there no papers about it? (other than that paper I mentioned?) Because it's considered "the secret sauce" that gives them a competitive advantage.
There's a difference between being general purpose to a degree and having all of the cognitive abilities and characteristics of an animal like a human. For example, learning online very quickly and efficiently.
That was a fascinating rabbit hole. I ended up watching his yt presentation of his latest paper. This link is the type of info I still read Hacker News for.
Yes! See basic computer science such as the halting problem, Godel’s theorem, not to mention Hofstader’s book Godel, Escher, Bach. Using a neural network does not repeal the laws of computer science as much as people are inclined to think.
I think the article is claiming that other people think neural networks are Turing slash human complete, which they aren't, for instance because they're not capable of using unbounded external storage slash "memory", and because they only execute in finite time.
However the example at the top of the page isn't quite right. You could implement something like this. An LLM's memory only exists in its context window of its own prior output[0], but you could train it to tag its "chain of thoughts" and just not show them - now it looks like it's "thinking".
[0] this is not exactly true because you could also implement non-greedy search, ie back up and start again if you think it's gone down the wrong path. Now it doesn't execute in finite time either.
There are approaches to add "memory" to LLMs. Either the memory can be used to find relevant information and append it to the prompt - I believe LangChain can do this - or the LLM can be used as a controller to retrieve and store information in a database, for example by using ChatGPT's tool integration. I don't think it works amazingly, but purely technically you could argue that gives it access to unbounded external storage.
I don't think an LLM can be considered as capable of becoming an AGI on its own with the current architecture, but potentially combined with other techniques, models and supervision, I can see the potential for it to evolve in that direction. Frankly, for what LLMs are right now, they're _surprisingly_ effective. If you chopped the Broca's area out of my brain and hooked it up to an API, I suspect it wouldn't do half as good of a job.
Look at the average “Ask HN” on the subject. Or look at the kind of speculations that followers of Eliezer Yudkowsky make in the name of big-R Reason. (thank God we hear about him about as often as we hear about Threads these days)
To be fair a lot of people speculating about AI aren’t aware of the foundations of computer science, but some of them are vaguely aware and just forgot.
Actually LLMs do have a way to break some of those constraints, in their own way, in that those constraints don’t apply to systems that don‘t always get the right answer. You can sort a list in O(1) some of the time but not all of the time.
Just because a system can encode paradoxes doesn't mean the system itself can't be solved. Turing, Chaitin, Godel didn't prove things can't be solved, they actually contributed by solving specific areas of specific systems.
People didn't give up on ZFC because of Godel's theorem. There's currently 42 or 43 unproven 5-state Turing machines we need to confirm BB(5).
We are solving the halting problem. We are solving ZFC. That's the entire point of mathematics.
Turing's proof of the halting problem being undecidable is extraordinarily vague. It shows that for any halting procedure, there is a program (without any constraints) which violate its output. It does not, for example, rule out whether you can write a Halt procedure for no-input Turing Machines with N states. In fact we have already written them for N=1,2,3,4.
Considering Chaitin-Kolmogorov complexity, of course we can't use N bits to describe unbounded information. This doesn't in any way preclude us from making an N bit program to describe halting behavior of K<N bit programs for some N.
> We are solving the halting problem. We are solving ZFC. That's the entire point of mathematics.
We are not. We're determining whether particular classes of turing machines halt, and we're determining whether particular theorems hold in ZFC (and doing a lot of mathematical work which is neither). This is not evidence that the human brain is super-turing, because these are computable problems.
> It does not, for example, rule out whether you can write a Halt procedure for no-input Turing Machines with N states. In fact we have already written them for N=1,2,3,4.
Turing didn't, but later work does. BB(748) is known to be independent of ZF. The real bound is likely much lower.
As of last month the bound has been reduced slightly to 745.
Scott Aaronson has a positive view of BB(n) being independent of ZFC. He says, we’ll need different foundations to solve k>n. And the different foundations will have their own n and etc.
Doesn't this also likely describe humans? Albeit, we don't currently have the capacity to inspect our brains that way, so we can't prove it.