What proof do you have that human reasoning involves "symbolic logic and abstractions"? In daily life, that is, not in a math exam. We know that people are actually quite bad at reasoning [1][2]. And it definitely doesn't seem right to define "reasoning" as only the sort that involves formal logic.
Some very intelligent people, including Gödel and Penrose, seem to think that humans have some kind of ability to arrive directly on correct propositions in ways that bypass the incompleteness theorem. Penrose seems to think this can be due to Quantum Mechanics, Göder may have thought it came frome something divine.
While I think they're both wrong, a lot of people seem to think they can do abstract reasoning for symbols or symbol-like structures without having to use formal logic for every step.
Personally, I think such beliefs about concepts like consciousness, free will, qualia and emotions emerge from how the human brain includes a simplified version of itself when setting up a world model. In fact, I think many such elements are pretty much hard coded (by our genes) into the machinery that human brains use to generate such world models.
Indeed, if this is true, concepts like consciousness, free will, various qualia and emotions can in fact be considered "symbols" within this world model. While the full reality of what happens in the brain when we exercise what we represent by "free will" may be very complex, the world model may assign a boolean to each action we (and others) perform, where the action is either grouped into "voluntary action" or "involuntary action".
This may not always be accurate, but it saves a lot of memory and compute costs for the brain when it tries to optimize for the future. This optimization can (and usually is) called "reasoning", even if the symbols have only an approximated correspondence with physical reality.
For instance, if in our world model somebody does something against us and we deem that it was done exercising "free will", we will be much more likely to punish them than if we categorize the action as "forced".
And on top of these basic concepts within our world model, we tend to add a lot more, also in symbol form, to enable us to use symbolic reasoning to support our interactions with the world.
> While I think they're both wrong, a lot of people seem to think they can do abstract reasoning for symbols or symbol-like structures without having to use formal logic for every step.
Huh.
I don't know bout incompleteness theorem, but I'd say it's pretty obvious (both in introspection and in observation of others) that people don't naturally use formal logic for anything, they only painstakingly emulate it when forced to.
If anything, "next token prediction" seems much closer to how human thinking works than anything even remotely formal or symbolic that was proposed before.
As for hardcoding things in world models, one thing that LLMs do conclusively prove is that you can create a coherent system capable of encoding and working with meaning of concepts without providing anything that looks like explicit "meaning". Meaning is not inherent to a term, or a concept expressed by that term - it exists in the relationships between an the concept, and all other concepts.
> I don't know bout incompleteness theorem, but I'd say it's pretty obvious (both in introspection and in observation of others) that people don't naturally use formal logic for anything, they only painstakingly emulate it when forced to.
Indeed, this is one reason why I assert that Wittgenstein was wrong about the nature of human thought when writing:
"""If there were a verb meaning "to believe falsely," it would not have any significant first person, present indicative."""
Sure, it's logically incoherent for us to have such a word, but there's what seems like several different ways for us to hold contradictory and incoherent beliefs within our minds.
... but I'd say it's pretty obvious (both in introspection and in observation of others) that people don't naturally use formal logic for anything ...
Yes. But some place too much confidence in how "rational" their intuition is, including some of the most intelligent minds the world has seen.
Specifically, many operate as if their intuition (that they treat as completely rational) has some kind of supernatural/magic/divine origin, including many who (imo) SHOULD know better.
While I think (like you do) that this intuition has a lot in common with LLM's and other NN architectures than pure logic, or even the scientific method.
> Some very intelligent people, including Gödel and Penrose, seem to think that humans have some kind of ability to arrive directly on correct propositions in ways that bypass the incompleteness theorem. Penrose seems to think this can be due to Quantum Mechanics, Göder may have thought it came frome something divine.
Did Gödel really say this? It sounds like quite a stretch of incompleteness theorem.
It's like saying because halting problem is undecidable, but humans can debug programs, therefore human brains must having some supernatural power.
Gödel mostly cared about mathematics. And he seems to have believed that human intuition could "know" propositions to be true, even if they could not be proven logically[1].
It appears that he was religious and probably believed in an immaterial and maybe even divine soul [2]. If so, that may explain why he believed that human intuition could be unburdend by the incompleteness theorem.
[1] https://philsci-archive.pitt.edu/9154/1/Nesher_Godel_on_Trut...
[2] https://en.wikipedia.org/wiki/G%C3%B6del%27s_ontological_pro...