I would argue that Lenat was at least directionally correct in understanding that sheer volume of data (in Cyc's case, rules and facts) was the key in eventually achieving useful intelligence. I have to confess that I once criticized the Cyc project for creating an ever-larger pile of sh*t and expecting a pony to emerge, but that's sort of what has happened with LLMs.
I suspect at some point the pendulum will again swing back the other way and symbolic approaches will have some kind of breakthrough and become trendy again. And, I bet it will likely have something to do with accelerating these systems with hardware, much like GPUs have done for neural networks, in order to crunch really large quantities of facts
The Bitter Lesson has a few things to say about this.
The Bitter Lesson says "general methods that leverage computation are ultimately the most effective". That doesn't seem to rule out symbolic approaches. It does rule out anything which relies on having humans in the loop, because terabytes of data plus a dumb learning process works better than megabytes of data plus expert instruction.
(I know your message wasn't claiming that The Bitter Lesson was explicitly a counterpoint, I just thought it was interesting.)
Imho, this is wrong. Even independent of access to vast amounts of compute, symbolic methods seem to consistently underperform statistical/numerical ones across a wide variety of domains. I can't help but think that there's more to it than just brute force.
I've lost count how many times I've written the same words in this thread but: SAT Solving, Automated Theorem Proving, Program Verification and Model Checking, Planning and Scheduling. These are not domains where symbolic methods "consistently underperform" anything.
You guys really need to look into what's been going on in classical AI in the last 20-30 years. There are two large conferences that are mainly about symbolic AI, IJCAI and AAAI. Then there's all the individual conferences on the above sub-fields, like the International Conference on Automated Planning and Scheduling (ICAPS). Don't expect to hear about symbolic AI on social media or press releases from Alpha and Meta, but there's plenty of material online if you're interested.
Real AGI will need a way to reason about factual knowledge. An ontology is a useful framework for establishing facts without inferring them from messy human language.
These guys are trying to combine symbolic reasoning with LLMs somehow: https://www.symbolica.ai/
Or maybe program synthesis combined by LLMs might be the way?
It does seem like the Cyc people hit the wall with simply collecting facts. Having to have a human in the loop.
The problem I think is if you have LLMs figuring out the propositions, the whole system is just as prone to garbage-in-garbage-out as LLMs are.
But
a) The pile of LLM training data is vastly larger. b) The data is actual human utterances in situ--these are ponies, not pony shit. c) LLMs have no intelligence ... they channel the intelligence of a vast number of humans by pattern matching their utterances to a query. This has indeed proved useful because of how extremely well the statistical apparatus works, but the fact that LLMs have no cognitive states puts great limits on what this technology can achieve.
With Cyc, OTOH, it's not even clear what you can get out of it. The thing may well be useful if combined with LLMs, but it's under lock and key.
The big conclusions about symbolic AI that the author reaches based on this one system and approach are unwarranted. As he himself notes, "Even Ernest Davis and Gary Marcus, highly sympathetic to the symbolic approach to AI, found little evidence for the success of Cyc, not because Cyc had provably failed, but simply because there was too little evidence in any direction, success or failure."
>> they channel the intelligence of a vast number of humans by pattern matching their utterances to a query.
Just a little problem with that: to understand the utterances of a vast number of humans you need to channel it to something that can understand the utterances of humans in the first place. Just channeling it around from statistic to statistic doesn't do the trick.
Um, the "something" is the person reading the LLM's output. I'm afraid that you have completely missed the context and point of the discussion, which was not about LLMs understanding things--they understand nothing ("LLMs have no cognitive states"). But again, "because of how extremely well the statistical apparatus works", their outputs are useful to intelligent consumers who do have cognitive states--us.
That’s hilarious, but at least Llama was trained on libgen, an archive of most books and publications by humanity, no? Except for the ones which were not digitized I guess
So there is probably a big pile of Reddit comments, twitter messages, and libgen and arxiv PDFs I imagine
So there is some shit, but also painstakingly encoded knowledge (ie writing), and yeah it is miraculous that LLMs are right as often as they are
libgen is far from an archive of "most" books and publications, not even close.
The most recent numbers from libgen itself are 2.4 million non-fiction books and 80 million science journal articles. The Atlantic's database published in 2025 has 7.5 million books.[0] The publishing industry estimates that many books are published each year. As of 2010, Google counted over 129 million books[1]. At best an LLM like Llama will have have 20% of all books in its training set.
0. https://www.theatlantic.com/technology/archive/2025/03/libge...
1. https://booksearch.blogspot.com/2010/08/books-of-world-stand...
On libgen.mx they claim to have 33,569,200 books and 84,844,242 articles
Still an order of magnitude short of "all", and falling farther behind every year.
It's a miracle, but it's all thanks to the post-training. When you think of it, for so-called "next token predictors", LLMs talk in a way that almost no one actually talks, with perfect spelling and use of punctuation. The post-training somehow is able to get them to predict something along the lines of what a reasonably intelligent assistant with perfect grammar would say. LLMs are probably smarter than is exposed through their chat interface, since it's unlikely the post-training process is able to get them to impersonate the smartest character they'd be capable of impersonating.
I dunno I actually think say Claude AI SOUNDS smarter than it is, right now
It has a phenomenal recall. I just asked it about "SmartOS", something I knew about, vaguely, in ~2012, and it gave me a pretty darn good answer. On that particular subject, I think it probably gave a better answer than anyone I could e-mail, call, or text right now
It was significantly more informative than wikipedia - https://en.wikipedia.org/wiki/SmartOS
But I still find it easy to stump it and get it to hallucinate, which makes it seem dumb
It is like a person with good manners, and a lot of memory, and which is quite good at comparisons (although you have to verify, which is usually fine)
But I would not say it is "smart" at coming up with new ideas or anything
I do think a key point is that a "text calculator" is doing a lot of work ... i.e. summarization and comparison are extremely useful things. They can accelerate thinking
https://ai-2027.com/ postulates that a good enough LLM will rewrite itself using rules and facts... sci-fi, but so is chatting with a matrix multiplication.
I doubt it. The human mind is a probabilistic computer, at every level. There’s no set definition for what a chair is. It’s fuzzy. Some things are obviously in the category, and some are at the periphery of it. (Eg is a stool a chair? Is a log next to a campfire a chair? How about a tree stump in the woods? Etc). This kind of fuzzy reasoning is the rule, not the exception when it comes to human intuition.
There’s no way to use “rules and facts” to express concepts like “chair” or “grass”, or “face” or “justice” or really anything. Any project trying to use deterministic symbolic logic to represent the world fundamentally misunderstands cognition.
> There’s no set definition for what a chair is.
Sure there is: a chair is anything upon which I can comfortably sit without breaking it.
I find this very amusing. In philosophy of science some 20+ years ago I had a wonderful prof who went through 3(?) periods of thought. He laid out this argument, followed by the arguments seen below in this thread in various ways, in a systematic way where he convinced you that one way of thinking was correct, you took the midterm, then the next day he would lead with "everything you know is wrong, here's why.". It was beautiful.
He noted that this evolution of thought continued on until people generally argued that concepts/definitions that let you do meaningful things (your definition of meaningful, doesn't really matter what it is), are the way to go. The punchline at the very end, which happened to be the last thing I regurgitated on my last undergraduate exam, was him saying something along the lines of "Science, it beats hanging out in malls."
All this to say that if we read a little philosophy of science, that was done a long time ago (way before the class I took), things would make more sense.
I have definitely broken chairs upon sitting in them, which someone else could have sat in just fine. So it's unclear why something particular to me would change the chair-ness of an object.
Similarly, I've sat in some very uncomfortable chairs. In fact, I'd say the average chair is not a particularly comfortable one.
For a micro-moment before giving in it was a chair, then it broke. Now its no longer a chair. Its a broken chair.
That's not one, but two particularities that aren't latent to the chair itself: me (the sitter), and time.
Do you really have a personal ontology that requires you to ask the tense and person acting on a thing to know what that thing is? I suspect you don't; most people don't, because it would imply that the chair wouldn't be a chair if nobody sat on it.
A stump isn't a chair until someone decides to sit on it, at that point it becomes chair _to_ that person. Chair is only capable of acting as "chair" object if constraints are met in regards to sitter.
This is very complicated, because it now implies:
1. I can intend to sit on a chair but fail, in which case it isn't a chair (and I didn't intend to sit on it?)
2. I can intend to have my dog sit on my chair, but my dog isn't a person and so my chair isn't a chair.
This is-use distinction you're making is fine; most people have an intuition that things "act" as a thing in relation to how they're used. But to take it a step forwards and claim that a thing isn't its nature until a person sublimates their intent towards it is very unintuitive!
(In my mind, the answer is a lot simpler: a stump isn't a chair, but it's in the family network of things that are sittable, just like chairs and horses. Or to borrow Wittgenstein, a stump bears a family resemblance to a chair.)
I'm the person who asked about the definition of a chair up thread.
Just to make a very obvious point: Nobody thinks of the definition for a chair as a particularly controversial idea. But clearly:
- We don't all agree on what a chair is (is a stump a chair or not?).
- Nobody in this thread has been able to give a widely accepted definition of the word "chair"
- It seems like we can't even agree on what criteria are admissible in the definition. (Eg, does it matter that I can sit on it? Does it matter that I can intend to sit on it? Does it matter that my dog can sit on it?)
If even defining what the word "chair" means is beyond us, I hold little hope that we can ever manually explain the concept to a computer. Returning to my original point above, this is why I think expert systems style approaches are a dead end. Likewise, I think any AI system that uses formal or symbolic logic in its internal definitions will always be limited in its capacity.
And yet, I suspect chatgpt will understand all of the nuance in this conversation just fine. Like everyone else, I'm surprised how "smart" transformer based neural nets have become. But if anything has a hope of achieving AGI, I'm not surprised that:
- Its something that uses a fuzzy, non-symbolic logic internally.
- The "internal language" for its own thoughts is an emergent result of the training process rather than being explicitly and manually programmed in.
- That it translates its internal language of thought into words at the end of the thinking / inference process. Because - as this "chair" example shows - our internal definition for what a chair is is seems clear to us. But it doesn't necessarily mean we can translate that internal definition into a symbolic definition (ie with words).
I'm not convinced that current transformer architectures will get us all the way to AGI / ASI. But I think that to have a hope of achieving human level AI, you'll always want to build a system which has those elements of thought. Cyc, as far as I can tell, does not. So of course, I'm not at all surprised its being dumped.
What if it breaks in a way which renders it no longer a chair for you but not others?
This seems to imply that what is or is not a chair is a subjective or conditional.
> Sure there is: a chair is anything upon which I can comfortably sit without breaking it.
« It is often said that a disproportionate obsession with purely academic or abstract matters indicates a retreat from the problems of real life. However, most of the people engaged in such matters say that this attitude is based on three things: ignorance, stupidity, and nothing else.
Philosophers, for example, argue that they are very much concerned with the problems posed by real life.
Like, for instance, “what do we mean by real?”, and “how can we reach an empirical definition of life?”, and so on.
One definition of life, albeit not a particularly useful one, might run something like this: “Life is that property which a being will lose as a result of falling out of a cold and mysterious cave thirteen miles above ground level.”
This is not a useful definition, (A) because it could equally well refer to the subject’s glasses if he happens to be wearing them, and (B) because it fails to take into account the possibility that the subject might happen to fall onto, say, the back of an extremely large passing bird.
The first of these flaws is due to sloppy thinking, but the second is understandable, because the mere idea is quite clearly, utterly ludicrous. »
— Douglas Adams
So a warm and smelly compost pile is a chair? A cold metal park bench is not a chair (because it's uncomfortable)?
A beanbag is a chair? Perhaps a chair should be something on which one can comfortably sit without breaking that has a back and four legs. I suppose then a horse would be a chair.
Is a mountain a chair?
So you're just ignoring all the probabilistic, fuzzy etc. Prologs etc. which do precisely that? https://github.com/lab-v2/pyreason
> Any project trying to use deterministic symbolic logic to represent the world fundamentally misunderstands cognition.
The counterposition to this is no more convincing: cognition is fuzzy, but it's not really clear at all that it's probabilistic: I don't look at a stump and ascertain its chairness with a confidence of 85%, for example. The actual meta-cognition of "can I sit on this thing" is more like "it looks sittable, and I can try to sit on it, but if it feels unstable then I shouldn't sit on it." In other words, a defeasible inference.
(There's an entire branch of symbolic logic that models fuzziness without probability: non-monotonic logic[1]. I don't think these get us to AGI either.)
Which word will I pick next in this sentence? Is it deterministic? I probably wouldn’t respond the same way if I wrote this comment in a different mood, or at a different time of day.
What I say is clearly not deterministic for you. You don’t know which word will come next. You have a probability distribution but that’s it. Banana.
I caught a plane yesterday. I knew there would be a plane (since I booked it) and I knew where it would go. Well, except it wasn’t certain. The flight could have been delayed or cancelled. I guess I knew there would be a plane with 90% certainty. I knew the plane would actually fly to my destination with a 98% certainty or something. (There could have been a malfunction midair). But the probability I made it home on time rose significantly when I saw the flight listed, on time, at the airport.
Who I sat next to was far less certain - I ended up sitting next to a 30 year old electrician with a sore neck.
My point is that there is so much reasoning we do all the time that is probabilistic in nature. We don’t even think about it. Other people in this thread are even talking about chairs breaking when you sit on them - every time you sit on a chair there’s a probability calculation you do to decide if the chair is safe, and will support your weight. This is all automatic.
Simple “fuzzy logic” isn’t enough because so many probabilities change as a result of other events. (If the plane is listed on the departures board, the prediction goes up!). All this needs to be modelled by our brains to reason in the world. And we make these calculations constantly with our subconscious. When you walk down the street, you notice who looks dangerous, who is likely to try and interact with you, and all sorts of things.
I think that expert systems - even with some fuzzy logic - are a bad approach because systems never capture all of this reasoning. It’s everywhere all the time. I’m typing on my phone. What is the chance I miss a letter? What is the chance autocorrect fixes each mistake I make? And so on, constantly and forever. Examples are everywhere.
To be clear, I agree that this is why expert systems fail. My point was only that non-monotonic logics and probability have equal explanatory power when it comes to unpredictability: the latter models with probability, and the former models with relations and defeasible defaults.
This is why I say the meta-cognitive explanation is important: I don’t think most people assign actual probabilities to events in their lives, and certainly not rigorous ones in any case. Instead, when people use words like “likely” and “unlikely,” they’re typically expressing a defeasible statement (“typically, a stranger who approaches me on the street is going to ask me for money, but if they’re wearing a suit they’re typically a Jehovah’s Witness instead”).
> I don’t think most people assign actual probabilities to events in their lives, and certainly not rigorous ones in any case.
Interesting. I don't think I agree.
I think people do assign actual probabilities to events. We just do it with a different part of our brain than the part which understands what numbers are. You can tell you do that by thinking through potential bets. For example, if someone (with no special knowledge) offered a 50/50 bet that your dining chair will break next time you sit on it, well, that sounds like a safe bet! Easy money! What about if the odds changed - so, if it breaks you give them $60, and if it doesn't break they give you $40? I'd still take that bet. What about 100-1 odds? 1000-1? There's some point where you start to say "no no, I don't want to take that bet." or even "I'd take that bet if we swap sides".
Somewhere in our minds, we hold an intuition around the probability of different events. But I think it takes a bit of work to turn that intuition into a specific number. We use that intuition for a lot of things - like, to calibrate how much surprise we feel when our expectation is violated. And to intuitively decide how much we should think through all the alternatives. If we place a bet on a coin flip, I'll think through what happens if the coin comes up heads or if it comes up tails. But if I walk into the kitchen, I don't think about the case that I accidentally stub my toe. My intuition assigns that a low enough probability that I don't think about it.
Talking about defeasible statements only scratches the surface of how complex our conditional probability reasoning is. In one sense, a transformer model is just that - an entire transformer based LLM is just a conditional probability reasoning system. The entire model of however many billions of parameters is all a big conditional probability reasoning machine who's only task is to figure out the probability distribution over the subsequent token in a stream. And 100bn parameter models are clearly still too small to hit the sweet spot. They keep getting smarter as we add more tokens. If you torture an LLM model a little, you can even get it to spit out exact probability predictions. Just like our human minds.
I think these kind of expert systems fail because they can't do the complex probability reasoning that transformer models do. (And if they could, it would be impossible to manually write out the - perhaps billions - of rules it would need to accurately reason about the world like chatgpt can.)
> I think people do assign actual probabilities to events. We just do it with a different part of our brain than the part which understands what numbers are. You can tell you do that by thinking through potential bets.
I think these are different things! I can definitely make myself think about probabilities, but that's a cognitive operation rather than a meta-cognitive one.
In other words: I think what you're describing as "a bit of work" around intuitions is our rationalization (i.e., quantification) of an process that's internally non-statistical, but defeasible instead. Defeasibility relationships can have priorities and staggerings, which we turn into fuzzy likelihoods when we express them.
My intuition for this comes from our inability to be confidently precise in our probabilistic rationalizations: I don't know about you, but I don't know whether I'm 57.1% or 57.01983% confident in an expression. I could make one up, but as you note with torturing the LLM, I'm doing it to "make progress," not because it's a true statement of probability.
(I think expert systems fail for a reason that's essentially not about probability reasoning, but dimensionality -- as the article mentions Cyc has at least 12 dimensions, but there's no reason to believe our thoughts have only or exactly these 12. There's also no reason to believe we can ever model the number of dimensions needed, given that adding dimensions to an encoded relation set is brutally exponential.)
>My intuition for this comes from our inability to be confidently precise in our probabilistic rationalizations: I don't know about you, but I don't know whether I'm 57.1% or 57.01983% confident in an expression.
LLMs are probabilistic and notoriously unable to be confidently precise in their probabilistic rationalizations.
> LLMs are probabilistic and notoriously unable to be confidently precise in their probabilistic rationalizations.
Sure. To tie these threads together: I think there are sufficient other different properties that make me reasonably confident that my thought process isn't like an LLM's.
(Humans are imprecise, LLMs are imprecise, thermometers are imprecise, but don't stick me or my computer in an oven, please.)
>Sure. To tie these threads together: I think there are sufficient other different properties that make me reasonably confident that my thought process isn't like an LLM's.
Doesn't have to be like an LLM's to be probabilistic
>I don't look at a stump and ascertain its chairness with a confidence of 85%
But i think you did. Not consciously, but i think your brain definitely did.
https://www.nature.com/articles/415429a https://pubmed.ncbi.nlm.nih.gov/8891655/
These papers don't appear to say that: the first one describes the behavior as statistically optimal, which is exactly what you'd expect for a sound set of defeasible relations.
Or intuitively: my ability to determine whether a bird flies or not is definitely going to be statistically optimal, but my underlying cognitive process is not itself inherently statistical: I could be looking at a penguin and remembering that birds fly by default except when they're penguins, and only then if the penguin isn't wearing a jetpack. That's a non-statistical set of relations, but its external observation is modeled statistically.
>which is exactly what you'd expect for a sound set of defeasible relations.
This is a leap. While a complex system of rules might coincidentally produce behavior that looks statistically optimal in some scenarios, the paper (Ernst & Banks) argues that the mechanism itself operates according to statistical principles (MLE), not just that the outcome happens to look that way.
Moreover, it's highly unlikely, bordering on impossible, to reduce the situations the brain deals with even on a daily basis into a set of defeasible statements.
Example: Recognizing a "Dog"
Defeasible Attempt: is_dog(X) :- has_four_legs(X), has_tail(X), barks(X), not is_cat(X), not is_fox(X), not is_robot_dog(X).
is_dog(X) :- has_four_legs(X), wags_tail(X), is_friendly_to_humans(X), not is_wolf(X).
How do you define barks(X) (what about whimpers, growls? What about a dog that doesn't bark?)? How do you handle breeds that look very different (Chihuahua vs. Great Dane)? How do you handle seeing only part of the animal? How do you represent the overall visual gestalt? The number of rules and exceptions quickly becomes vast and brittle.
Ultimately, the proof as they say, is in the pudding. By the way, the CyC we are all talking about here is non-monotonic. https://www.cyc.com/wp-content/uploads/2019/07/First-Orderiz...
If you've tried something for decades and it's not working, and it doesn't even look like it's working and experiments with the brain suggest probabilistic inference and probabilistic inference machines work much better than the alternatives ever did, you have to face the music.
> How do you define barks(X) (what about whimpers, growls? What about a dog that doesn't bark?)? How do you handle breeds that look very different (Chihuahua vs. Great Dane)? How do you handle seeing only part of the animal? How do you represent the overall visual gestalt? The number of rules and exceptions quickly becomes vast and brittle.
This is the dimensionality mentioned in the adjacent post, and it's true of a probabilistic approach as well: an LLM trained on descriptions of dogs is going to hallucinate when an otherwise sensible query about dogs doesn't match its training. As others have said more elegantly than I will, this points to a pretty different cognitive model than humans have; human beings can (and do) give up on a task.
(I feel like I've had to say this a few times in threads now: none of this is to imply that Cyc was a success or would have worked.)
>This is the dimensionality mentioned in the adjacent post
LLMs are only a few years old but symbolic ai was abandoned for NLP, Computer Vision etc long before that. Why ? Because the alternative was just that bad and more importantly, never seemed to really budge with effort. Companies didn't wake up one morning and pour hundreds of millions into LMs. In fact, NNs were the underdog for a very long time. They poured more and more money into it because it got better and better with investment.
There is zero reason to think even more dimensionality would do anything but waste even more time. At least the NN scalers can look back and see it work in the past. You don't even have that.
>an LLM trained on descriptions of dogs is going to hallucinate when an otherwise sensible query about dogs doesn't match its training. As others have said more elegantly than I will, this points to a pretty different cognitive model than humans have; human beings can (and do) give up on a task.
It doesn't take much to train LLMs to 'give up'. Open AI talk about this from time to time. It's just not very useful with a tendency to overcorrect. And humans hallucinate (and otherwise have weird failure modes) all the time. We just call them funny names like dunning kruger and optical illusions. Certainly less than current SOTA LLMs but it happens all the same.
>I feel like I've had to say this a few times in threads now: none of this is to imply that Cyc was a success or would have worked.
The point is not about Cyc. They're hardly the only attempt at non-monotonic logic. The point is that they should work much better than they do if there's anything to it. Again, forget recent LLMs. Even when we were doing 'humble' things like spelling error detection & correction, text compressors, voice transcription boosters, embeddings for information retrieval, recommenders, knowledge graph creation (ironically enough), machine translation services, etc these systems were not even in the conversation. They performed that poorly.
I think we're talking past each other. I'm not interested in defending symbolic AI at all; it's clear it's failed. All told, I would not say I'm particularly interested in any kind of AI.
I'm interested in theory of mind, and I think defeasibility with a huge number of dimensions is a stronger explanation of how humans behave and think than something resembling an LLM. This doesn't somehow mean that LLMs haven't "won" (they have); I just don't think they're winning at human-like cognition. This in turn does not mean we could build a better alternative, either.
> The human mind is a probabilistic computer, at every level.
Fair enough, but an airplane's wing is not very similar to a bird's wing.
That argument would hold a lot more weight if Cyc could fly. But as this article points out, decades of work and millions of dollars have utterly failed to get it off the ground.
right, but as others have pointed out the amount of money invested in Cyc is approx 2 orders of magnitude less than what was invested in LLMs. So maybe the method was OK, but it was insufficiently resourced.
Maybe, but I doubt it. Transformer models got all that investment because small transformers work and large transformers work better than small transformer models. We can extend the line and predict what happens when the model is scaled.
Small Cyc doesn't do anything useful. Large Cyc doesn't do anything useful. Why should we make huge Cyc? If we extend the line out, all evidence predicts that it'll still be useless.
> There’s no way to use “rules and facts” to express concepts like “chair” or “grass”, or “face” or “justice” or really anything. Any project trying to use deterministic symbolic logic to represent the world fundamentally misunderstands cognition.
Are you sure? In terms of theoretical foundations for AGI, AIXI is probabilistic but godel-machines are proof based and I think they'd meet criteria for deterministic / symbolic. Non-monotonic and temporal logics also exist, where chairness exists as a concept that might be revoked if 2 or more legs are missing. If you really want to get technical then by allowing logics with continuous time and changing discrete truth values, then you can probably manufacture a fuzzy logic where time isn't considered but truth/certainty values are continuous. Your ideas about logic might be too simple, it's more than just Aristotle
Not person you are replying to, just FYI.
I don't know, it all seems like language games to me. The meaning is never in its grammar, but in its usage. The usage is arbitrary and capricious. I've not discovered how more nuanced forms of logics have ever really grappled with this.
In the 1980s, we used to talk about the silliness of the “grandmother neuron” - the idea that one neuron would capture an important thing, rather than a distributed representation.
> This kind of fuzzy reasoning is the rule, not the exception when it comes to human intuition.
That is indeed true. But we do have classic fuzzy logic, and it can be used to answer these questions. E.g. a "stool" maybe a "chair", but "automobile" is definitely not.
Maybe the symbolic logic approach could work if it's connected with ML? Maybe we can use a neural network to plot a path in the sea of assertions? Cyc really seems like something that can benefit the world if it's made open under some reasonable conditions.
> That is indeed true. But we do have classic fuzzy logic, and it can be used to answer these questions. E.g. a "stool" maybe a "chair", but "automobile" is definitely not.
I’m not convinced that classical fuzzy logic will ever solve this - at least not if every concept needs to be explicitly programmed in. What a “chair” is sort of subtly changes at a furniture store and at a campsite. Are you going to have someone explicitly, manually program all of those nuances in? No way! And without that subtlety, you aren’t going to end up with a system that’s as smart as chatgpt. Challenge me on this if you like, but we can play this game with just about any word you can name - more or less everything except for pure mathematics.
And by the way, modern ML approaches understand all of those nuances just fine. It’s not clear to me what value - if any - symbolic logic / expert systems provide that chatgpt isn’t perfectly capable of learning on its own already.
"The human mind is a probabilistic computer, at every level."
We don't know that. It's mostly probabilistic. That innate behavior exists suggests some parts might be deterministic.
Words are used due to the absence of things. They fill an immediate experiential void and stand in for something else, because you want or need another person to evoke some fantasy to fill this absence and make understanding possible.
If you have a mind and it is a computer, then it is because of nurture, because the brain is nothing like a computer, and computers simulating language are nothing like brains.
That is not what is suggested. Llm still fuzzy mess, but supervisor / self editing is rules based
The way I see it:
(1) There is kind of a definition of a chair. But it's very long. Like, extremely long, and includes maybe even millions to billions of logical expressions, assuming your definition might need to use visual or geometric features of a given object to be classified as a chair (or not chair).
This is a kind of unification of neural networks (in particular LLMs) and symbolic thought: large enough symbolic thought can simulate NNs and vice versa. Indeed even the fact that NNs are soft and fuzzy does not matter theoretically, it's easy to show logical circuits can simulate soft and fuzzy boundaries (in fact, that's how NNs are implemented in real hardware! as binary logic circuits). But I think specific problems have varying degrees of more natural formulation as arithmetic, probabilistic, linear or fuzzy logic, on one hand, and binary, boolean-like logic on the other. Or natural formulations could involve arbitrary mixes of them.
(2) As humans, the actual definitions (although they may be said to exist in a certain way at a given time[1]) vary with time. We can, and do, invent new stuff all the time, and often extend or reuse old concepts. For example, I believe the word 'plug' in english likely well predates modern age, probably used to refer to original electrical power connectors. Nowadays there are USB plugs, which may not carry power at all, or audio plugs, etc. (maybe there are better examples). In any case the pioneer(s) usually did not envision all a name could be used for, and uses evolve.
(3) Words are used as tools to allow communication and, crucially, thought. There comes a need to put a fence (or maybe a mark) in abstract conceptual and logic space, and we associate that with a word. Really a word could be "anything we want to communicate", represent anything. In particular changes to the states of our minds, and states themselves. That's usually too general, most words are probably nouns which represent classifications of objects that exist in the world (like the mentioned chair) -- the 'mind state' definition is probably general enough to cover words like 'sadness', 'amazement', etc., and 'mind state transitions' probably can account for everything else.
We use words (and associated concepts) to dramatically reduce the complexity of the world to enable or improve planning. We can then simplify our tasks into a vastly simpler logical plan: even something simple like put shoes, open door, go outside, take train, get to work -- without segmenting the world into things and concepts (it's hard to even imagine thought without using concepts at all -- it probably happens instinctively), the number of possibilities involved in planning and acting would be overwhelming.
Obligatory article about this: https://slatestarcodex.com/2014/11/21/the-categories-were-ma...
---
Now this puts into perspective the work of formalizing things, in particular concepts. If you're formalizing concepts to create a system like Cyc, and expect it to be cheap, simple, reliable, and function well into the future, by our observations that should fail. However, formalization is still possible, even if expensive, complex, and possibly ever changing.
There are still reasons you may want to formalize things, in particular to acquire a deeper understanding of those things, or when you're okay in creating definitions set in stone because they will be confined to a group being attentive and restrictive to their formal definitions (and not, as natural language, evolving organically according to convenience): that's the case with mathematics. The peano axioms still define the same natural numbers; and although names may be reused, you can usually specify them to a particular axiomatic definition that will never change. And thus we can keep building facts on those foundations forever -- while what is a 'plug' in natural language might change (and associated facts about plugs become invalid), we can define mathematical objects (like 'natural numbers') with unchanging properties, and ever-valid and potentially ever-growing valid facts to be known about them, reliably. So fixing concepts in stone more or less (at least when it comes to a particular axiomatization) is not such a foolish endeavor it may look like, quite the opposite! Science in general benefits from those solid foundations.
I think eventually even some concepts related to human emotions and specially ethics will be (with varying degrees of rigor) formalized to be better understood. Which doesn't mean human language should (or will) stop evolving and being fuzzy, it can do so independently of formal more rigid counterparts. Both aspects are useful.
[1] In the sense that, at a given time, you could (theoretically) spend an enormous effort to arrive at a giant rule system that would probably satisfy most people, and most objects referred to as chairs, at a given fixed time.
How will the rules and facts be connected? By some discrete relationship? This stuff only works for math, and is the basis for the bitter lesson.
Intelligence is compression, and this is the opposite of that