rglover 4 days ago

I speculated a ways back [1] that this was why Elon Musk bought Twitter. Not to "control the discourse" but to get unfettered access to real, live human thought that you can train an AI against.

My guess is OpenAI has hit limits with "produced" content (e.g., books, blog posts, etc) and think they can fill in the gaps in the LLMs ability to "think" by leveraging raw, unpolished social data (and the social graph).

[1] https://news.ycombinator.com/item?id=31397703

2
godelski 4 days ago

But collecting more data is just a naive task. The reason scale works is because of the way we typically scale. By collecting more data, we also tend to collect a wider variety of data and are able to also collect more good quality data. But that has serious limits. You can only do this so much before you become equivalent to the naive scaling method. You can prove this yourself fairly easily. Try to train a model on image classification and take one of your images and permute one pixel at a time. You can get a huge amount of scale out of this but your network won't increase in performance. It is actually likely to decrease.

chewbacha 4 days ago

If that were the case he (Musk) wouldn’t have turned it into a Nazi-filled red pilled echo chamber.