OpenAI‘s early investment in Cursor was a masterstroke. Acquiring Windsurf would be another.
Next advances in coding AI depend on real-world coding data, esp how professional developers use agentic AI for coding + other tasks.
RL works well on sufficiently large base models as shown by rapid progress on verifiable problems with good training data, e.g. competition math, competitive coding problems, scientific question answering.
Training LLMs on detailed interaction data from AI-powered IDEs could become a powerful flywheel leading to the automation of practical coding.
How many developers want to have usage analytics of their editors helping companies build functionality that aspires to replace them? This is silly.
> How many developers want to have usage analytics of their editors helping companies build functionality that aspires to replace them? This is silly.
Honestly, too many. Software engineers can be really, really dumb. I think it has something to do with assuming they're really smart.
But even unwilling developers may be forced to participate (see the recent Shopify CEO email), despite knowing full well what's going on. I mean, tons of people have already had to go through the humiliation of training their offshore replacements before getting laid off, and that's a much more in-your-face situation.
Developers know that AI will replace some of their coworkers. But only the "bad" ones who can't code as well as them. AI will learn from all of their good code and be used to generate similar code that's better than the bad devs but not as good as theirs. The problem is that every developer thinks that the coding bar is going to be just barely below their skill level.
The frustrating part is that this is another area where the realities of capitalism seem misaligned with anyone's well-being. Nobody wants to be out of a job, but doing the opposite of the Shopify CEO's strategy, like severely restricting AI usage, looks like a great way to ensure your competitors catch up with you and eat your lunch faster. I don't see any answers, just different ways to destroy ourselves.
I agree: the incentives to use more and more AI are too strong. We're all stuck in some form of the prisoner's dilemma and the odds that nobody will defect are much too low.
So it seems the most rational position is to embrace the tools and try to ride the wave before the gravy-train is over.
> Honestly, too many. Software engineers can be really, really dumb. I think it has something to do with assuming they're really smart.
Maybe I am one of the stupid ones but I don't get you people.
This is going to happen whether you want it or not. The data is already out there. Our choice is either learn to use the tool so that we could have that in our arsenal for the future; or grumble in the corner that devs are digging their own graves and cry ourselves to sleep. I'd consider the latter to be stupid.
If you had issues with machines replacing your hands in the industrial age, you had a choice of learning how to operate the machines, I consider this to be a parallel.
It's not about having another tool in your arsenal. This thing is meant to replace you - the developer (role). Others are correctly pointing out that developers can be really really dumb by assuming that this A-SWE will be just below their skill level and only the subpar humans will be replaced.
> It's not about having another tool in your arsenal. This thing is meant to replace you - the developer (role).
You realize that it's what I am saying? Having the tool in our arsenal means being able to do another job (prompt engineering, knowing how to evaluate the AI etc...) in case we are made obsolete in the next couple of years. What happens after that is a mystery...
They're probably just short-sighted - take the easy win ("get more done in less time") now and worry about the future later.
> Honestly, too many. Software engineers can be really, really dumb. I think it has something to do with assuming they're really smart.
I've found the enthusiasm towards software that ostensibly aims to replace their skillset utterly bizarre.
Many of them likely won’t switch immediately. They could also try to keep them with sweet offers, like generous usage quotas, early access to the latest models, etc.
Once sufficient data is gathered, the next generation models will be among the very best at agentic coding, which leads to stronger stickiness, and so on.
> Training LLMs on detailed interaction data from AI-powered IDEs could become a powerful flywheel leading to the automation of practical coding.
I agree. But this is a more general flywheel effect. OpenAI has 500M users generating trillions of interactive tokens per month. Those chat sessions are sequences of interaction, where downstream context can be used to judge prior responses. Basically, in hindsight, you check "has this LLM response been good or bad?", and generate a score. You can expand the window to multiple related chats. So you can leverage extended context and hindsight for judging response quality. Using that data you can finetune a RLHF model, and with it finetune the base model.
But it's not just hindsight analysis. Sometimes users test or implement projects in the real world, and the LLM gets to see idea validation. Other times they elicit tacit experience from humans. That is what I think forms an experience flywheel. LLM being together with humans during problem solving, internalizing approaches, learning from outcomes.
Besides problem solving assistance LLMs are used for counselling/keeping company/therapeutic role. People chat with LLMs to understand and clarify their goals. These are generative teleological models. They are also used by 90% of students if I am to believe a random article.
So the triad of uses for LLMs are: professional problem solving, goal setting/therapy, and learning. All three benefit from the flywheel effect of interacting with millions of people.
How is it supposed to learn to automate development by watching us not do things? Which is what the LLMs are used for currently.
Reinforcement learning - with vibe coding, it just needs us to give it the reward signal.
So a bunch of people who can't code are going to train it? Or, rather, how will you know it is the right reward? Doesn't seem like a good way to train.
Essentially they have paying customers upvoting the results they like and downvoting the ones they don't like. This is economics 101 for continuous improvement.
That's a recipe for a tool that impresses the hell out of new/casual users but doesn’t work.
It’s the same reason you should never choose an oncologist using yelp reviews.
I don't understand the argument you're making. I'm grateful to have not had a need for an oncologist, but I would assume that honest reviews from patients whose cancer is either in remission or not should be relevant. What am I missing?
Yelp reviews for doctors are all about how nice the doctors act and how much they tell the patients what they want to hear. It's not like people leave an "I died, 1 star" review.
Usually with Business or Enterprise plans, there's a privacy mode where OpenAI and Cursor / Windsurf can't train on user data.