I get it and I see the same problems as the author.
I'm working on a few toy projects and I am using LLM for 90% of it.
The result is 10x faster than if I coded it "by hand", but the architecture is worse and somewhat alien.
I'm still keeping at it, because I'm convinced that LLM driven code is where things are headed, inevitably. These tools are just crazy powerful, but we will have to learn how to use them in a way that does not create a huge mess.
Currently I'm repeatedly prompting it to improve the architecture this way or that way, with mixed results. Maybe better prompt engineering is the answer? Writing down the architecture and guidelines more explicitly?
Imagine how the whole experience will be if the latency was 1/10th of what it is right now and the tools are 10x better.
I hope we get to that "10x better" point. I think the problem right now is people advertising LLMs as if we're there already. And it's not just the providers, it's also the enthusiasts on X/Reddit/etc that think they have found the perfect workflow or prompt.
Just like you're mentioning "maybe better prompt engineering", I feel like we're being conditioned to think "I'm just not using it right" where maybe the tool is just not that good yet.
Well "I'm just not using it right" is a perfectly reasonable thought for a new technology. Isn't this the default? When new powerful tools come along, they often fit awkwardly into existing processes.
One thing you can do is to define the classes and methods that you want to have, and have the LLM implement them. For tricky things, you can leave additional notes in the empty method body as to how things should be implemented.
This way you're doing the big picture thinking while having the LLM do what's it's good at, generating code within the limits of its context window and ability to reason about larger software design.
I mostly treat the LLM as an overly eager to please junior engineer that types very quickly, who can read the documentation really quickly, but also tends to write too much code and implement features that weren't asked for.
One of the good things is that the code that's generated is so low effort to generate that you can afford to throw away large chunks of it and regenerate it. With LLM assistance, I wrote some code to process a dataset, and when it was too screwy, I just deleted all of it and rewrote it a few times using different approaches until I got something that worked and was performant enough. If I had to type all of that I would've been disappointed at having to start over, and probably more hesitant to do so even if it's the right thing to do.
I've found a lot of value in this approach as well, I don't delegate any architecture decisions to LLMs. I build out the high-level and I see if the LLM can fill the gaps. I've found they are good at writing pure functions, and am good at composing them and managing state.
This is where they shine, for prototyping greenfield projects, but as the project gets closer to production that 10x erodes. You have to be really intentional about the architecture, fixing core design issues later can turn 10x into 0.1x.
At least currently, the only use pattern that can withstand complex codebases is as advanced speech-to-text, but without the speech if that makes sense. The main issue with that is phrasing things in English is often far more verbose, so without the speech, it's very often faster to just do it by hand.
> Writing down the architecture and guidelines more explicitly?
Yes, very explicit like “if (condition) do (action)” and get more explicit when… oh wait!
Yeah. I never understood where people are coming with “you need guardrails, extensive architecture docs, coding rules,…”. For every software and features I wrote, I already have a good idea of the objectives before I even start to code. I do the specific part with code, going back to the whiteboard when I need to think.
It’s an iterative process, not a linear one. And the only hige commits are the scaffolding and the refactorings. It’s more like sculpture than 3d printing, a perpetual refinement of the code instead of adding huge lines of code.
This is the reason I switched to Vim, then Emacs. They allow for fast navigation, and faster editing. And so easy to add your own tool as the code is a repetitive structure. The rare cases I needed to add 10s of lines of code is with a code generator, or copy-pasting from some other file.