I suspect it probably won't work, although it's not necessarily because an LLM architecture could never perform this type of work, but rather because it works best when the training set contains inordinate sample data. I'm actually quite shocked at what they can do in TypeScript and JavaScript, but they're definitely a bit less "sharp" when it comes to stuff outside of that zone in my experience.
The ridiculous amount of data required to get here hints that there is something wrong in my opinion.
I'm not sure if we're totally on the same page, but I understand where you're coming from here. Everyone keeps talking about how transformational these models are, but when push comes to shove, the cynicism isn't out of fear or panic, its disappointment over and over and over. Like, if we had an army of virtual programmers fixing serious problems for open source projects, I'd be more excited about the possibilities than worried about the fact that I just lost my job. Honest to God. But the thing is, if that really were happening, we'd see it. And it wouldn't have to be forced and exaggerated all the time, it would be plainly obvious, like the way AI art has absolutely flooded the Internet... except I don't give a damn if code is soulless as long as it's good, so it would possibly be more welcome. (The only issue is that it most likely actually suck when that happens, and rather just be functional enough to get away with, but I like to try to be optimistic once in a while.)
You really make me want to try this, though. Imagine if it worked!
Someone will probably beat me to it if it can be done, though.
Imo they are still extremely limited compared to a senior coder. Take python, still most top ranking models struggle with our codebase, every now and then I try to test few, and handling complex part of the codebase to produce coherent features still fails. They require heavy handholding from our senior devs, which I am sure they use AI as assistants.
> the cynicism isn't out of fear or panic, its disappointment over and over and over
Very much this. When you criticize LLM's marketing, people will say you're a ludite.
I'd bet that no one actually likes to write code, as in typing into an editor. We know how to do it, and it's easy enough to enter in a flow state while doing it. But everyone is trying to write less code by themselves with the proliferation of reusable code, libraries, framework, code generators, metaprogramming,...
I'd be glad if I could have a DAW or CAD like interface with very short feedback (the closest is live programming with Smalltalk). So that I don't have to keep visualizing the whole project (it's mentally taxing).
> no one actually likes to write code
between this and..
> But everyone is trying to write less code by themselves with the proliferation of reusable code, libraries, framework, code generators, metaprogramming
.. this, is a massive gap. Personally speaking, I hate writing boilerplate code, y'know, old school Java with design patterns getter/setter, redundant multi-layer catch blocks, stateful for loops etc. That gets on my nerves, because it increases my work for little benefits. Cue modern coding practices and I'm almost exclusively thinking how to design solution to the problem at hand, and almost all the code is business logic exclusive.
This is where a lot of LLMs just fail. Handholding them all the way to correct solution feels like writing boilerplate again, except worse because I don't know when I'll be done. It doesn't help that most code available for LLMs is JS/TS/Java where boilerplate galore, but somehow I doubt giving them exclusively good codebases will help.
I like writing code. It's a fun and creative endeavor to figure out how to write as little as possible.
>I'd bet that no one actually likes to write code
And you'd be wrong. I, for one, enjoy the process of handcrafting the individual mechanisms of the systems I create.
Do you like writing all the if, def, public void, import keywords? That is what I’m talking about. I prefer IDE for java and other verbose languages because of the code generation. And I configure my editors for templates and snippets because I don’t like to waste time on entering every single character (and learned vim because I can act on bigger units; words, lines, whole blocks).
I like programming, I do not like coding.
I'm not bothered by if nor def. public void can be annoying but it's also fast to type and it doesn't bother me. For import I always try my best at having some kind of autoimport. I too use vim and use macros for many things.
To be honest I'm more annoyed by having to repeat three times parameters in class constructors (args, member declaration and assignment), and I have a macro for it.
The thing is, most of the time I know what I want to write before I start writing. At that point, writing the code is usually the fastest way to the result I want.
Using LLMs usually requires more writing and iterations; plus waiting for whatever it generates, reading it, understanding it and deciding if that's what I wanted; and then it suddenly goes crazy half way through a session and I have to start over...
> if that really were happening, we'd see it.
You're right, instead what we see is the emergence of "vibe coding", which I can best describe as a summoning ritual for technical debt and vulnerabilities.
the typescript and javascript business though - the ais definitely trained on old old javascript.
i kinda think "javacript, the good parts" should be part of the prompt for generating TS and JS. I've seen too much of ai writing the sketchy bad parts
So yesterday I wanted to convert a color pallet I had in Lua that was 3 rgb ints, to Javascript 0x000000 notation. I sighed, rolled my eyes, but before I started this incredibly boring mindless task, asked Gamini if it would just do it for me. It worked, and I was happy, and I moved on.
Something is happening, its just not exciting as some people make it sound.
Be a bit more careful with that particular use case. It usually works, but depending on circumstances, LLMs have a relatively high tendency to start making the wrong correlations and give you results that are not actually accurate. (Colorspace conversions make it more obvious, but I think even simpler problems can get screwed up.)
Of course, for that use case, you can _probably_ do a bit of text processing in your text processing tools of choice to do it without LLMs. (Or have LLMs write the text processing pipeline to do it.)