This old obscure blog post about framework patterns has resonated with me throughout my career and I think it applies here too. LLMs are best used as "libraries" rather than "frameworks", for all the reasons described in the article and more, especially now while everything is in such flux. "Frameworks" are sexier and easier to sell though, and lead to lock-in and add-on services, so that's what gets promoted.
This is so good…
“… you can find frameworks not just in software, but also in ordinary life. If you buy package holidays, you're buying a framework - they transport you to some place, put you in a hotel, feed you and your activities have to fit into the shape provided by the framework (say, go into the pool and swim there). If you travel independently, you are composing libraries. You have to book your flights, find your accommodation and arrange your program (all using different libraries). It is more work, but you are in control - and you can arrange things exactly the way you need.”
My favorite blog post / presentation is Sandi Metz "The Wrong Abstraction", but this one is up there. Definitely punches above its weight for a small obscure post.
Yeah duplication way better than the wrong abstraction. Just write the dang switch statement
> switch
Hmm, that hit a bit of a nerve. My experience with switch blocks is it can be a gateway drug for teams A, B, C to add their special-case code to team D's repo within a `switch(calling_service)` block. My read of the presentation is more, factor your stuff so that any "switch" is a higher level concern that consumers can do in their own services. Then if you start to see all your consumers write very similar consumption logic, then start thinking about how to pull that down into the library/service itself.
But beyond that trigger nerve, agreed.
fair enough. I think switch statement is a broader category for "basic programming primitives that you should just do yourself" -
agree big switch statements can be an anti-pattern, e.g. when an interface is clearly better suited
oh heck yeah this rocks. I'm gonna add to the links section
Additionally in terms of career development, you're going to be a lot better off learning the low level LLM interfaces rather than being dependent on a framework (or their even more evil cousin, platforms). Once you learn those, jumping to a platform is usually trivial, whereas the reverse can be more challenging. Junior devs often think that the more frameworks they have on their resume the better, but it often pigeonholes you more than it helps.
And I don't mean to imply that frameworks are always bad. Things like security best practices out of the box can be worth it. But especially in AI right now, nobody knows what those best practices are going to be. So it's best to spend this time learning how to do things at a low level rather than attaching to some framework that may be obsolete in a year.
exactly - we keep trying to figure out the right interfaces, but we jump to assume that we know what they are.
If we had the right interface, we would set up the black box, and then put holes/knobs on the box to allow anyone to change the things they should actually need to change.
if we have the wrong interface, then the knobs aren't interesting, and instead we keep end up opening the box, or reaching into the holes at weird angles to do things that nobody knew we'd want to, but that are obviously the right things to do to maximize performance
someday we'll have the right interface, but for now, better to skip the box and do the extra cycles. You're an engineer, you can write a for loop and a switch statement. don't outsource your prompts and give up control flow to save a few hundred lines that will eventually become pretty customized anyway