I found that the PydanticAI [0] framework strikes a perfect balance between control and abstraction.
I’m building a non trivial AI app and the validation and dependency injection is such a great addition compared to using the LLM libraries directly.
They recently added MCP support: https://ai.pydantic.dev/mcp/
Have you tried it?
Not yet! Really looking forward to it. Their development pace is hard to keep up with.
Thanks, this looks great. I've been playing with Huggingface's Smolagents, which is fun to tinker with and relatively easy to read through. But it is so tightly coupled to its two agent implementations - ToolAgent and CodeAgent - that it's not trivial at all to add your own state transformations.
This framework looks really well designed, I'm going to take it for a spin.
The main problem that I found playing with smolagents is the lack of documentation and examples. Anyway the framework has potential
Thanks just discovered https://ai.pydantic.dev/graph/
from the above link, which seems to use FSMs instead of DAGs.
The one thing I wish was better developed is persistence and streaming - they give sample code to stream, but it’s essentially a complete implementation of streaming that every client needs to implement.
I read the article but have yet to understand why someone would want to use a framework that introduces meaningless abstractions that are not properly documented or well maintained -aka, they often introduce breaking changes.
I’m interested in a useful agentic framework but LangGraph doesn’t seem to cut it.
Agreed if you are building a typical service. The abstractions will slow you down and don’t and anything.
The use case where they are helpful is “bring your own keys” apps. I maintain https://github.com/kiln-ai/kiln which allows you to bring keys for 13 different providers. The abstraction is very much worth it for me.
That said:
- I migrated from LangChain to LiteLLM and never looked back
- I have over 1000 automated integration tests that check the grid of LLM features (tools, json), model, and provider. Without them it would still be a mess.
fully agree that LangChain is a meaningless abstraction but I've found that the graph abstraction that LangGraph uses is a very useful mental model for thinking about an agentic flow
Would you use different framework?
For what LangChain does, most of the time I see no need for any framework. I would rather directly work with a vendor's official package. LangGraph is different. It is a legitimate piece of workflow software and not a wrapper framework. Now, when it comes to workflow there are many other well established engines out there that I will consider first.
I think the main thing LangGraph adds is a state machine framework for human in the loop with time travel.
So if you have an authoring workflow where a doc goes through a bunch of steps, and at some steps the analyst might want to fix some LLM output manually, and try a couple of things and then go back to the way it was before and try again, it will do that and you won't have to make your own state machine.
Now is it actually a state machine or is it just well logged?
LangGraph implements a variant of the Pregel/BSP algorithm for orchestrating workflows with cycles (ie. not DAGs) and parallelism without data races. You can design your graph as a state machine if you so desire
Pretty much this
Not to mention most of those frameworks were almost vibecoded.
They offer very little on top of what you could do it yourself
But of course some people think "if it exists we need to use it"
One thing that's nice in LangGraph is the flexibility in what a node is. A node doesn't have to be an agent it can be any State manipulation function. So you can create nodes that exclusively do preprocessing or postprocessing and get your run loop as close as possible to just ainvoke on your graph repeatedly.
You can have a subclass of your Node class be an AgentNode class and then subclass that for each type of Agent and then when you declare your Graph object you pass in the data to instantiate the AgentNode with the type of data it needs. It is a bit weird that LangGraph doesn't have a default Node class but it sort of makes sense that they want you to write it in a way that makes sense for how you use it.
I do highly recommend abstracting your graph into Node and Edge classes (with appropriate subclasses) and being able to declare your graph in a constant that you can pass to a build_graph method. Getting as much code reuse as possible dramatically simplifies debugging graph issuses.
What are the benefits of these frameworks(Langgraph, Llamaindex) over a dedicated orchestration platform like Temporal (or DBOS as a lightweight alternative)?
How can one deploy LangGraph as an API (with production like features)? I have worked with langgraph serve to deploy locally, but are there other frameworks to deploy langgraph?
You can check out https://github.com/JoshuaC215/agent-service-toolkit for a pretty comprehensive template for deploying a langgraph service, with a streamlit UI as an example client