You've laid out the dream for a lot of developers around the world.
Agency life is probably 80% sales and 20% delivery. The agency pattern you've mentioned is common because it consistently works and everyone (myself included) fawns over the basecamp model but they're about the only ones who've made it work.
AI-generated SaaS tools just remind me of white-label SaaS products that were the hot thing however many years ago. They do about 50% of what the customer wants and a PITA to customize.
Having said that if I started again I'd still build one or more AI-centric B2B products with a focus on closed AI systems and look for agency work (ad-hoc development) as an offshoot.
You'd be able to build up a client base with your B2B products and have tangible examples to demonstrate what you're capable of while playing on the fact that the client's data is secure and never shared with third parties.
Your comment "They do about 50% of what the customer wants" made me thing, what if the agency could rapidly iterate and support the other 50% they want by leveraging AI?
I think you would have to design your code system on the idea of simplicity and good building blocks. Then iterate on top of that to customize what the customer wants.