The biggest things are the number of "clicks" required to get to a particular podcast, the long-ish loading times after those clicks, and some little papercuts like RSS feeds not updating unless you click the "search" button, then click "search" to force it to refresh. I also get a lot of server crashes as a result of various issues consuming RSS feeds. I can't use ABS for maybe 30% of the podcasts I listen to because of that. I do love having the podcast episodes downloaded automatically onto my host though. That's an absolutely killer feature and for that alone I have some podcasts I really care about still on there.

FTR, I absolutely love Aubiobookshelf. It's an amazing project and I recommend it without hesitation to everyone. I think some or all of those will be fixed eventually, but it may take a bit of time. I do wonder if the ideal UI for books may just not be a good fit for podcasts, though I would need to think on that quite a bit before coming to such a conclusion.