I wrote something that does this (in R of all things)! I still need to polish it up a little in share it though. It goes: podcast mp3 -> whisper transcription -> Gemini Flash to identify ad blocks -> FFMPEG to replace ad blocks with skip sound.
That sounds really cool!
This is probably paranoid, but IIWM I'd be a little hesitant to use Gemini since this is a slightly gray area. If Google decides you're violating any ToS they might nuke your entire Google account and any "associated" accounts.
Depending on the podcast I would also worry that the podcast content might violate some safety guidelines they have that might also get you in hot water. For example, some of the podcasts I listened to in 2020/2021 timeframe had discussion about the Lab Leak theory of Covid back when that was a capital offense on Youtube and would get you taken down and sometimes banned. I'd be worried about Gemini seeing that content as a violation and triggering some sort of automated action against you. Also worth considering that even if stuff you listen to is fine now, it might not be the next time they decide to change the rules about what speech is allowed and what isn't.