This year I’ve reviewed more than 1000 code changes. Most of the time was spent catching obvious mistakes rather than debating complex design decisions. If we estimate ~10 minutes per review, that’s 160+ hours spent reviewing code in just one year.
So I thought: could I get some of that time back using LLMs?
That's why I spent the last few weekends building an LLM-based prereviewer that should take a first pass before the actual human reviewer. The results so far are promising: I estimate it can reduce the review time by 50%, which in my case would mean I save 80hours (~10 working days) per year.
Linked above is an example of a PR where I'm testing the AI reviewer and it showcase how it can detect bugs, suggest best practices about token validity, generate summary and title, and even chat with me in review comments.
The AI reviewer is a simple Github action that runs everytime I open or synchronize a pull request and you can see the source code at https://github.com/presubmit/ai-reviewer.
I use PR-Agent for this, it's great
Cool! Curious what do you like most about it compared to other solutions?
Commenting on PRs with suggested improvements is a big win to reduce the context switching senior engineers face reviewing junior developers PRs all day, giving a review complexity rating that can be used to route PRs to the right people is good as all. Of course being open source, platform agnostic is a massive win.
The review complexity rating sounds pretty cool. I'll look into that, thanks! Is there anything you'd wish PR-Agent had and it doesn't?
One idea I had was if there was an optional dashboard overview across all the repos it's enabled on where users could directly trigger / interact with specific PRs or issues - that would be neat.
Oh something else great it has - you can run it locally against any repo or branch, great for testing or pre-commit hooks.