rjakob 6 days ago

Based on my experience, many reviewers are already using AI extensively. I recently ran reviewer feedback from a top CS conference through an AI detector, and two out of three responses were clearly flagged as AI-generated.

In my view, the peer-review process is flawed. Reviewers have little incentive to engage meaningfully. There’s no financial compensation, and often no way to even get credit for it. It would be cool to have something like a Google Scholar page for reviewers to showcase their contributions and signal expertise.

1
AStonesThrow 6 days ago

The only thing worse than an LLM for making stuff up and giving fake numbers is an LLM "Detector". They are so full of false positives and false negatives and bogus percentages, as to be actively harmful to human trust and academic integrity. And how do you follow up, to verify or falsify their results?

rjakob 6 days ago

Fair. Though in this case, it was obvious even without a detector.