A wrong comment sends someone looking for a bug that does not exist. No comment can be just as misleading: the change might be clean, or the reviewer might have run out of time, missed a file, or lacked a tool it needed. Those two outcomes cannot look the same to the person deciding whether to merge.
I chose to make the public review conservative. AI Review first investigates a change and identifies possible defects. A separate pass checks each one against the source and looks for reasons it might be wrong. Only findings with enough evidence become comments. Weaker findings stay in the local report, where a developer can inspect them without adding noise to the pull request.
That choice creates an obligation. If the reviewer comments less often, it has to explain its silence. Each report says what was examined and where coverage stopped. A run that could not inspect part of the change says so, even if it found nothing in the parts it did inspect.
The source has to stay put
A second problem shaped that flow: code can change during a review. A pull request can receive another commit, and local files can be edited while the reviewer is working. AI Review captures the before and after versions of the change at the start of a run. A finding can then point to the exact code that produced it.
The rules governing the review come from a trusted version of the repository. Otherwise, a change could weaken the instructions used to review itself. That sounds like an edge case until you remember what an AI reviewer reads: source files, documentation, comments, and tool output. Some of those files are precisely what the change is trying to modify.
I kept code execution behind a separate boundary for the same reason. The reviewer can read related code and tests to understand a change, but reproducing a suspected defect happens in an isolated environment without the credentials used to access models or publish comments. A serious finding can block a merge only when the team has enabled blocking and the defect has been reproduced there. By default, AI Review advises; a person decides.
What I would want to see as a developer
The finished result is a self-hosted reviewer that works before a push and on a GitHub pull request. It can show the evidence behind a finding, the concerns it held back, and the parts of the change it could not assess. The team keeps control of its model backend and review policy.
AI Review will become a fully open source project. Teams will be able to run and inspect the reviewer themselves, using supported subscriptions and APIs they already have instead of taking on a new model provider just for code review. That choice belongs with the rest of the design: the people relying on a review should be able to examine how it reached its conclusions.
I cannot make an AI reviewer infallible. I can make its output easier to question. If it flags a defect, a developer can follow the evidence. If it says nothing, they can see how much of the change it actually reviewed. That makes the tool more useful to me than a longer list of confident comments.