Imagine your code writing code that earns money. This ambitious concept became a reality for one developer who built an AI Agent capable of autonomously reviewing GitHub Pull Requests (PRs) and claiming cryptocurrency bounties.
This article details the development of `claude-review-agent`, a CLI tool designed to fetch GitHub PR diffs, generate structured code reviews, and which successfully earned its creator a $150 bounty.
The Problem: Overwhelmed Open Source Maintainers
Open source project maintainers frequently face an overwhelming backlog of PRs. Code reviews can consume hours, leaving contributors waiting for days for crucial feedback. The question arises: what if an AI could handle the initial pass of these reviews?
RustChain, a DePIN blockchain project, recognized this need and offered a bounty for a solution:
“Build a CLI tool that reviews PRs and posts structured comments.”
The reward for this task was $150 in RTC tokens.
The Solution: claude-review-agent
A Node.js CLI tool was developed with the following key functionalities:
- Fetches PR diffs using the GitHub API.
- Generates structured code reviews leveraging Claude AI.
- Posts comments directly onto PRs.
- Outputs review reports in easily readable Markdown format.
Architecture Overview
The system architecture is straightforward:
- The `claude-review.js` tool first interacts with the GitHub API to obtain the PR diff.
- This diff is then sent to Claude AI for analysis and review generation.
- Finally, the structured Markdown review is ready to be posted as a comment on the PR.
Implementation Steps
Step 1: Parse the PR URL
The initial step involves extracting critical information from the GitHub PR URL: the repository owner, repository name, and the PR number. This is achieved through a parsing function that uses regular expressions to match and extract these components, ensuring the URL is valid and its parts are correctly identified.
Step 2: Fetch the PR Diff
Once the PR details are parsed, the tool proceeds to fetch the complete PR diff using the GitHub API. This is typically done by sending a `curl` request with an `Authorization: Bearer [token]` header for authentication and an `Accept: application/vnd.github.v3.diff` header. The latter specifically requests the unified diff format, which is essential for detailed code analysis.
Step 3: Generate the Review
This is where Claude AI's capabilities are put to use. The fetched PR diff is fed into a carefully constructed prompt, instructing Claude AI to act as an "expert code reviewer." The prompt demands a structured analysis comprising:
- Summary: A 2-3 sentence overview of the changes.
- Risks: A bulleted list of potential issues or concerns.
- Suggestions: Bulleted recommendations for improvements.
- Confidence: A rating of the AI's confidence in its review: High/Medium/Low.
Claude AI processes the diff against these instructions, generating a comprehensive review that can then be formatted and posted back to the GitHub PR.