Built a real-time code review assistant with GPT-4o
I built ReviewBot — a GitHub App that provides real-time code review comments on pull requests using GPT-4o. It's been running on our team's repos for 3 months and catching real bugs.
How it works
1. Listens for PR webhooks 2. Fetches the diff 3. Sends each changed file to GPT-4o with context 4. Posts review comments directly on the PR
Stats after 3 months
The system prompt includes our team's coding standards and common patterns. Fine-tuning on our codebase would probably reduce false positives further.
Source code coming soon (cleaning up for open source).
I made this transition 2 years ago from traditional NLP. Here's what helped:
1. Traditional ML knowledge transfers well — understanding evaluation, data pipelines, and model selection is still crucial 2. Most valuable skills: RAG architecture, eval pipeline design, and production engineering 3. Companies want builders, not researchers (unless you're at a lab) 4. Build a portfolio project that shows end-to-end: data → fine-tuning → deployment → monitoring
As a startup founder who hires LLM engineers: I look for people who can build reliable systems, not people who can write clever prompts. Show me you can handle errors gracefully, monitor costs, and evaluate model outputs systematically.
Traditional ML actually gives you a huge advantage. Most LLM-only engineers don't understand evaluation rigor. Being able to set up proper train/test splits, handle data contamination, and calculate confidence intervals is rare and valuable.
Log in to reply to this topic.