Review Engine: Bringing Automated Code Review Together

Welcome to this lesson on building a Review Engine! So far, you have learned how to set up an AI client, parse code changes, and gather useful context for code review. Now, you will see how these pieces come together in a Review Engine — a tool that automates the process of reviewing code changes using AI.

A Review Engine is a program that takes a set of code changes (called a "changeset"), gathers all the important information about those changes, and then asks an AI model to review them. This helps developers catch mistakes, improve code quality, and save time.

By the end of this lesson, you will understand how to build a Review Engine that can review both individual files and entire changesets, using all the tools you have learned so far.

Before we dive in, let's quickly remind ourselves how the main parts work together:

• OpenAI Client: This is the tool that sends code and context to the AI model and gets back a review.
• Diff Parser: This breaks down the code changes into a format that is easy to work with.
• Context Generator: This gathers extra information about the code, like recent changes and related files, to help the AI give better feedback.

In this lesson, you will see how the Review Engine uses all these parts to review code changes automatically.

Let's start by looking at how the Review Engine reviews one file at a time. This is the basic building block for reviewing larger sets of changes.

The first thing the Review Engine does is parse the diff for the file. The diff shows what has changed in the code.

import logging
import time

logger = logging.getLogger(__name__)

def review_changeset_file(self, session, changeset_file):
    start_time = time.time()
    file_path = changeset_file.file_path
    
    logger.info(f"Starting review for file: {file_path}")
    
    try:
        diff = parse_unified_diff(changeset_file.diff_content)

Here, parse_unified_diff is a function that takes the raw diff text and turns it into a structured object. This makes it easier to work with the changes.

• changeset_file.diff_content is the text showing the changes for this file.
• diff will now hold information like the file path and the specific lines that changed.

Next, the Review Engine gathers extra information about the file. This helps the AI understand the code better.

        file_context = get_file_context(session, diff.file_path)
        recent_changes = get_recent_changes(session, diff.file_path)
        related_files = find_related_files(session, diff.file_path)
        
        logger.debug(f"Gathered context for {file_path}: {len(recent_changes)} recent changes, {len(related_files)} related files")

• get_file_context gets the current content of the file.
• get_recent_changes finds recent changes made to this file.
• find_related_files lists other files that are related to this one.

The Review Engine then combines this information into a summary that will be sent to the AI.

        context_parts = []

        if recent_changes:
            recent_summary = "; ".join([
                f"{change['hash']}: {change['message']}" 
                for change in recent_changes[:2]
            ])
            context_parts.append(f"Recent changes: {recent_summary}")

        if related_files:
            context_parts.append(f"Related files: {', '.join(related_files)}")

        context = " | ".join(context_parts)

• This code creates a list called context_parts.
• If there are recent changes, it adds a summary of the last two changes.
• If there are related files, it adds their names.
• Finally, it joins everything into a single string called context.

Now, the Review Engine asks the AI to review the changes, using the context we just built.

        review = self.llm_client.analyze_changeset(
            file_path=diff.file_path,
            diff=changeset_file.diff_content,
            context=context
        )
        
        duration = time.time() - start_time
        logger.info(f"Successfully reviewed {file_path} in {duration:.2f} seconds")
        
        return review
        
    except Exception as e:
        duration = time.time() - start_time
        logger.error(f"Failed to review {file_path} after {duration:.2f} seconds: {str(e)}")
        raise

• analyze_changeset is a method that sends the file path, the diff, and the context to the AI.
• The AI returns a review, which is stored in the review variable.
• The engine logs both successful reviews and failures, including timing information.

Example Output:

INFO: Starting review for file: example.py
DEBUG: Gathered context for example.py: 2 recent changes, 2 related files
INFO: Successfully reviewed example.py in 3.42 seconds
Review for example.py with context: Recent changes: abc12345: Initial commit; def67890: Refactor code | Related files: utils.py, helpers.py

This output shows that the AI has reviewed the file and included the context we provided, along with logging information about the process.

Now that you know how to review a single file, let's see how the Review Engine reviews all files in a changeset.

The Review Engine goes through each file in the changeset and reviews them one by one.

def review_changeset(self, session, changeset):
    changeset_start_time = time.time()
    total_files = len(changeset.files)
    
    logger.info(f"Starting changeset review for {total_files} files")
    
    reviews = {}
    successful_reviews = 0
    failed_reviews = 0
    
    for changeset_file in changeset.files:
        try:
            review = self.review_changeset_file(session, changeset_file)
            reviews[changeset_file.file_path] = review
            successful_reviews += 1
        except Exception as e:
            logger.warning(f"Skipping file {changeset_file.file_path} due to error: {str(e)}")
            failed_reviews += 1
    
    total_duration = time.time() - changeset_start_time
    logger.info(f"Changeset review completed in {total_duration:.2f} seconds. "
               f"Success: {successful_reviews}, Failed: {failed_reviews}")

• changeset.files is a list of all the files that were changed.
• For each file, the engine calls review_changeset_file, which does everything we just covered.
• The results are stored in a dictionary called reviews, with the file path as the key.
• The engine tracks and logs success/failure statistics for the entire changeset.

After all files are reviewed, the engine returns the results.

    return reviews

Example Output:

INFO: Starting changeset review for 2 files
INFO: Starting review for file: example.py
DEBUG: Gathered context for example.py: 2 recent changes, 2 related files
INFO: Successfully reviewed example.py in 3.42 seconds
INFO: Starting review for file: utils.py
DEBUG: Gathered context for utils.py: 1 recent changes, 1 related files
INFO: Successfully reviewed utils.py in 2.15 seconds
INFO: Changeset review completed in 5.73 seconds. Success: 2, Failed: 0

Review for example.py:
Review for example.py with context: Recent changes: abc12345: Initial commit; def67890: Refactor code | Related files: utils.py, helpers.py

Review for utils.py:
Review for utils.py with context: Recent changes: abc12345: Initial commit | Related files: example.py

This shows that each file in the changeset has been reviewed, the results are organized by file, and comprehensive logging tracks the entire process.

The sequential approach shown above works well for small changesets, but for large changesets with many files, reviewing them one by one can be slow. Here are strategies to improve performance:

Instead of reviewing files individually, you can group them into batches and send multiple files to the AI in a single request:

def review_changeset_batch(self, session, changeset, batch_size=5):
    files = changeset.files
    reviews = {}
    
    # Group files into batches
    for i in range(0, len(files), batch_size):
        batch = files[i:i + batch_size]
        batch_contexts = []
        
        for changeset_file in batch:
            # Gather context for each file in the batch
            diff = parse_unified_diff(changeset_file.diff_content)
            context = self.build_file_context(session, diff.file_path)
            batch_contexts.append({
                'file_path': diff.file_path,
                'diff': changeset_file.diff_content,
                'context': context
            })
        
        # Send entire batch to AI for review
        batch_reviews = self.llm_client.analyze_changeset_batch(batch_contexts)
        
        # Add batch results to overall reviews
        for file_review in batch_reviews:
            reviews[file_review['file_path']] = file_review['review']
    
    return reviews

For even better performance, you can review multiple files or batches concurrently:

import concurrent.futures
import threading

def review_changeset_parallel(self, session, changeset, max_workers=3):
    reviews = {}
    reviews_lock = threading.Lock()
    
    def review_single_file(changeset_file):
        try:
            review = self.review_changeset_file(session, changeset_file)
            with reviews_lock:
                reviews[changeset_file.file_path] = review
            return True
        except Exception as e:
            logger.warning(f"Failed to review {changeset_file.file_path}: {str(e)}")
            return False
    
    # Process files in parallel
    with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = [executor.submit(review_single_file, f) for f in changeset.files]
        
        # Wait for all reviews to complete
        concurrent.futures.wait(futures)
    
    return reviews

• Batching: Reduces API calls but may hit token limits with very large batches
• Parallelization: Faster processing but consumes more API rate limits simultaneously
• Memory usage: Large changesets require more memory to store all reviews
• Error handling: Parallel processing makes error handling more complex

When to use each approach:

• Sequential: Small changesets (< 10 files) or when debugging
• Batching: Medium changesets (10-50 files) with token limit considerations
• Parallel: Large changesets (> 50 files) when speed is critical and rate limits allow

In a production Review Engine, proper logging is essential for monitoring, debugging, and performance optimization. Here are the key logging practices demonstrated above:

• INFO: Start/completion of reviews, timing, and success summaries
• DEBUG: Detailed context information that helps with troubleshooting
• WARNING: Non-fatal errors that allow the process to continue
• ERROR: Fatal errors that prevent a file from being reviewed

• File path: Always log which file is being processed
• Success/failure status: Track whether each review completed successfully
• Timing: Measure how long each operation takes
• Context statistics: Log how much context was gathered (number of changes, related files)

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('review_engine.log'),
        logging.StreamHandler()
    ]
)

This configuration ensures that all review activities are logged both to a file and to the console, making it easy to monitor the Review Engine in production.

The quality of the AI's review depends on the context you provide. Let's look at how the Review Engine builds and uses this context.

Suppose you have a file called example.py that was recently changed. The Review Engine gathers:

• The last two changes:
  • abc12345: Initial commit
  • def67890: Refactor code
• Related files:
  • utils.py
  • helpers.py

It combines this information into a single string:

context = "Recent changes: abc12345: Initial commit; def67890: Refactor code | Related files: utils.py, helpers.py"

This context is then sent to the AI, helping it understand not just the current change, but also the history and connections to other files.

Why is this important?
By giving the AI more information, you help it make better suggestions and catch issues that might be missed if it only saw the code change by itself.

In this lesson, you learned how to build a Review Engine that brings together the OpenAI client, diff parser, and context generator to review code changes automatically. You saw how to:

• Review a single file by parsing its diff, gathering context, and generating a review.
• Review an entire changeset by looping through all changed files.
• Build and use context summaries to help the AI give better feedback.
• Implement production-ready logging to track file paths, success/failure status, and timing.
• Optimize for large changesets using batching and parallelization strategies.

Next, you will get a chance to practice these ideas by working with code that reviews changesets using the Review Engine. This hands-on practice will help you solidify your understanding and prepare you to use AI-powered code review in real projects.