Review Then Edit Pattern

Review Then Edit Pattern | CodeSignal Learn

Introduction

Welcome back to Mastering Advanced AI Tooling in Codex! In previous lessons, we established the foundation: configuring Codex through config.toml, enabling secure web search, creating Skills (custom specialized capabilities) with arguments, and using inline shell capture (!) to inject real-time repository state into our prompts. Today, we are building a review-then-edit workflow template that embodies safe, automated code modification. Unlike simple prompts that execute immediately, this pattern enforces a structured workflow: clarify intent, propose a plan, apply minimal changes, verify results, and report outcomes. This approach prevents the common pitfalls of AI-driven editing (such as scope creep, untested changes, or modifications to protected files) while maintaining the efficiency and intelligence that make Codex valuable. We will construct a reusable SAFE-EDIT workflow template that transforms a high-level goal into vetted, tested code changes. What we're building: We're creating workflow template files (like safe_edit.txt) that you invoke by pasting their contents into prompts or referencing them with @filename.txt . These are NOT Skills from Unit 3 — they're structured prompt patterns that guide Codex through a multi-phase process. Think of them as recipes for AI interactions rather than registered capabilities in the Codex system.

The Problem with Direct Edits

When we ask an AI agent to modify code directly, several risks emerge. The agent might misunderstand our intent and change unrelated code; it might apply overly broad refactorings when we wanted a targeted fix; or it might modify files we consider protected (lockfiles, configuration, vendor dependencies). Without verification, we have no confidence that changes preserve functionality. The fundamental issue is mixing cognition with action : the agent decides what to do and does it simultaneously, leaving no checkpoint where we can assess its understanding before committing to changes. This creates an all-or-nothing dynamic: either we accept whatever the agent produces, or we reject everything and start over. In collaborative development, this lacks the deliberation we would expect from code review or pair programming. A better approach separates these concerns. First, the agent demonstrates understanding by articulating what needs to change and why; second, it proposes a specific, minimal plan; third, it executes that plan under defined constraints; finally, it proves the changes work through automated verification. This multi-phase structure transforms unpredictable agent behavior into a reliable, auditable process.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Defining the Workflow Structure

We begin by declaring our workflow template and establishing its primary contract. This is a standardized prompt pattern — a convention we invoke consistently, not a built-in Codex command or registered Skill. We'll name this pattern SAFE-EDIT : text Use the SAFE-EDIT workflow for this goal: <goal> Rules: 1) Ask up to 3 clarifying questions if needed; otherwise list assumptions. 2) Propose a minimal patch plan (files + what changes). 3) Stop for approval before editing. Use the SAFE-EDIT workflow for this goal: <goal> Rules: 1) Ask up to 3 clarifying questions if needed; otherwise list assumptions. 2) Propose a minimal patch plan (files + what changes). 3) Stop for approval before editing. The <goal> parameter is a natural-language description of what we want to accomplish, such as "add a regression test for CLI flag parsing bug #123" or "fix null pointer handling in user authentication". This parameter drives the entire workflow. The numbered rules establish the workflow's contract: it must clarify, plan, and wait for approval. This creates explicit checkpoints where humans can assess understanding and correctness before any code changes occur. Note on approvals: If your Codex config uses approval_policy = "on-request", Codex will also prompt you before running commands or writing files. This means you'll get two approval points: one for the plan itself, and one when Codex attempts to write changes.

Step 1: Clarifying Questions and Assumptions

The first phase of our workflow ensures the agent understands the goal before proceeding. We instruct it to ask questions when ambiguity exists or state assumptions when the goal seems clear: text 1) Ask up to 3 clarifying questions if needed; otherwise list assumptions. 1) Ask up to 3 clarifying questions if needed; otherwise list assumptions. This step prevents the agent from guessing. If the goal says "add regression test for bug #123" but the repository has no issue tracker or the bug number is ambiguous, the agent should ask: "Which bug #123? Can you describe the expected behavior?" If the goal is clear ("add email validation to the signup form"), the agent restates: "I will add email format validation using a regex pattern, applied in the validate_signup function before database insertion." The restatement serves two purposes: it confirms understanding and gives us a chance to correct misconceptions. If the agent says "I will refactor the entire authentication module," but we only wanted a targeted fix, we can intervene immediately rather than discovering the mismatch after code changes. The limit of "up to 3 questions" prevents analysis paralysis; if the agent needs more than three questions to understand the goal, the goal itself is probably too vague and we should rewrite it. This phase produces no code changes; it is purely communicative. The output is either a set of questions — requiring our response before proceeding — or a clear statement of intent — allowing the workflow to continue automatically.

Step 2: Planning the Patch

Once the goal is clarified, the agent identifies exactly which files and lines need modification and proposes a concrete patch plan: text 2) Propose a minimal patch plan (files + what changes). 2) Propose a minimal patch plan (files + what changes). The plan should specify file paths, function or class names, and approximate line ranges. For example: "Modify src/auth.py, function validate_user, lines 45-52: add null check before accessing user.email . Add test in tests/test_auth.py, new test function test_validate_user_with_null_email ." The emphasis on minimal is crucial. The agent should not propose refactoring the entire file, reorganizing imports, or updating unrelated functions. The plan should be the smallest change that achieves the goal. This minimalism reduces risk, makes diffs easier to review, and prevents feature creep. This phase is where human review is most valuable. We can assess whether the proposed changes align with our intent, whether the scope is appropriate, and whether any risks exist (such as touching a critical path or modifying a fragile module). If the plan looks wrong, we can redirect before any code is altered.

Step 3: Waiting for Approval

Before applying any edits, the workflow explicitly pauses and requests approval: text 3) Stop for approval before editing. 3) Stop for approval before editing. This checkpoint is the core safeguard of the review-then-edit pattern. Unlike prompts that execute immediately, this workflow gives us a chance to review the plan and either approve it ("proceed"), reject it ("stop, I'll revise the goal"), or modify it ("approved, but also add a test for the edge case where email is None"). The waiting step makes the workflow interruptible and correctable . If we catch a misunderstanding here, we save the time and complexity of reverting changes, re-running verification, and starting over. The cost of waiting a few seconds for approval is trivial compared to the cost of fixing incorrect edits.

Step 4: Applying Edits with Safeguards

With approval granted, the agent applies the patch under strict constraints: text 4) Edit only src/ and tests/ (ask before touching anything else). 4) Edit only src/ and tests/ (ask before touching anything else). The file restriction is the key safeguard here. By limiting edits to src/ and tests/ directories, we protect build files (Makefile, CMakeLists.txt), dependency manifests (package.json, requirements.txt, Cargo.toml), configuration files (.env, config.yaml), and vendor or generated code. These files are either fragile, managed by tools, or contain sensitive settings; unintended modifications can break builds, introduce security issues, or create difficult-to-debug problems. The phrase "ask before touching anything else" provides an escape hatch: if the goal legitimately requires editing a protected file (such as adding a new dependency), the agent must explicitly ask for permission rather than proceeding automatically. This confirmation ensures deliberate action rather than accidental damage. Tests may write temporary files to verify behavior (such as result.txt in output tests), but these should be created in temporary directories or cleaned up after the test completes. The agent should not create persistent artifacts in the repository root unless explicitly requested. During the edit phase, the agent should apply the changes specified in the plan, no more and no less. If the plan said "add a null check at line 47," the agent should add exactly that, without also reformatting the function, reorganizing imports, or "improving" nearby code.

Step 5: Running Verification

After applying edits, the agent must verify that the changes preserve correctness and meet quality standards: text 5) Verify: (a) quickest safe tests, (b) lint/format/typecheck if present, (c) CI-equivalent if safe. 5) Verify: (a) quickest safe tests, (b) lint/format/typecheck if present, (c) CI-equivalent if safe. The verification phase runs three categories of checks: a) Quickest safe verification — Run the fastest test command that still provides meaningful confidence. The agent should look for standard test patterns in package.json scripts, Makefile targets, or the repository's README . Common patterns include npm test, pnpm test, yarn test, pytest -q, cargo test --lib, or go test ./... . Prefer unit tests or targeted test commands; avoid full builds unless needed. If multiple test commands exist, choose the quickest and explain the choice. b) Lint, format, and type checking — Ensure the code meets project standards. Commands like ruff check, eslint, mypy, cargo clippy, or tsc --noEmit catch style violations, type errors, and common mistakes. Running formatters like black or prettier ensures consistent style. c) CI-equivalent if safe — Run the same checks that continuous integration would execute on a pull request. This might be a Makefile target (make ci), a script (scripts/ci.sh), or a composed command (pytest && mypy && ruff). Running this locally before pushing prevents CI failures. Important: If CI scripts require network access, access external services, or modify files outside src/ and tests/, the agent should describe what will happen and ask for approval before running them. This prevents unexpected side effects during verification. The agent should capture the output of all verification commands and include it in the final report. If any check fails, the agent notes which one, provides the error output, and suggests either a fix or an explanation of why the failure might be expected.

Step 6: Testing Determinism

Beyond basic verification, we want confidence that test results are consistent and reproducible: text 6) Re-run quick verification once; if mismatch, report flakiness and stop. 6) Re-run quick verification once; if mismatch, report flakiness and stop. Determinism matters because flaky tests undermine confidence in verification. If tests pass once but fail on a second run (or vice versa), we cannot trust that our changes are correct; the variance might mask real issues or falsely indicate problems. By re-running the quickest verification command, the agent confirms that results are stable. If both runs produce identical outcomes (all pass or all fail), we have evidence of determinism. If results differ, the agent must report the flakiness and stop — it should not keep retrying in hopes of getting consistent results. Flaky tests are a signal that something deeper is wrong: tests might depend on timing, randomness, external state, or order-dependent side effects. The agent should explicitly state: "Tests are flaky. First run: 42 passed. Second run: 41 passed, 1 failed (test_auth::test_login failed with timeout). This indicates non-deterministic behavior; recommend investigating before proceeding." If the quickest verification command takes more than a few minutes, the agent may skip the determinism check but should note: "Skipped determinism check because verification takes 5 minutes; consider running pytest -q --count=2 manually to verify stability."

Step 7: Generating the Final Report

The last step packages all the workflow outputs into a comprehensive, actionable report: text 7) Report: summary, commands+results, git diff, follow-ups. 7) Report: summary, commands+results, git diff, follow-ups. The summary of changes describes what was modified in plain language: "Added null check in validate_user function; added test case for null email input." This high-level overview helps humans quickly understand the agent's work. The commands run and results section lists every verification command executed and its outcome: text Commands run: - pytest -q: PASS (42 tests in 1.2s) - ruff check .: PASS (no violations) - make ci: PASS (all checks passed) - pytest -q (determinism check): PASS (42 tests in 1.1s) Commands run: - pytest -q: PASS (42 tests in 1.2s) - ruff check .: PASS (no violations) - make ci: PASS (all checks passed) - pytest -q (determinism check): PASS (42 tests in 1.1s) The git diff provides the exact changes made, allowing line-by-line review. This is the authoritative record of what changed; humans can inspect it, apply it to other branches, or revert it if needed. If git diff output is huge (hundreds of lines), use git diff --stat or git diff <path> to scope the report and prevent context overflow. Finally, follow-ups for failures acknowledge when verification does not pass cleanly. If linting fails, the agent might say: "Linting detected unused import in test_auth.py line 5; recommend removing it." If tests fail, it might provide the error traceback and suggest next steps. This turns failures into actionable insights rather than dead ends.

Introduction

The Problem with Direct Edits

Join the 1M+ learners on CodeSignal

The Review-Then-Edit Pattern

Defining the Workflow Structure

Step 1: Clarifying Questions and Assumptions

Step 2: Planning the Patch

Step 3: Waiting for Approval

Step 4: Applying Edits with Safeguards

Step 5: Running Verification

Step 6: Testing Determinism

Step 7: Generating the Final Report

Putting It All Together

Workflow Templates vs Skills

Conclusion and Next Steps