Verifying Real APIs
Introduction
Welcome back to Review AI-Assisted Python Changes. This is lesson 3, and we are building on our work reviewing and dividing AI-generated changes. A method name can look perfectly reasonable and still not exist in the project. In this lesson, we will check a proposed ticket resolution call against the actual code, show what fails, and then use a focused test to protect the behavior we really need.
The Problem: Plausible Is Not the Same as Real
A generated method call can sound right, accept the expected argument, and read naturally while still being absent from the codebase. If we trust how it looks instead of checking what is defined, the change may not run at all.
As in earlier lessons, we will rely on evidence rather than intuition. Here, that means comparing the proposal with the real Ticket methods and tracing what happens when the proposed call runs. The code, not the method’s convincing name, tells us whether the call is valid.
The Running Example: A Generated Resolution Patch
Imagine a patch that promises two effects when we resolve a support ticket: it records a resolution note and closes the ticket. The generated patch tries to combine those effects in one call:
The name suggests a convenient operation, but names are not proof that a method exists. Before accepting the patch, we need to find the Ticket definition and check whether it provides this call. If it does not, we must find out how to achieve both promised effects using the actual code.
Inspecting the Actual Ticket API
In app/tickets.py, the Ticket constructor stores the ticket ID and title, starts with an empty list of notes, and sets the status to "open". Its methods provide separate operations for the two effects we need:
add_note stores text in notes, while close changes the status. The same file’s create_ticket helper creates a Ticket, stores it in the provided collection under its ID, and returns it. These are real parts of the contract. There is no resolve_with_note method in this class.
Proving the Failure
Now we can trace the proposed call against the actual class. Creating a ticket gives us an object with the methods we just inspected, but calling a name that the object does not define stops execution:
Because Ticket has no resolve_with_note method, Python raises an AttributeError at that call. Execution stops there, so neither the note nor the closed status is produced by this proposed operation.
This is a concrete failure, not a concern based only on naming or style. We have shown that the generated call is not supported by the project’s API and that its promised effects do not happen.
Composing Resolution from Real Operations
We can still provide both effects without adding an invented method to Ticket. In app/tickets.py, resolve_ticket acts as a small coordinator: it uses the existing note and status operations in order.
First, the resolution is stored as a note; then the ticket is closed. resolve_ticket is a new function, but it is not a new Ticket method: it only combines operations the class already has, so the existing Ticket contract stays exactly as it was. This is the YAGNI idea, “you aren’t gonna need it,” applied to the contract: we do not widen what Ticket promises, for example with a resolve_with_note method, until a requirement needs it. Recording a resolution and closing the ticket is needed now, so this small helper is justified. As reviewers, this is the shape of revision we ask the author for; the next step gives us a way to check that the revision they send actually delivers it.
Locking In Behavior with a Focused Test
In tests/test_tickets.py, the focused test creates a ticket, resolves it, and checks both observable effects. Like the tests in earlier lessons, it is a plain pytest function that uses assert:
The first assertion checks that the exact resolution message was stored; the second checks that the status changed. Keeping both checks in this test matters: an implementation that only adds the note or only closes the ticket will fail. The test provides focused evidence for the complete behavior rather than simply showing that the function ran, which makes it a reliable way to re-review the author’s revision.
Demonstrating and Documenting the Corrected Flow
In main.py, the example creates the “Reset password” ticket, resolves it, and displays its ID, status, and stored resolution. Running python3 solution.py calls main() through the small solution.py entry point, which runs this code:
The program displays the ticket’s closed status and its saved note:
app/__init__.py is an empty package marker. REVIEW.md records that resolve_with_note is absent, that the generated call causes an attribute error, and that the required revision should use add_note and close. It also points to the focused test as evidence for both effects. Running python3 -m pytest checks that behavior.
Conclusion and Next Steps
We have followed a practical evidence trail: inspect the real API, prove that the invented call fails, identify the required behavior from existing operations, and use one focused test to check both effects. The runnable example and review record make the corrected flow easier to verify and understand. Next, we will practice tracing the API, writing the resolution test, and then using that test to re-review the author’s fix before approving it.
