Handling Writable Backpressure
Respecting Writable Backpressure with drain
Hello, stream mechanic! 🔧 In the previous lesson, you built a pull-driven readable source that stops producing when push() reports readable-side pressure and resumes from the correct next record. That completed source stays unchanged here. Now the focus moves downstream: a slow writable can fill its own buffer, so the code consuming the source must honor write(), wait for "drain", and wait for destination completion before reporting results.
The completed diagnostic endpoint is:
Lab-only diagnostic: This intentionally slow endpoint has no deadline, disconnect cancellation, or concurrency control. Do not expose it publicly; a real service needs bounded aggregate work plus timeouts, cancellation, and rate or concurrency limits.
It returns JSON with manual-pump metrics. The provided Backpressure Lab control invokes that same backend request and displays the result, but the work here is entirely in the backend.
A successful constrained request reports:
mode: "manual"sinkHighWaterMark: 64- positive processed
chunksandbytes - positive
backpressureWaits
The exact number of drain events, waits, buffered bytes, and elapsed milliseconds can vary by run. They are observations, not fixed response values.
Establish the Pressure Model First
Before following the HTTP route, separate the two pressure signals: push() returning false pauses producer-side filling of the readable queue, while write() returning false requires the destination caller to pause until it receives "drain".
Register the Diagnostic Route
The router must do two jobs for the new path:
- Recognize
/api/lab/backpressureas a known route so unsupported methods receive the established405response. - Dispatch valid
GETrequests to the route handler while preserving the original URL for query parsing.
The first part of backend/src/http/router.ts imports the handler and declares the router-local path constant.
runBackpressurebelongs in a route module because it translates an HTTP request into a configured stream experiment. The router only selects handlers.BACKPRESSURE_PATHfollows the existing local-path convention. It is not added to the sharedAPI_PATHSobject.RouterDependenciesdoes not change. The new route does not need the database, sample path, or start time directly.- The router will match a parsed pathname but still pass
request.urlto handlers that need query parameters.
The next source-order region defines the known paths and applies the GET-only policy before dispatching any route.
new URL(...).pathnameremoves query parameters for routing. Both/api/lab/backpressureand/api/lab/backpressure?records=80match the same pathname.- Adding
BACKPRESSURE_PATHtoknownPathspreserves method semantics: aPOSTto this real route returns405, not404. - The
{ allow: "GET" }argument becomes the response’sAllow: GETheader. - Since the method check runs before dispatch, non-GET requests never construct streams or validate experiment controls.
The final region of backend/src/http/router.ts preserves existing routes and adds the backpressure branch.
- The backpressure branch receives
responseto send JSON andrequest.urlto parserecords,delayMs, andsinkHwm. - Existing generated and transform branches remain intact. Extending a router must not shadow or remove established behavior.
- The final
404now applies only to unmatched paths; known paths with unsupported methods already returned405. - Successful routing alone does not implement flow control. It only reaches the code that creates and coordinates the streams.
