Strangling a 15-Year-Old Monolith: Black-Box Parity Testing in Production

Photo by Ivett M on Unsplash

How we’re migrating a legacy monolith to FastAPI with zero downtime, treating the old system as an oracle instead of documentation.

For the past couple of years I’ve been working on a 15-year-old monolith that powers two high-traffic e-commerce storefronts. It’s the kind of codebase every engineer recognizes: multiple applications tangled together, Python versions frozen in time, and business logic that nobody fully remembers writing but that makes real money every single day.

One of the bigger tasks in that time has been replacing it. The reasons had been piling up for years: security patches no longer land on a stack that old, modern SDKs won’t even install, and with no async IO a long-running request can only be absorbed by massive over-scaling. Moving to an async stack means faster servers, more throughput, and a smaller bill.

A big-bang rewrite was never on the table. Peak traffic events like Black Friday don’t forgive downtime, and the monolith’s behavior is the spec, including its quirks.

We did weigh a more radical option: CDC-stream the data into a new database and build a completely decoupled system on top. As architecture it’s the dream job. Reality voted the other way: tight deadlines, business approval that’s hard to win, and a team of two or three engineers facing millions of lines of code that all have to keep working perfectly. Building that system, hardening it, and getting it to production under those constraints carried a real risk of failing on timeline alone. And timeline is its own constraint: pitch a migration that takes too long and the project doesn’t get approved.

So we went with the strangler fig pattern: extract one endpoint at a time into a new FastAPI async service, prove it behaves identically, shift traffic, repeat.

The interesting part isn’t the pattern itself. It’s how you prove parity when the legacy code is too old and too tangled to trust your own reading of it. Our answer was black-box testing, applied at four different layers.

The core idea: legacy as oracle #

Early on we made a decision that shaped everything else: correctness is defined by what the old system does, not by what we think its code means.

Reading 15-year-old views to reconstruct intent is unreliable. Comments lie, dead branches abound, and behavior often depends on middleware interactions nobody documented. Instead, we treat the legacy system as a black box, a behavioral oracle. If you want to know what an endpoint should return, don’t read the code. Call it.

This reframing turned “do we understand the old system?” (unanswerable) into “do both systems answer the same?” (testable).

The seam: routing without a rewrite #

Strangler fig needs a seam where you can split traffic. We didn’t introduce a new gateway. Instead, we added a small Lua routing layer to the NGINX already sitting in front of the monolith, driven by per-endpoint database switches. Flip a switch on, and traffic for that route goes to the new FastAPI service. Flip it off, and traffic reverts to legacy within minutes, without a deploy.

Two other choices made this cheap:

• Shared data stores. Both systems read and write the same databases and caches, so nothing needs migrating or syncing. The migration moves compute, not data.
• A custom routing header. Requests carrying a special header hit the new service even when its switch is off. This gives us a “dark” lane: the new code runs in production, against production data, while real users never touch it.

Four layers of black-box parity #

There was also a step zero. Fifteen years of A/B tests and branching behaviors had grown into the codebase the way a tree grows around a metal fence: the experiments were long over, but their branches had become part of the structure. You can’t diff two systems that answer differently depending on which experiment bucket a request lands in. So before any parity work, we made every code path addressable with runtime feature overrides that force a request down a chosen branch. We first proved the toggles were safe and nothing critical, since they use the same mechanism as the A/B tests themselves. That’s what makes 100% scenario coverage possible for the layers below.

“Black-box testing” sounds like one thing. In practice we ended up with four distinct layers, each catching a different class of drift.

1. The dual-call differ. Our end-to-end suite, written with WebdriverIO, calls each endpoint twice: once against legacy, once against the new service via the routing header. It then compares the two responses byte by byte. Same request, two black boxes, one diff. This is the closest thing we have to ground truth, and it runs automatically after every deploy, in production.

2. Contract tests with legacy citations. Inside the new codebase, every endpoint is built test-first, and each assertion carries a reference to the legacy behavior it encodes: which filter, which serializer, which default. When a test fails six months later, you know exactly which legacy behavior you’re about to break, instead of arguing about whether the test is even right.

3. Wire-shape snapshots. Some outputs, like image URL formats and response envelopes, are consumed by clients that depend on exact string shapes. We pin those byte-for-byte, snapshot-style, against captured legacy output.

4. Bug-for-bug error parity. The unglamorous layer, and the one that surprised people most. When a downstream service times out, the legacy system returns a 408 with an empty body. Bad API design? Sure. But mobile clients in the wild depend on it, so the new service does it too, and a test pins it.

That last layer needs a counterweight, or you end up faithfully reproducing garbage forever. So we wrote the rule down explicitly: legacy bugs are not requirements. When the old system does something clearly wrong (a 500 where a 400 belongs), the new system may fix it, and we record the difference as a clean divergence: a deliberate, documented improvement rather than a parity miss. The distinction matters because it forces a decision. Every difference between the systems is either a bug in the new code or a documented improvement. There is no third category called “probably fine.”

The cutover gate #

Passing tests isn’t enough to move production traffic. Every endpoint goes through the same release gate:

1. Deploy dark. The new endpoint ships to production with its switch off, reachable only via the routing header.
2. Soak for 24 hours. Dark code shares containers with endpoints already serving live traffic, so a memory leak in “dark” code is not actually dark. We watch error rates, latency, and container memory for a full day before trusting it.
3. Size for peak, not average. We derive each endpoint’s capacity target from its real Black Friday peak traffic, not its average load, and verify with k6 load tests: breakpoint tests to find the per-container ceiling, then soak tests at holiday-level throughput.
4. Validate, then flip. Manual verification through the dark lane, the dual-call differ green, QA sign-off. Then the switch flips and we watch the dashboards.
5. Keep rollback boring. Reverting means turning a switch off, not running a deploy or opening an incident bridge. Because rollback is cheap, flipping the switch is a calm decision instead of a bet.

Where AI fits #

The newest ingredient: AI coding agents are embedded in the migration pipeline, and the black-box discipline is what makes them safe to use.

With capable agents on hand, it was tempting to just spend the tokens and rewrite millions of lines in one shot. We said no. A wholesale rewrite means freezing the business while it happens, which is its own kind of software engineering failure, and no one on either side, human or AI, holds enough knowledge of the system to test every edge case that fifteen years of production traffic has carved into it. The endpoint-by-endpoint gate is slower, but it never asks the business to stop.

We split the work into separate agent tasks with strict boundaries: one plans a migration by running the dual-call differ and tracing legacy behavior (it writes no code), one implements from that plan, one audits parity read-only and produces a go/no-go report with file-and-line evidence from both codebases, and one reviews the pull request.

The agents don’t run on generic prompts, either. Beyond the skills that write tests for full old-versus-new coverage, we encode the things we know matter into dedicated parity skills: small instruction files that carry the team’s hard-won knowledge about where parity actually breaks across the two codebases. A trimmed example:

---
name: parity-check
description: Audit a migrated endpoint against the legacy implementation
---

When comparing an endpoint across the two codebases, always check:

- Side effects: custom metrics and analytics events must match legacy.
  A metric that is dropped, renamed, or downgraded to a log line
  breaks dashboards and monitors. That is a parity miss.
- Feature flags: every branch driven by a flag or A/B test in legacy
  must exist in the new code and default to the same behavior.
- Async IO: independent downstream calls should run concurrently in
  the new stack. Sequential awaits that could be parallel are a finding.

Tag every item: match, miss, clean divergence, or unclear
(needs runtime verification).

It’s a small example, but that’s the point: knowledge that used to live in senior engineers’ heads becomes an executable checklist that every agent applies to every endpoint the same way.

Two guardrails did the heavy lifting. First, explicit bail-out criteria: an agent must stop and escalate to a human when the legacy behavior is ambiguous, when a schema change is needed, or when it has looped on the same problem more than a few times. Second, a rule we borrowed from how we treat any junior engineer’s findings: agent claims are hypotheses until independently verified. We only trust a parity report after a human (or a separate verification pass) re-checks every claimed mismatch against both codebases.

AI didn’t lower our safety bar. The oracle, the differ, and the gate are what let us hand real migration work to AI at all.

Lessons #

• Define correctness by observation, not comprehension. You will misread old code. You won’t misread a response diff.
• Legacy bugs are not requirements, but make divergence explicit. Every difference is a bug or a documented improvement; nothing gets to sit silently in between.
• Dark code isn’t dark on shared infrastructure. It shares containers, connection pools, and memory with live traffic. Soak it like it’s live, because it is.
• Size for peak, not average. The endpoint that idles at low traffic in March is the one that takes an order of magnitude more on Black Friday.
• Make rollback so boring it’s not a decision. A switch flip beats a redeploy, and a redeploy beats a war room.
• Not every endpoint is a perfectly deterministic black box. Recommendation vendors return different results on every call, and replaying paginated requests against a metered third-party API burns real quota. For those endpoints, compare structure and invariants (shape, counts, ranges) instead of exact bytes, and budget the replay traffic like you’d budget a load test.
• Don’t redesign mid-migration. Every cut-over tempts you to improve old patterns along the way (swapping backend sessions for stateless JWTs, say) with no obvious business win. Hidden business rules and other teams’ measurements hang off the current behavior, and every extra change drifts you from the finish line. Move to the new stack first; modernize on top of it in the next iteration.
• AI agents need an oracle and an exit. Give them a source of ground truth to test against, explicit conditions to stop and escalate, and verify their claims like you’d verify anyone else’s.

The read-heavy catalog surface is fully live on the new stack, with write paths following behind. The clearest win is capacity: search components that needed around 200 containers on the old stack now run on 3, thanks to async IO, a newer Python, and code that doesn’t leak memory. The pattern has already paid for itself: every cutover so far has shipped with zero downtime, and every rollback has been a non-event.

That’s what black-box migration buys you: not certainty that the new code is right, but certainty that it behaves the same, except where you decided otherwise and wrote it down.