Upgrading Laravel 5.5 to 13 without changing a single API response

A frontend we don't own talks to an API we do. The API ran on Laravel 5.5 and PHP 7.3. It had to end up on Laravel 13 and PHP 8.3, in about fifteen steps, and the other company's frontend had to keep working without a single change on their side.

That last requirement is the hard one. "The tests pass" is not the same as "every response is byte-for-byte what it was". This post is about how we closed that gap, what kinds of differences actually show up during a long framework upgrade, and the tool we built for it, DriftGate, which is now open source.

The problem with "it still works"

A framework upgrade across that many versions changes things nobody asked for:

• a database driver starts returning native integers, so "42" becomes 42;
• the JSON encoder stops escaping slashes, or starts writing 1.0 where it wrote 1;
• a serializer changes the order of keys or drops a null field;
• a validation rule behaves differently and a 422 becomes a 200 (or the other way round);
• a cookie or cache header changes;
• an endpoint keeps the same response but starts writing to a different table.

Feature tests rarely catch these. They assert what the developer thought mattered when the test was written, and they usually compare by meaning: assertJson(['id' => 42]) is happy with both 42 and "42". A JavaScript frontend written by another team is not always that forgiving.

Here is one from our upgrade. The login response includes the token lifetime, which comes from .env, so it was the string "60". Carbon 3 accepts only numbers in addMinutes(), and every login started failing. The obvious fix, casting the value to an integer, made login work again, but the response now said 60 instead of "60". The feature test checks that login succeeds, so it stayed green. A JavaScript client doing ttl + 1 or a typed mobile client gets a different type. DriftGate reported it as type string → int, and the final fix changed only the internal use and left the response as it was.

Others were quieter. Laravel 11 fills missing config keys with defaults, so the session cookie gained samesite=lax on all 622 responses while every test passed. Since Laravel 6, PDO sends floats as strings, so MySQL rounded 792.5 up instead of to even, and one value in the API changed from 792 to 793.

What we needed was simpler and stricter: record what every route returns today, and after every upgrade step prove that it still returns exactly that, on the same data.

The approach

The idea is old: golden-master, or snapshot testing, applied to a running HTTP API. The details are what make it usable on a real project.

1. Same data every time. Every run starts from a restored copy of the database (a pristine snapshot), so a difference in a response is a difference in the code, not in the data. DriftGate also takes a fingerprint of the database after each request, so it notices when a request changes what it writes, even if its response stays the same. A GET that suddenly updates a row is drift.

2. Literal comparison. Status, headers and body are compared literally: the JSON type and the written form of every value, the key order, the encoding. When something differs you get the path and the kind of difference, not a wall of diff:

DRIFT get-api-news-id header-extra header:x-powered-by → ["demo"] key-extra $.entry.meta → (private) type $.entry.id string → int 

3. No flaky baselines. The baseline is captured twice on the same restored state and only accepted when both captures agree. Values that legitimately change between runs (timestamps, tokens) are handled explicitly, by format, instead of being silently ignored.

4. Every route covered. The tool reads the application's own route list (for Laravel, route:list) and proposes a request for each route. driftgate coverage fails while a route has neither a request nor an exclusion with a reason, so nothing slips through because nobody thought of it.

5. One step at a time. One version per branch: 5.5 → 5.6, and so on. On each branch: upgrade, driftgate run, fix what drifted (usually a cast or an old serializer setting), repeat until it's clean, then open the pull request. When a change was intended, it gets accepted with a reason and a ticket reference, and that is logged next to the baseline:

driftgate accept group:3 --reason "Dates are ISO 8601 since Laravel 7; clients updated" --ref APP-512 

By the end, the acceptance log explains every difference between the old and the new API, and nothing else changed.

The steps that produced the most drift:

• Laravel 5.6: the shape of the error body changed; JSON clients got a generic error instead of a 429.
• Laravel 6: floats sent to the database as strings, so rounding changed.
• Laravel 7: Symfony 5 removed getClientSize(), so every file upload returned 500.
• Laravel 8: new HTML error pages for 404, 419, 429 and 500 (31 requests).
• PHP 8.0: method_exists(null) throws a TypeError, so one capture returned 500.
• Laravel 11: config defaults changed headers on every response.
• Carbon 3: numbers as strings are rejected, and diffIn* now returns a signed float.
• Laravel 13: performance. One 116 MB list went over the 30 s limit; after the fix, CPU time dropped from 30.4 s to 18.4 s.

Two findings had nothing to do with the upgrade itself. Capturing with four parallel requests showed that after view:clear, concurrent first requests collide while compiling Blade templates: 5 of 24 got the wrong error page (fixed with artisan view:cache once before capturing). And after a table is restored, InnoDB recalculates its statistics about ten seconds later, so a query in that window can return the same rows in a different order. We confirmed it on MariaDB 10.3 and MySQL 8.4.

Security fixes made after the upgrade benefited too. DriftGate measured exactly how many responses changed on purpose, so "only what we meant to change, changed" could be reviewed: 102 of 622 requests where one field became null, 15 with a smaller nested object, 8 with only new rate-limit headers.

Personal data

The API serves real user data, and a baseline made from it ends up close to git. DriftGate scans every response for personal data (emails, phone numbers, IBANs, national ID numbers and similar). A response that holds any is stored encrypted on the machine that captured it, and the repository refers to it only by a keyed hash. Reports never print those values. For test data there is driftgate data anonymize, which replaces personal values consistently across tables, so a dump can be shared with the team.

Results

• Two production Laravel APIs: one from 5.5 to 13 in about 15 steps (622 requests), one from 6 to 13 in about 12 steps (494 requests). Each step was closed only at 622/622 and 494/494 identical responses. DriftGate reached the same verdict as the internal script we used before it, on every request.
• More than ten regressions that the feature tests did not see were caught and fixed before the final pull request. None reached production.
• On the real application, a full capture of 622 requests takes 148.9 s with 44 MB of memory (the time is the application's own), and the comparison takes 0.26 s. The script we used before needed about four minutes per capture on the same project.
• It works over HTTP, so it doesn't care what the application is written in. Besides Laravel it finds routes in Symfony, Rails, Django, Spring Boot, Next.js, Nuxt and anything that serves an OpenAPI description.

What it is not

It's not a replacement for unit and feature tests: it knows nothing about your code from the inside. It's not a load-testing tool or a production monitor; by default it only talks to local and private addresses, and requests that write to a remote host need an explicit flag. And it doesn't check how a page looks in a browser, only what the server sends.

Try it

DriftGate is a single static binary for macOS, Linux and Windows, Apache 2.0, with signed releases:

go install github.com/orangehill/driftgate/cmd/driftgate@latest cd your-laravel-app # with the app running driftgate setup # finds routes, database and address, makes the first baseline # ... upgrade one step ... driftgate run # exit code 1 and a report if anything drifted 

There is also a GitHub Action, if you want CI to run the same check on every pull request of the upgrade.

The repository is at github.com/orangehill/driftgate. If you're in the middle of a framework upgrade, we'd like to hear what drifted for you, and what DriftGate missed.