All posts
debuggingproductionauthbackend

The Bug I Couldn't Reproduce

One user. One Apple ID. userId returning 0 in production. Here's how I found it without being able to trigger it myself.

December 4, 20256 min read

The ticket came in on a Tuesday morning. A specific user — one account, one device, one Apple ID — was getting assigned userId 0 after logging in through Apple SSO. In our system, userId 0 is the super_admin. They were seeing data they had no business seeing.

I could not reproduce it. Nobody on the team could. The user was the only one hitting it, and we couldn't ask them to trigger a production auth issue on demand just so we could watch the logs.

What I knew going in

The Apple SSO flow runs through a standard PKCE exchange. We validate the identity token, extract the Apple user ID (`sub`), look that up in our users table, and issue a JWT. The getUser query predated some recent auth service changes. That gap was the first thing I flagged.

We had also just shipped soft-delete. When a user deletes their account, we set `isactive = 0` rather than wiping the row — so their data can be recovered for 30 days. The auth flow needed to treat a soft-deleted account as non-existent.

Bug

The getUser query used by the Apple SSO path was the old version. It was missing the `WHERE isactive = 1` filter that the refactored auth service had added everywhere else. A soft-deleted user matched as valid — and their auto-increment PK happened to collide with the super_admin lookup fallback, returning userId 0.

How I found it

I added trace logs at every branch of the Apple SSO if-conditions — before each DB call, after each DB call, what was returned, what was matched. Then I waited. The logs shipped in the next deploy. The user triggered the flow the next day. The logs came back.

The trace showed getUser being called with the Apple `sub` value, returning a row — but then the next step showed a userId of 0, not the returned row's ID. That told me: the query found a row, but something in the mapping discarded the real ID and fell through to a default.

I pulled up the old getUser SQL. No `isactive` filter. The soft-deleted row matched. The mapping layer had a zero-check: `if userId == 0 { return errUserNotFound }` — but it was checking after the assignment, and the assignment was pulling the wrong field.

sql
-- old query (missing soft-delete filter)
SELECT id, email, role
FROM users
WHERE apple_sub = $1

-- fixed query
SELECT id, email, role
FROM users
WHERE apple_sub = $1
  AND isactive = 1

Then the second bug landed

Twenty minutes after the fix deployed, the same user couldn't log in at all. A different error: login blocked.

We had a Redis-based login guard: after logout, we block logins using the same token for 30 minutes, keyed by the token's `tid` (token ID) claim. The idea was to prevent replay attacks — if someone sniffs a token during the logout window, they can't use it.

Bug

Apple SSO returns the same `tid` value for the same Apple ID on every authentication. Google SSO returns a random one. So the Apple user's logout was blocking their next login — the tid they logged out with was the same tid they were trying to log in with.

Google users never hit this because their next-login tid was always different. Apple users hit it every single time — and silently, because the error was generic.

The fix

Key the block on `{userId}:{tid}`, not just `tid`. A logout by user A can't block login for user B, and the same tid reused by the same user after 30 minutes is fine — because the userId changed after re-authentication creates a new session.

Two bugs, same user, same day. The first was a query that never got updated. The second was an assumption baked into token design that was only true for one provider.

What I took from it

  • ›When auth service changes, audit every query path in that service — not just the refactored ones.
  • ›If you have multiple SSO providers, verify every assumption you've made about tokens is true for all of them.
  • ›Production debugging without repro means logs first, everything else second. The logs found this in one cycle. No other approach would have been faster.
  • ›Most auth bugs live at the seams between systems that were individually correct.