Quick translation if you don’t build AI agents for a living: think of an AI agent as a program that can read requests, decide what to do, and take multi-step actions on its own (send a message, look something up, run a task) instead of just answering one question at a time. This post is about a single missing setting that silently broke one of ours — and the same mistake can happen in any piece of software with a hidden dependency like this. You don’t need to know how AI works to get the lesson.
We run a small fleet of AI agents — one per client, each isolated in its own environment with its own credentials and its own memory. This week, we upgraded one of them to a newer model version. Within minutes, it stopped responding to anything with a generic error: “Something went wrong while processing your request.”
No stack trace visible to the user. No obvious cause. Just a wall.
This is a short field note on how we found the actual bug in under ten minutes, and why it’s a good example of a class of failure that’s becoming more common as more people run their own AI agent infrastructure: the silent adapter mismatch.
The Symptom
The agent (nicknamed “Cleo” internally) had just been switched from an older Claude model to a newer one. Immediately after, every message produced the same generic failure. The obvious first suspects:
- Bad model name
- Expired or wrong API key
- Rate limiting
All three checked out clean. The API key worked. The model name existed. There was no rate-limit signal anywhere. This is the frustrating middle ground of debugging — where all the “usual suspects” are innocent and you have to go one layer deeper.
Reading the Actual Logs
Instead of guessing, we went straight to the gateway’s structured logs — timestamped, machine-readable events for every request. The key line looked like this (paraphrased):
error: All models failed (2):
anthropic/claude-[model]: 404 status code (no body) (model_not_found)
anthropic/claude-[fallback-model]: can't find the model you're using right now.Two things stood out immediately:
- A 404, not a 401 or 403. Authentication was fine — the request never even found a matching route on the provider’s server. A 404 on a known-good model name usually means the request went to the wrong URL, not that the model doesn’t exist.
(Quick gloss: these numbers are HTTP status codes — the universal “how did that request go?” codes every website and app uses. 401/403 mean “you’re not allowed in.” 404 means “that address doesn’t exist.” Different numbers, completely different problems.)
- The fallback model failed too, with a slightly different error: “provider is in cooldown.” The system had automatically flagged the whole provider as unhealthy after repeated failures — a self-protective circuit breaker kicking in. This masked the real, simpler root cause behind a secondary symptom.
The Root Cause
Modern AI gateways (the piece of software that routes your requests to whichever AI provider — OpenAI, Anthropic, Google — you’ve configured) don’t talk to every model provider the same way. Even providers using ostensibly similar request formats (JSON in, JSON out — a common structured text format software uses to pass data around) have different web addresses to send requests to, different required headers, and different response shapes. A gateway needs to know which API adapter to use for a given provider — think of it as a translator that knows which “dialect” each provider speaks — essentially, “speak this dialect, not that one.”
In this case, the agent’s configuration was missing one field: an explicit adapter declaration (a setting saying “use the Anthropic translator”) for the Anthropic provider. Without it, the system fell back to a default adapter — one built for a different vendor’s chat API. That adapter constructs requests differently and points at a different web address entirely.
The result: every request was formatted correctly in isolation, authenticated correctly, and referenced a real, valid model — but it was being sent to a web address that simply doesn’t exist for that provider. Hence the 404. Hence “model not found,” even though the model was perfectly real.
The fix was one line: explicitly declaring which API adapter that provider should use, restoring the correct endpoint routing.
Why This Matters Beyond Our Setup
If you’re running any kind of multi-provider AI orchestration — routing between OpenAI, Anthropic, Google, or local models — this class of bug is worth knowing about, because it’s easy to trigger and hard to spot:
- It doesn’t happen on day one. It happens when you change something — a model upgrade, a config migration, a copy-pasted provider block from a template that assumed a different default.
- The error message lies to you. “Model not found” sounds like a typo in the model name. It’s not. It’s a routing problem wearing a model-name costume.
- Automatic fallback and cooldown logic can hide the real signal. Good systems protect themselves by suspending a misbehaving provider after repeated failures. That’s the right behavior — but it means your second error message describes the protection mechanism, not the original bug. Always trace back to the first failure in a chain, not the last.
The Debugging Checklist We Actually Used
For anyone hitting a similar wall with self-hosted or custom-configured AI agents:
- Rule out the obvious two first: valid API key, valid model name. Test the key directly against the provider’s API outside your application if you can.
- Read the raw error code, not just the message. 404 vs. 401 vs. 429 vs. 500 tell very different stories. A 404 on a model call almost always means “wrong endpoint,” not “wrong model.”
- Check when the config last changed relative to when the failures started. If a model swap or config edit lines up exactly with the first error timestamp, that’s your prime suspect — not something unrelated.
- Look for adapter/protocol fields, not just model names and keys, in any provider configuration block. Every provider entry needs to know how to talk, not just who to talk to.
- Compare against a known-working config for the same provider elsewhere in your fleet, if you have one. A diff often finds in seconds what an hour of log-reading might miss.
- Apply the fix, then verify against a fresh request — not just “no more errors in the logs” or a “reload applied” message, but an actual successful round trip. Some config changes need a full restart even when the system claims a live reload succeeded.
The Fix Itself (And a Second Lesson We Didn’t Expect)
We applied the config change, and the system’s own logs reported “config hot reload applied” for the adapter field. Good sign, right? Except when we sent a live test message right after, it failed the exact same way. The reload event had fired — but the actual request-building logic underneath was still using the old adapter until the process was fully restarted.
That’s a subtle trap worth calling out: not everything that logs “applied” is actually applied where it matters. Some configuration values live at a level deep enough in a running process (things baked into request builders, transport clients, or connection pools at startup) that a live reload can update the stored config value without updating the code path that consumes it. The fix in that case: do a full restart, then verify with a real request — not by reading the log line that says success.
Total time from error report to fully confirmed fix: about 15 minutes, including one false-positive “fixed” moment that required a follow-up restart.
The lesson isn’t really about this one field. It’s about treating your AI agent stack like real infrastructure — with structured logs, config version control, and a habit of comparing against known-good baselines — rather than a black box you just restart and hope. It’s also a reminder that “the log said it worked” is not the same as verified. Trust the test request, not the reload message. The same debugging discipline that works for a web server or a database applies here. AI agents are software. Debug them like software — and always confirm the fix with a live check, not a log line.
Update: The Same Symptom Came Back — From a Different Field
A few days after this fix shipped, the identical user-facing symptom returned: generic “LLM request failed,” no visible cause. Same agent, same provider, same error class — but a completely different missing field this time.
The adapter field from the original bug was still correctly in place. This time, the provider’s model list was missing an explicit max output tokens value on each model entry. Without it, the request-building layer had nothing to send for that parameter, and the provider rejected the call outright — a different failure mode with the same generic error message wrapped around it on the receiving end.
We applied the same debugging checklist from above almost verbatim: confirmed the API key and model name were fine, read the raw provider response instead of the generic wrapper message, diffed the provider config block against a known-good sibling agent’s config, found the missing field, and added an explicit token-limit default per model.
Then — learning fully applied from the first incident — we didn’t trust the config reload log line or the restart’s exit code as proof. We sent a real, live request through the fixed model configuration and confirmed an actual successful round trip with real response content before calling it closed.
Two incidents, two different missing fields, same root category of bug: a provider configuration block that looked complete at a glance but was missing one field the request-building layer silently depended on. Neither omission caused an error on save. Neither was caught by a syntax check. Both only surfaced the moment a real request tried to flow through the gap — and both wore the exact same generic “request failed” mask on the way out.
The pattern holding across both incidents: generic errors plus recent config change equals “go read the provider config block field-by-field,” not “check the API key again.” And in both cases, the fix wasn’t confirmed until a live request proved it — not a log line, not a clean restart, not a lack of errors. An actual round trip, checked directly.
Field note from building and operating a small fleet of AI agents for consulting clients. If you’re running your own AI orchestration layer and hit something similarly opaque, the fix is almost always closer to “check the plumbing” than “the AI is broken.”