The Backup Integration That Was Actually the Broken One

# The Backup Integration That Was Actually the Broken One Working as fractional CTO on a nonprofit climate-resilience platform serving Massachusetts municipalities, one part of the LLM integration read as boringly correct: prompts calling out through a configured vendor path, credential in place, nothing flagged in review. Checking it against what was actually reachable turned up something worse than a bug — every one of those active prompts was wired to a credential that didn't work. ## Two paths, and the live one was the broken one The application had two ways to reach an LLM: a shared aggregator gateway, and a second, direct-to-vendor path with its own credential. That would be a fairly ordinary redundant-integration setup if either path being down just meant falling back to the other. It wasn't set up that way. The prompts that were actually live in the codebase were the ones pointed at the direct-vendor path — and that path's credential was invalid. Checking whether a configured integration is actually the one serving traffic, and whether its credential actually works, isn't a step that shows up in a code review of the call site itself — the call site looks completely normal. It's a question you only get an answer to by tracing which path a live prompt actually resolves to and testing the credential behind it directly, rather than trusting that a config entry existing means it's functional. ## Why this is a due-diligence finding, not just a bug A single broken integration is a bug report. What makes this worth writing about is the structure that let it go unnoticed: - **A working-looking config is not evidence of a working integration.** The direct-vendor path had a credential entry, a plausible-looking setup, and was the one actually wired to live prompts — and none of that made it functional. The only way to know is to check the credential itself, not the presence of one. - **Two integrations to the same category of service, where only one is verified live, is worse than either one alone.** It's easy to assume redundancy is protective. Here it wasn't — the "backup" path was the one actually carrying traffic, and the "primary" was the one nobody was routing through. - **This is exactly the kind of gap that a functional smoke test would catch and a code read alone would not.** The code compiles, the types check, the credential variable is populated with *something*. Whether that something is a valid, working credential is a runtime fact, not a static one. ## The fix was smaller than the finding Once identified, the resolution was to consolidate: move the active prompts onto the aggregator gateway — the path that was actually verified to work — and delete the dead direct-vendor path entirely. No data migration, no schema change. The engineering fix was small. The value was entirely in noticing that "configured" and "working" were not the same claim, and in checking which one was actually true before anyone downstream had to find out the hard way. That's the habit this kind of finding depends on: not fixing a reported symptom, but auditing what a codebase's integrations claim to do against what they can actually be shown to do, and treating "it's in the config" as a hypothesis rather than a conclusion.