Reliability engineering
Agent Preflight
A pre-production check for a voice agent’s webhook tools and the n8n workflows behind them. It prints what will break, and fails the build when it matters.
01 / THE PROBLEM
Why this matters.
A voice agent fails in production for dull reasons: a tool still points at a test webhook, a parameter is called phone on one side and phone_number on the other, a workflow is inactive, a secret is still the placeholder. None of that shows up until a caller is on the line, and none of it is visible from either side alone — the agent’s tools and the workflows behind them are two halves of a contract nobody checks.
02 / THE DIRECTION
The solution.
A checker that reads both halves and compares them. It loads tools from files, an agent export or the API, indexes the n8n webhooks, follows the nodes downstream of each, and reports every mismatch as a blocker or a warning. Optionally it calls each webhook for real — read-only unless writes are explicitly allowed. Blockers exit non-zero, so it can gate CI.
My role.
Designed and built the checker, the check set, the live probe, the web interface and the fake n8n server used to test it.
03 / ARCHITECTURE
How the pieces connect.
System flow
- 01Agent tools
- 02n8n workflows
- 03URL · auth · schema checks
- 04Contract match
- 05Optional live probe
- 06Report · CI gate
04 / ENGINEERING DECISIONS
Designed with intent.
- Separate blockers from warnings, so a release stops for a broken contract and not for a style note.
- Default the live probe to read-only, and skip tools that change data unless writes are explicitly allowed.
- Use API keys for a single run and never store them.
- Ship a stand-in n8n server that can be told to misbehave, so the checker’s own failure handling is testable.
- Keep it to the standard library, so it runs anywhere without a dependency install.
05 / EVALUATION APPROACH
What needs to hold true.
- 34 tests, including a deliberately broken copy of a real agent’s config with seven planted faults, all of which must be caught.
- A stand-in n8n server misbehaves on command — slow, HTTP 500, oversized reply, plain text instead of JSON, missing workflow — and the probe path is proven end to end against it.
- A parameter mismatch between a tool and the workflow that reads it must be reported, in both directions.
- A blocker must set the exit code, because a report nobody fails on is a report nobody reads.
- Not yet run against a live ElevenLabs agent or n8n instance — that needs production keys, and the read-only run is documented rather than claimed.
Technology direction.
Results
- Run against my own bank agent, it found real problems: every tool file still carrying a placeholder secret id, an unfilled PASTE_…_SECRET_HERE in the post-call workflow, and a field one tool sends that no node downstream ever reads.
- Catches test-webhook URLs, placeholder secrets, missing or inactive workflows, and parameter mismatches before release.
- Runs as a CLI, in CI, or as a web app where a tools folder and an n8n export can be checked in the browser.
Lessons learned
- The failures worth catching are contract failures between two systems, which is exactly what neither system can see on its own.