The Edge Probe Harness: Budgeting Honesty Into Live Edge-Case Tests

Most of my test suite never touches a broker. That's by design: unit tests are fast, deterministic, and offline, and they catch the overwhelming majority of what I break. But a broker-facing bot has a category of failure that offline tests structurally cannot reach — the ones that only appear when a real gateway is on the other end of the socket, behaving like real infrastructure: slow, occasionally rude, sometimes silent. This week I built a small framework for exercising exactly those cases against the live gateway, on purpose, under a budget. I'm calling it the Edge Probe Harness.

Why a live harness at all

The failures I most fear aren't logic errors. They're the seams between my code and something I don't control. A response that arrives in a shape my parser didn't anticipate. A precaution the gateway raises that I've never seen because it only fires under a specific gateway-side condition. A call that returns nothing at all, forever, with no error to catch.

A mock can prove my code satisfies a contract I already understand, but it cannot prove the real gateway behaves like my mental model at the edges — because the mock is written from that same mental model. If I misunderstood how the gateway behaves at a boundary, my fake gateway will confidently reproduce my misunderstanding and my test will pass. The only cure is to point a probe at the real thing and watch what actually comes back.

But "point it at the real thing" is dangerous by default. A live probe can hang. It can cost real time on a session I have limited access to. Run enough of them carelessly and the harness becomes its own outage. So the first design principle wasn't what to probe — it was how to make a live probe safe to run at all.

The budget is the feature

The core of the harness is a budget: every probe declares up front how much wall-clock time and how much of my scarce session resource it's allowed to consume, and the framework enforces that ceiling. If a probe exceeds it, the probe is killed and reported as a failure, not left to dangle.

This inverts the usual relationship between a test and time. Normally a test runs until it's done and you hope it's quick. Here, time is the primary constraint and the probe is a tenant inside it. That's what makes running edge cases against live infrastructure defensible: nothing can run away, because the framework holds the clock, not the probe.

I built this in tiers — the essential probes first, the nice-to-have ones behind them — and had the design adversarially reviewed before I wrote the enforcing code, because a budgeting mechanism that has a hole in it is worse than none at all. A hole gives you false confidence that you're protected.

The bug that made it honest

And there was a hole. My first cut put the wall-clock cap around the probe — the actual edge-case interaction I wanted to measure. That felt right. The probe is the thing being tested, so the probe is the thing you time.

Except the probe isn't the first thing that happens. Before any probe can run, the harness has to connect to the gateway. And connecting is precisely one of the operations that can hang indefinitely against real infrastructure — arguably the most likely thing to hang, because it's the very first handshake with a system that might be cold, busy, or wedged.

So my "budgeted" harness had an unbudgeted preamble. If the connect hung, the wall-clock cap never started counting, because it only wrapped the code after the connection was established. The timer was guarding the wrong half of the operation. A probe that hung on connect would sit there past every limit I thought I'd set — the exact failure mode the whole framework existed to prevent.

The fix was conceptually small and honestly a little embarrassing: move the cap outward so it covers the connect itself, and the probe, as one continuous budgeted span. The clock starts when I intend to do work, not when the work finally begins. Then I wrote tests that induce a genuine hang — not a slow response, an actual never-returns — and assert that the harness kills it inside the budget. Real-hang tests, not slow-response tests, because those are different animals and only one of them was covered before.

What I took from it

The lesson isn't "wrap more code in your timeout." It's that a safety mechanism is only as honest as its boundaries, and the boundary you draw reveals what you were subconsciously treating as safe. I'd implicitly assumed connecting was free — instantaneous, reliable, not part of the risky surface. The bug was in that assumption, not in the timer.

Hang detection that doesn't cover the thing most likely to hang isn't hang detection. It's a comforting decoration. Now the budget covers the whole span from intent to result, the framework can genuinely say "this took too long" about any part of the interaction, and I can point probes at real infrastructure without the harness itself becoming the outage I was trying to catch.

Small tool. But now when it says a probe passed, I believe it — including the part where it connected at all.

This is an engineering journal about building an autonomous options-trading bot. Nothing here is financial advice, a recommendation, or a signal of any kind.