An agent follows a reasoning contract — plan → tool → validate → respond. Your dashboard counts spans and reports the validate step ran 100% of the time. But it only checks the span exists, not that it happened in order. Ask the same traces as an ordered funnel and the real number is 64% — validate is firing before the tool it's supposed to check. That 36-point gap is agents shipping unchecked answers, invisible to a counter.
The money moment
One number keeps you asleep. The other is what actually happened.
The funnel
Each bar is the share of traces that reached that step in the contract's order. The break is exactly where a counter goes blind.
Why a counter can't see it
A span can be present and still be wrong. Here's one of the 45 traces the counter waved through.
A counter asks a yes/no question: did the agent.validate span exist?
If yes, it scores the trace healthy — 100%. It has no notion of time, so it can't tell a
validation that checked the tool result from one that ran before the tool even started.
A funnel asks a stricter question: did each step happen after the one before it?
It's the same trace data, read as an ordered sequence. When agent.validate starts before
agent.tool ends, the funnel counts a drop-off — because the contract was broken, even though
every span is technically present.
That's the whole idea: A counter measures presence; a funnel measures sequence.
validate finished before the tool it was checking even started.
The span is present, so the counter scores it 100%. The funnel requires
agent.validate to start after agent.tool ends — so it scores it a drop-off.
Run metrics
Yes — the error count is also 45, so here you could find these traces by
counting errors instead. That only works because this agent labels its
own premature validation: it knows the ground truth because I injected it.
A real agent that validates before its evidence arrives gets a confident
SUPPORTED back and records a success — there is no error to count.
The funnel finds those runs without the agent having to know it went wrong.
Closing the loop
Everything above is a human looking at a chart. The point of measuring a reasoning contract is that the agent can check its own — and it couldn't.
An agent could already reach traces, metrics, logs, dashboards and alerts
through MCP — everything except the one primitive that measures whether it
finished thinking. That count is measured, not claimed: the judge
script completes an MCP handshake against the live server and enumerates
tools/list. Run python scripts/mcp_gap.py --list
to see all 41 yourself, and it fails loudly if SigNoz ever ships funnel tools.
signoz-funnel-mcp ships those five. Point an MCP client at it and
ask "build a funnel over my agent's reasoning steps and tell me which step
is losing traces." The read path makes zero LLM calls, so an agent can
measure itself between turns.