Fisher is Molt AI’s adversarial testing platform for tool-using agents. The paper explains how Fisher finds cognitive vulnerabilities, proves what happened, tests the fix, and learns from every authorized campaign.
Tool-using agents hold standing access to files, mail, databases, and internal APIs. A hidden failure lives in the testing tools themselves: most evaluation grades the agent’s final text, and a fluent refusal can conceal tool calls that already executed the forbidden task. The paper documents this blind spot with a controlled head-to-head on an identical database agent, where transcript grading passed sessions whose action traces showed protected records had been queried.
It then describes the alternative: an assurance loop — Probe → Prove → Replay → Remediate → Re-attack → Learn — run as adaptive, multi-turn campaigns against an explicit target contract. Where the target exposes action evidence, Fisher evaluates tool calls, retrievals, writes, and boundary crossings. Findings carry replay provenance, remediation verdicts are recorded as fixed, partial, bypassed, or unknown, and the matched adaptive experiments measured a 6.6 to 14.6 percentage-point uplift over non-adaptive testing.
Why the attack surface of a tool-using agent is cognitive · the Probe → Prove → Replay → Remediate → Re-attack → Learn loop · action-level proof and evidence levels · exact, guided, and free replay provenance · remediation re-attack and the fixed / partial / bypassed / unknown verdicts · the strategy flywheel and authorized learning · the matched adaptive experiments · framework mapping as governance context, not certification.