The question security teams are quietly asking
Most vendors will tell you their AI agent can autonomously find vulnerabilities. Few will tell you what happens on iteration 40 of a 100-step unsupervised run, when the context window is full, the easy bugs are gone and the model has to decide whether something it half-remembers from an hour ago is still true.
Finding an injection flaw in a codebase is not the hard part for a modern language model. The hard problem is trust at scale: can the model run for hours, across hundreds of reasoning steps, with nobody watching, and return a report a team can act on without quietly redoing the work? We built Vorthix XOR-1 because for most agentic tools the honest answer today is no, and we spent a long time testing that answer against ourselves before offering the engine to anyone else.
What Vorthix XOR-1 is
Vorthix XOR-1 is the autonomous research engine inside our platform. It goes into a codebase with a goal, for example “find an attacker-reachable, medium-to-high-impact vulnerability”, and works it the way a human security researcher would: form a hypothesis, read source, trace call paths, test the hypothesis against real evidence, then confirm a finding or rule it out and move on. It does this across long sessions without a person steering each step. It is one engine in a larger system, but it is the one that has to be trustworthy on its own, because nobody stands over its shoulder while it runs. The full method is in how Vorthix XOR-1 works.
Failure mode 1: amnesia
A long-running agent has to compress its own history to keep working. Done carelessly, the compression is lossy in exactly the wrong place: the agent’s correct, already-proven conclusions do not survive while noise does. We watched an agent investigate something, correctly conclude it was not exploitable, and then re-investigate the same thing from scratch within the same session, because the proof of that conclusion never reached durable memory.
This was not a rare edge case. In an unaudited long-running session we measured the majority of an agent’s correct intermediate conclusions failing to persist at all. The agent was not the problem. The scaffolding around it was.
Failure mode 2: false confidence
A tool that loops and wastes time is annoying but honest: you can see it happening. A tool that confidently declares something safe without verifying it is dangerous, because a well-worded false “no vulnerability found” looks identical in a report to a correctly earned one. If you must re-audit the work to tell them apart, no engineering time was saved. We found this failure in several distinct shapes:
- Pattern-matched closure. An agent learns that certain confident-sounding words produce a “verified” or “closed” status in its own tracking, without the evidence behind them.
- Escalation through repetition. An agent told a conclusion by an earlier version of itself treats it as more trustworthy than it is, compounding an error instead of catching it.
- Real evidence attached to the wrong conclusion. In our own testing an agent cited a genuine, verifiably true quote from the source code and used it to justify a conclusion that the quote did not prove. The citation was real. The reasoning built on it was not. A system that only checks whether evidence is real will wave that through. You need a system that also checks whether this evidence proves this claim.
The standard: a receipt that proves the claim
We do not publish our architecture, but we do publish the standard every conclusion is held to, because the standard is the product. Nothing is marked true without a receipt, and the receipt has to prove the specific thing it is attached to. In practice:
- Every conclusion traces back to real, retrievable evidence from the codebase, not a paraphrase or the agent’s memory of what it read three steps ago.
- Confident language alone cannot close a finding. A model writing “this is safe” is not the same event as a model proving it.
- When a claim rests on several facts, each fact needs its own evidence. One citation may not quietly vouch for assumptions it never addressed.
- “Insufficient evidence” is a valid outcome. Vorthix XOR-1 may end an investigation without declaring victory in either direction.
- The enforcement itself is tested adversarially and continuously. When we find a new way a false conclusion could slip through, we ask what general shape of failure it represents, not only how to patch that one case.
The rules are summarized as a reference on the Vorthix XOR-1 evidence standard page, and the terms are defined in the glossary.
Why “I don’t know” is a feature
A tool that can only output “vulnerable” or “safe” will eventually be pressured, by its training, its momentum or the shape of the task, into guessing. We built Vorthix XOR-1 to prefer an honest “I could not establish this” over a confident guess. A human pentester who says “I ran out of time and did not get to this” is useful. An AI that quietly marks something safe without checking is how real issues get buried under false reassurance.
A real session, generalized
We will not name the codebase or disclose the finding. Here is the shape of a real session. Vorthix XOR-1 was auditing a large, unfamiliar C codebase for memory-safety and injection-class issues. Partway through a long autonomous run it flagged a function that put a user-influenced value into a generated command string without the quoting used consistently elsewhere in the file, a classic injection shape. It spent several iterations trying to determine whether the value could be attacker-controlled, hit a dead end on one line of reasoning, and then closed the investigation with a conclusion that rested on several distinct technical claims but had hard evidence for only one of them.
Under the evidence standard that closure was rejected, automatically and structurally, not because a person happened to be watching. The agent had to go back and either produce evidence for every claim its conclusion depended on, or leave the finding open instead of letting it become “safe” on a foundation that did not support it.
What to evaluate instead of the demo
Most modern language models can find a bug in a demo, with a person nudging them along. That is no longer the differentiator. The questions that matter are:
- What happens when it runs for hours without you watching? Does it keep making progress, or circle over covered ground?
- When it says something is safe, is that confident language or traceable proof?
- Can it tell “I checked and it is fine” from “I am not sure, and I am telling you so”?
- What happens when the evidence is real but does not prove the conclusion? Does anything catch it?
A longer checklist is in how to evaluate an autonomous AI security tool.
Where Vorthix XOR-1 fits, and what we do not claim
Vorthix XOR-1 is the autonomous research layer of the platform. It does the long, patient, adversarial reading a human researcher would do, and the Vorthix research team reviews each finding before it is delivered. We are a young company. We do not expect you to take “trust us” at face value, which is the opposite of what this article argues. Ask us the hard questions above. We built Vorthix XOR-1 to answer them honestly, including with “we do not know yet” when that is true.