Skip to content

Evidence standard

What a security finding has to prove

A finding is a claim about how software can fail. This standard sets out the evidence we expect behind that claim, how strong it needs to be, and how a reader can check it independently.

Claim

Out-of-bounds write while parsing a length field in a file header.

memory safetyattacker-controlled input

Everything else on this cube exists to support this one sentence.

Source · parser.c
210int parse_header(buf, in) {
211  len = read_u16(in);
212  /* no upper bound */
213  memcpy(buf, in, len);214}

buf holds 32 bytes. len can reach 65535.

Trace · AddressSanitizer
heap-buffer-overflow
WRITE of size 64
#0 parse_header parser.c:213
#1 main         main.c:58

Observed behavior that matches the claim.

Reproduction
$ cc -fsanitize=address p.c
$ ./poc minimized.bin
→ heap-buffer-overflow
isolated buildsame result

Input, environment and steps are included.

Review

Verified by a researcher

  • Evidence checked
  • Impact reasoning checked
Limits

What was not examined

  • other input formats
  • code after this commit

Scope is stated, not assumed.

Illustrative · one finding, seen from every side. Keep scrolling to turn the cube.

Start from one claim

A finding begins as one specific claim. Everything else exists to support it, and nothing proves more than the claim it is attached to.

Evidence from the source itself

Evidence is taken from the artifact: the exact file and line the claim refers to, not a summary of it.

Behavior that matches the claim

Observed behavior ties the code to a real effect: what happened, where and how much. The trace has to match this claim, not a neighboring one.

Anyone can run it

The input, environment and steps are included, and the result is the same in an isolated build. A finding that only its author can trigger is hard to trust.

A person checks the reasoning

A qualified researcher verifies the evidence and the reasoning about impact before a finding is delivered or disclosed.

Limits are part of the finding

Every finding states what was not examined. Unknown is reported as unknown, never rounded to safe or vulnerable.

In short

An evidence standard defines what must accompany a security conclusion before it is treated as true.

It specifies which artifacts are required, how closely they must match the claim, and what to report when they are missing, so a finding is judged on its evidence rather than on the confidence of whoever, or whatever, produced it.

The standard

How a conclusion earns its status

Hover, tap or tab through the principles. Each one protects a different part of a finding.

Traceable
Traceable

Every conclusion points to retrievable evidence.

Evidence comes from the artifact under test, meaning the source, the binary or the execution, and not from a summary or a recollection of it. A reader can follow a claim back to the exact location that supports it.

Specific
Specific

Evidence must prove the claim it is attached to.

Real evidence attached to the wrong conclusion is still a failure. A crash trace shows that something crashed. It does not by itself show that the crash is reachable by an attacker or exploitable.

Complete
Complete

Each supporting fact stands on its own.

When a conclusion depends on several technical assertions, every one needs its own support. A single valid citation cannot vouch for assumptions it never addressed.

Reproducible
Reproducible

Another engineer can observe the same result.

A finding includes what is needed to see it independently: the input, the environment and the steps. A result that cannot be reproduced is reported as exactly that.

Independent of tone
Independent of tone

Confidence is not evidence.

Wording does not change the status of a finding. Phrases such as “this is safe” or “this is clearly exploitable” carry no weight unless the evidence supports them.

Honest about limits
Honest about limits

Unknown is a valid answer.

When evidence runs out, the correct result is “insufficient evidence”, not a guess in either direction. A negative result is scoped to what was actually examined.

Reviewed by people
Reviewed by people

A qualified person checks before delivery.

Researchers verify both the evidence and the reasoning before a finding is relied on or delivered.

Evidence strength

Not all evidence carries the same weight

A finding is described by the strongest evidence behind it, so readers know how much to rely on it. Pick a weight to see what it requires.

Assertion

A statement with no supporting artifact: a claim that something is vulnerable, or that it is safe. Assertions are not reported as findings, however confidently they are phrased.

Reported as: not a finding
no artifact

Indication

A pointer to a location or behavior that looks suspicious: a code reference, a pattern match or a partial trace. Worth investigating, but not yet a conclusion.

Reported as: lead
code referencepattern matchpartial trace

Corroborated

Independent signals agree, for example static reasoning and observed runtime behavior, but the issue has not been triggered end to end. Useful for prioritizing, not for declaring.

Reported as: candidate finding
static reasoningruntime observation

Reproduced

The issue has been triggered in an isolated environment with a working proof of concept that others can run. The evidence matches the specific claim, and each supporting fact is addressed.

Reported as: confirmed finding
minimized inputsanitizer traceisolated buildsteps to reproduce

Reviewed

Reproduced, and then verified by a qualified person who has checked the evidence and the impact reasoning. This is the level at which a finding is delivered or disclosed.

Reported as: delivered finding
reviewer sign-offimpact reasoningremediation guidance

Off the ladder: insufficient evidence is what is reported when evidence runs out before a conclusion can be supported in either direction.

Scope and limits

What the standard does and does not claim

A standard is only useful if it is clear about its boundaries.

It establishes

What you can rely on

  • Inspectable findingsA reported finding is backed by evidence you can examine yourself.
  • Matched evidenceEach claim is paired with evidence specific to it, not to a neighboring issue.
  • Honest unknownsGaps are reported as gaps instead of being filled with a guess.

It does not claim

What it cannot promise

  • Absence of vulnerabilitiesNo findings reflects what was examined, not a guarantee that nothing is there.
  • Permanent validityFindings describe a specific version of the code and can change when it does.
  • Objective severityImpact reasoning is shown so that it can be examined and challenged.

Putting it to use

Questions worth asking of any security finding

They apply to any report, from any source, human or automated.

Can I reach the evidence myself?

Ask for the artifacts rather than a summary: the input, the trace and the source location. If they cannot be retrieved, treat the finding as an indication.

What exactly does this evidence prove?

Check that it supports the specific claim and not an adjacent one. A crash is not automatically a vulnerability that an attacker can reach.

What happens when the evidence runs out?

A trustworthy process has a defined outcome for this. If the answer is always a confident conclusion, be cautious.

Can the result be reproduced independently?

A reproducible finding comes with the steps and the environment. One that only its author can trigger is hard to verify and hard to fix.

Who reviewed it, and what did they check?

Look for a named review step that covers both the evidence and the reasoning about impact, before the finding reaches you.

What was not examined?

Every analysis has a boundary. A clear statement of scope is a sign of rigor, and a missing one is worth asking about.

Request access

See how Vorthix works on your code

Access is currently by request for security teams and researchers.

  • Working proof of concept
  • Sanitizer-backed trace
  • Suggested patch and regression test