Confidence model

Every finding carries a confidence: High | Medium | Low value and a confidence_reason: String. The reason explains why the engine fired and what would mark this a false positive. This is how a high-recall scanner stays usable.

Levels

Level	Meaning	When you'd see it
High	Strong, often runtime-confirmed evidence.	Provider-format secrets, taint-confirmed injection paths, KEV-listed CVEs, vulkro probe confirmations.
Medium	Pattern matches but with credible false-positive shape.	Heuristic IDOR (path takes an :id and no auth check is visible, but might be wrapped). Generic secret patterns without provider format.
Low	Heuristic. Often fires on test/example/migration code.	Pattern-match-only checks where context confirms little.

vulkro scan defaults to --min-confidence medium. To see the lower-confidence findings too, or to tighten to the strictest report:

vulkro scan . --all              # show Low findings too (debug / forensic)
vulkro scan . --min-confidence high

Strict-confidence filter

For CI gates that want maximum precision, vulkro scan --strict-confidence (or VULKRO_STRICT_CONFIDENCE=1) adds a filter on top of the --min-confidence cut: any High finding with an empty evidence bag is dropped at output time. A finding survives the strict cut only if it carries at least one populated Evidence item, meaning at least one corroborating signal: a taint-flow source-to-sink, an exact-match invariant (such as a SUP-COMPROMISE catalog row), a schema-vs-code contradiction, or a pair of pattern signals whose weights sum past the cumulative threshold.

Off by default, and behaviour-preserving for callers who do not opt in. See the strict-confidence section on vulkro scan for the flag and its env-var equivalent.

Per-detector-category tiers

The High | Medium | Low table above gives the general shape. What each level specifically means depends on the detector. The table below is the authoritative per-category reading, and every rule page (under /docs/rules and siblings) links back here.

Detector category	High means	Medium means	Low means
Secrets (secrets, git_history)	Provider-format match (AWS AKIA, Stripe sk_live, GitHub ghp_, etc.) on a non-test path. The string is shaped like a real credential.	Generic high-entropy literal that matches no provider format, but assigned to a *_KEY | _TOKEN | _PASSWORD identifier. Could be a placeholder.	Long hex / base64 string with no naming signal. Frequently a hash, content checksum, or test fixture.
Auth (auth_dataflow, authz)	Route is reachable AND no auth middleware, dependency, decorator, or guard is visible in the import scope or call graph.	Route looks unprotected at the file level, but a global mount (app.use('/api', ...)) or framework convention (APP_GUARD) could be applying auth at runtime.	Route name pattern-matches admin/internal/private but no behavioural signal supports the claim.
IDOR / BOLA (idor)	Handler reads a path/query param like :id and resolves the object without a where(user=...), .filter(owner=...), or equivalent scoping clause; and taint flows from the param into the lookup.	Handler takes an :id and the lookup is unscoped, but no taint analysis confirmation (the param might be derived).	Heuristic only: an :id route exists with no visible check, used for forensic mode.
Mass assignment (mass_assignment)	Body-to-model bind via Model(**req.json), Object.assign(user, req.body), setattr loop over body, etc., on a model whose schema has fields the API surface does not allowlist.	Body-to-model bind on a model whose schema we cannot fully resolve (cross-file fragment, dynamic ORM).	Pattern match alone, no model resolution.
Injection (injection, injection_extra, taint)	End-to-end taint chain from request source to a SQL / shell / template / eval sink without intervening sanitiser.	Sink call with a likely-tainted argument but no full chain confirmation.	Sink call with no taint corroboration; pattern-only.
Crypto (crypto_weakness, crypto_new)	Exact-match invariant: md5(buf) for a password, MODE_ECB for confidentiality, verify=False on TLS. The construct is provably wrong regardless of context.	Weak primitive in use, but might be for a non-security purpose (e.g. md5 for cache key).	Reference to a weak primitive in a comment, docstring, or test name.
SCA / CVE (package_risk, slopsquat_known)	Direct dependency matches a KEV-listed advisory at the exact pinned version.	Transitive dependency matches an advisory; or the version match is a range hit not an exact one.	Heuristic typosquat / new-package signal without a CVE backing it.
SSRF (security::ssrf, taint)	Tainted URL flows into an HTTP client call without an allowlist check or hostname normalization between source and sink.	Tainted URL flows into a sink but the chain crosses a function we couldn't fully resolve.	Pattern only: URL-shaped argument to an HTTP client.
LLM (llm_security)	Tainted user input flows into a model SDK call (openai.chat, anthropic.messages, langchain) without isolation or structured-tool constraint. Or: an env var known to hold a secret flows into the prompt.	Tainted input near an LLM call but no full chain.	LLM call with any string-template usage that could in principle contain user data.
CSRF / cookies (csrf, cookies)	Session-cookie middleware is mounted AND CSRF middleware is absent or commented out; or cookie is set with SameSite: 'none' without Secure.	Session middleware present, CSRF middleware unclear; or cookie missing one of Secure/HttpOnly/SameSite but not all.	Pattern-only: a session-like cookie name with no other signal.

If a finding's tier surprises you, check its category here first: the per-category meaning above is what the rule actually asserts. The matching rule page at /docs/rules walks through the OWASP category and its detectors in detail.

confidence_reason

Every finding has a one-line explanation:

confidence_reason = "taint flowed from req.body to db.query without sanitiser"
confidence_reason = "auth helper not found in import scope"
confidence_reason = "runtime-confirmed via active probe"
confidence_reason = "AKIA-prefix matches AWS access-key format"

It appears in JSON, SARIF (properties.confidence_reason), and the desktop console, and is meant to be readable without going back to the source.

Calibration table

Vulkro holds a static calibration table that downgrades High -> Medium for detector categories scoring under 30% TP-rate on the 13-repo benchmark. The idea: if a category is 70%+ false-positive in the wild, callers should not be told "High confidence" by default.

The table is updated when benchmark numbers change, which keeps the default-mode signal-to-noise ratio honest as new detectors land.

Where the bar is

The published benchmark on 13 deliberately-vulnerable repos:

	vulkro default	vulkro --min-confidence high	community SAST	data-flow SAST
precision	0.21	0.62	0.60	0.45
recall	0.91	0.76	0.22	0.45
F1	0.34	0.68	0.32	0.45

--min-confidence high is the production-recommended cut. On the same corpus it leads the community SAST on precision (0.62 vs 0.60) and the data-flow SAST on recall (0.76 vs 0.45). Its F1 of 0.68 beats both. The other tools are described by category rather than named, as a matter of site policy. The corpus rules, the matching rule and the published scoring method behind these figures are documented in Benchmark.

Three changes on top of the Phase 4 AST-confirmation engine produced the lift:

• The cumulative-weight High threshold was calibrated from 1.5 to 1.1 to match what shipped detectors actually emit (Phase 4 pairs at 0.7 + 0.5 = 1.2).
• Four taint / template / autoescape emit sites were retrofitted to attach Evidence, so the aggregator survives the OWASP-category calibration downgrade.
• A (file, line, message) dedup pass removes duplicate emissions from overlapping intra- and inter-procedural taint engines.

See Benchmark for the full methodology and per-repo breakdown.

Offline false-positive triage (--triage)

vulkro scan --triage adds an optional, default-off, fully-local pass. It re-ranks findings by demoting the usual noise sources one confidence tier:

• test code (test/, tests/, __tests__/, *.spec.*, *_test.go, ...)
• fixtures / examples / samples / mocks
• generated / vendored / minified output (dist/, vendor/, node_modules/, *.min.js, ...)
• database migrations

It is deterministic (same input, same output) and makes no network call and uses no model, so VULKRO_OFFLINE=1 is honoured. It is a path-based heuristic re-ranker, nothing more: not an ML classifier, and not "AI". (An optional on-device model layer may come later.)

Triage only ever lowers confidence and never drops a finding, so recall is preserved: a demoted finding still appears under --all-confidence. Combine it with --min-confidence high so the test/fixture/generated noise leaves the default view while staying one flag away:

vulkro scan . --triage --min-confidence high