Skip to main content

3 posts tagged with "Token efficiency"

Getting more security analysis done with fewer model tokens.

View All Tags

Deterministic analysis vs asking a model

· 8 min read
Vulkro
Security research

"Paste the repo into the model and ask it to find the security bugs" has become a common first attempt at AI-assisted security review. It is easy to try, and the first run is often impressive: the model names a plausible injection, explains it well, and suggests a fix.

The trouble starts on the second run. And the third. And when somebody asks what it did not look at.

Give AI coding agents the code, not the whole file

· 7 min read
Vulkro
Security research

Watch an AI coding agent fix a small bug. It searches for the function name and gets forty matches across comments, tests and unrelated files. It opens the file with the best-looking match and reads all 900 lines. It searches again for the callers, opens three more files, and reads those whole as well. The fix it finally makes is four lines.

Every one of those reads stays in the conversation, and the conversation is sent again on every turn. The agent was not careless. It used the tools it had, cat, grep and sed, and none of them can answer the question it was really asking: show me this function, and who calls it.

Security analysis without wasting tokens

· 9 min read
Vulkro
Security research

Ask an AI coding agent to "check this repository for security issues" and watch what it does. It lists the tree. It opens the route files, then the middleware, then the database layer, then a few helpers it was not sure about. Each file goes into its context window. By the time it has an opinion about one endpoint, it has paid to read forty files that had nothing to say.

That is not a flaw in the agent. It is the only way a model can look for a vulnerability on its own: by reading. And reading a codebase is the most expensive thing you can ask a model to do.