CodeCome is the harness I built to let an AI agent help me audit source code without losing the trail. Six discrete phases, Markdown findings with structured YAML frontmatter, validation inside a Docker sandbox, demonstrated impact, and a report you can grep — every time, for every target.
After watching too many chat sessions produce confident-sounding "potential SQL injection" claims with zero evidence, the goal became simple: a workflow where every claim is a file on disk, every file points at specific lines of code, every finding either has evidence or gets rejected, and the whole thing is reviewable by a human in an afternoon.
CodeCome is not a vulnerability scanner. It is not a pentest tool. It is not a magic AI bug finder. It is a harness — a set of conventions, prompts, agents and Make targets — that encodes how a careful researcher actually audits code. The model helps you think. You stay in control.
A workspace layout, naming scheme, finding template and Make targets. Nothing you can't read in an afternoon.
Each phase has explicit prompts checked in under prompts/. Fork, audit and version them like any other code.
A finding is not "real" until it has been counter-argued, validated in a sandbox, and reproduced from an artifact on disk.
src/ can be a copied source tree, a git submodule, a checked-out repo, an extracted archive, or a benchmark corpus. CodeCome doesn't care which. The harness will try to build, test and run the target inside the sandbox — that's the point. Validation happens against a real build.
# Project + audit configuration. Defaults work out of the box. project: name: my-target audit: scope: include: ["src/**"] exclude: ["src/vendor/**", "src/**/_test.*"] focus: [sqli, ssrf, deserialization, authz] extra_prompts: reconnaissance: | Focus sandbox on ASAN builds. agents: auditor: model: anthropic/claude-opus-4-7 variant: high reviewer: model: anthropic/claude-haiku-4-5 validation: allowed_write_paths: ["itemdb/**", "sandbox/**", "tmp/**"]
# bootstrap a virtual env and install deps $ make venv # sanity-check the workspace, model creds, sandbox $ make check # six phases of an audit — each restartable $ make phase-1 # recon + sandbox bootstrap $ make phase-2 # generate candidate findings $ make phase-3 # counter-analysis (dedup / reject) $ make phase-4 FINDING=CC-0001 # validate one finding $ make phase-5 FINDING=CC-0001 # build a PoC, demonstrate impact $ make phase-6 # generate the report # walk ONE finding end-to-end first — you'll learn more # from a single CC-0001 than from twenty PENDING ones.
Avoid make validate-all / exploit-all on a fresh project. Walk one finding through end-to-end first.
Three layered ways to append instructions to any phase prompt. All additive, applied in this order:
Each phase has its own prompt under prompts/, its own agent under .opencode/agents/, its own outputs, and writes to a known location on disk. Phases 1–3 are batch operations; phases 4 and 5 run per finding — intentional, to keep evidence traceable.
1a: agent reads src/, infers target type, languages, build model, attack surface — notes under itemdb/notes/. 1b: picks a baseline from templates/sandboxes/, applies it to sandbox/, validates it, writes itemdb/notes/sandbox-plan.md.
Generate candidate findings under itemdb/findings/PENDING/. Each points at specific code, sources, sinks and a trust boundary. Gated by the sandbox.
A reviewer pass tries to disprove or deduplicate findings. Looks for unreachable code, input validation, authorization, framework protections, false assumptions. Weak hypotheses → REJECTED/, repeats → DUPLICATE/.
One finding at a time, inside the Docker sandbox. Build the target, write a small PoC, capture evidence under itemdb/evidence/<id>/, decide CONFIRMED or REJECTED.
Build a real PoC that shows concrete impact: code execution, data exfiltration, privilege escalation. The exploiter may adjust severity based on what is demonstrated and move the finding to EXPLOITED/. Artifacts under evidence/<id>/exploits/.
Generate a Markdown report grouping exploited and confirmed findings with evidence references. Default path: itemdb/reports/report.md. make report is the lightweight local variant (no agent).
Every finding lives in exactly one folder, named after its current state. Phase 3 moves to REJECTED/DUPLICATE. Phase 4 promotes to CONFIRMED. Phase 5 may promote to EXPLOITED. You can also move them manually with make findings-move.
An idea worth investigating. Filed by Phase 2 (or sweep). Not yet validated.
Validation captured evidence. Real bug, not weaponized yet.
A reproducible exploit exists under evidence/<id>/exploits/. Severity adjusted by Phase 5.
Counter-analysis or validation killed the hypothesis. Kept on disk so it isn't re-investigated for free.
Same root cause as another finding. Linked, not deleted.
A finding is a single Markdown file with a validated YAML frontmatter — the unit of work in CodeCome, not a Jira ticket or a row in a database. Below: the rendered view of CC-0022 (SQL injection in a PHP app's user.get JSON-RPC API), followed by the raw source on disk.
--- id: "CC-0022" title: "SQL injection via unvalidated selectRole in user.get JSON-RPC API" status: "EXPLOITED" severity: "CRITICAL" confidence: "CONFIRMED" category: "SQL Injection" cwe: ["CWE-89"] language: "php" target_area: "JSON-RPC API user.get method" files: - "src/app-1.4.1/ui/include/classes/api/services/CUser.php" symbols: - "CUser::addRelatedObjects()" sources: - "JSON-RPC options['selectRole'] parameter" sinks: - "DBselect() at CUser.php:2243-2248" trust_boundary: "authenticated API user -> raw SQL SELECT clause" validation: status: "CONFIRMED" methods: ["http_exploit", "runtime_reproduction"] evidence_dir: "itemdb/evidence/CC-0022" exploitation: status: "DEMONSTRATED" severity_before: "HIGH" severity_after: "CRITICAL" artifacts_dir: "itemdb/evidence/CC-0022/exploits" --- # Summary The user.get JSON-RPC API accepts a selectRole array whose elements are concatenated into a SQL SELECT clause without any allowlist check. Low-privilege authenticated users can inject SQL.
A vulnerability research project should still be readable in five years. Files survive renaming, forking, GitHub outages and SQL migrations. grep, git log and diff are the only tools you need.
Run make frontmatter to validate every finding's metadata via tools/check-frontmatter.py. Bad frontmatter fails fast. The Markdown body stays free-form so researchers aren't fighting a form.
Findings render in any Markdown viewer — GitHub, Obsidian, an editor preview pane. The "dashboard" is just tree itemdb/findings/ or make status.
Before a finding is marked CONFIRMED, CodeCome reproduces it against a real build of the project — in a Docker container, behind a network namespace, away from your host. Phase 1b bootstraps a sandbox suited to the stack; if the payload doesn't fire there, it doesn't make the cut.
→ sandbox already bootstrapped (Phase 1b) → starting container … healthy → replaying payload … itemdb/evidence/CC-0022/exploit.sh POST /api_jsonrpc.php HTTP/1.1 Authorization: Bearer <low-priv-token> Body: {"method":"user.get","params":{"selectRole":["roleid,(SELECT version())"]}} HTTP/1.1 200 OK Body contains: "10.6.18-MariaDB" → assertions ✓ status 200 ✓ response inlines server version() ✓ query log shows injected SELECT in r.* clause → result: CONFIRMED → moved itemdb/findings/PENDING/CC-0022 → itemdb/findings/CONFIRMED/CC-0022
Per-capability sandbox helpers exist as separate targets: sandbox-list, sandbox-detect, sandbox-inspect ID=python, sandbox-bootstrap, sandbox-validate, sandbox-regenerate, sandbox-status, plus runtime helpers sandbox-{setup,up,check,build,test,down,shell,logs,clean,reset}. See docs/sandbox.md.
Phase 2 is wide and fast. make sweep is the opposite: it runs the auditor agent once per file, forcing exhaustive line-by-line analysis on every high-risk file in itemdb/notes/file-risk-index.yml. It catches what broad audits miss; Phase 3 cleans the overlap.
# preview which files would be swept $ make list-risk-files # dry-run: show selected files + prompts, no agent calls $ python tools/run-sweep.py --dry-run # sweep everything scoring 4+ in file-risk-index.yml $ make sweep # sweep a specific file… $ make sweep FILE="src/path/to/file.ext" # …or a glob $ make sweep FILE="src/**/*.cs"
Sanitized snapshots from real audits — enough to show the workflow, not enough to leak target-specific exploit details or credentials. Click any tile for the full-size image.
Reviewable hypotheses awaiting validation.
on disk ↗Agentic, but auditable from the file system up.
six agents ↗Validation before belief — inside Docker.
phase 4 ↗Every confirmed claim leaves files behind.
disk-resident ↗Sandbox scripts produced on demand per finding.
on demand ↗Readable PoC writeups, not chat scrollback.
phase 5 ↗Try to disprove first — then validate.
phase 3 ↗Exploited findings with linked artifacts.
phase 6 ↗An asciinema cast of a full run is planned.
CodeCome won't turn a non-researcher into one. It will save a researcher hours of bookkeeping per audit. If you want a one-click vulnerability scanner, this is not it. CodeCome is for people who want the model to help them think, not to replace the thinking.
Audit codebases at your own pace, with a trail you can re-read months later or hand to someone else.
From recon to PoC, every step lands in the workspace as a Markdown finding with evidence references, ready to ship.
Intentionally simple — fork the prompts, swap the agent runner, compare runs across models. The harness is the experimental surface.
CodeCome runs on top of OpenCode (1.14.39 or newer) with your own LLM provider, plus a small Python + Make + Docker stack. make check warns about anything missing — the core workflow runs without the optional tools.
The open-source AI coding agent CodeCome drives. Install guide.
At least one of Anthropic, OpenAI, Google, xAI, Groq, Cerebras, GitHub Copilot, Google Vertex — or a local OpenAI-compatible endpoint. Provider setup.
For workspace tooling. make venv creates a local virtualenv at .venv/.
The entire workflow is driven through make targets.
Required for the sandboxed validation environment used by Phases 1b / 4 / 5.
Terminal recordings of exploit replays.
Renders .cast files to GIFs. CodeCome falls back to a Docker container if missing.
For GUI / browser exploits where video evidence matters. xvfb-run is fine too.
Comments, docstrings, READMEs, test fixtures, log strings, commit messages, filenames — even crafted binary blobs inside src/ — can carry instructions aimed at the agent ("ignore previous instructions…", "exfiltrate $HOME/.ssh/…"). The agent reads these as input, but LLMs are still susceptible.
Phase 1b will try to build and run the target. A malicious setup.py, package.json lifecycle hook, Makefile, Dockerfile, or configure script executes inside the sandbox container with whatever permissions Docker gives it.
Adversarial code may try to consume CPU, disk, or network from the validation phase. A prompt-injected runaway agent loop can burn tokens just as easily.
If the sandbox (or your host) can reach the internet, an injected agent or a malicious build step can attempt to send data out. The default policy assumes egress is possible.
Run the whole workspace inside an isolation boundary when auditing untrusted sources — a disposable VM (Multipass, Vagrant, UTM, Proxmox), a dedicated container, or a remote throwaway host. Do not run CodeCome on a machine that holds credentials, SSH keys, browser profiles, or production access you can't afford to lose.
Treat src/ as untrusted. CodeCome funnels execution through sandbox/, but the make runner itself, the agent, and any helper scripts still execute on the host.
Restrict network egress from the sandbox (and ideally from the outer VM) to only what you need for builds and package installs.
Use a fresh API key with low spend limits for the LLM provider, so a prompt-injected runaway loop can't rack up an unbounded bill.
Review what the agent writes under itemdb/, sandbox/ and tmp/ before trusting any of it. Findings, evidence and reports are all attacker-influenced when the target is untrusted.
Avoid make exploit-all and make validate-all on untrusted targets until you have walked at least one finding through manually and confirmed the sandbox behaves the way you expect.
A CodeCome workspace is a normal git repo with a small, fixed set of folders. The heart is itemdb/ — findings, evidence, notes and reports. Drop it into any IDE and read it as code.
CodeCome lets you pin a different model for every phase via agents.<name>.model in codecome.yml (or CODECOME_MODEL on the command line). Different models see different bugs — running Phase 2 with two providers and letting Phase 3 dedupe is a legitimate strategy.
Audit and exploit phases benefit from reasoning models — Opus, GPT reasoning variants, Gemini Pro reasoning. Pin with agents.auditor.model: anthropic/claude-opus-4-7.
Counter-analysis and reporting are well-served by smaller, cheaper models. Mix freely; no requirement to run a single provider end-to-end.
Point any agent at an OpenAI-compatible local endpoint (Ollama, vLLM, llama.cpp). No code leaves your machine. Use make show-model to print the resolution table per agent.
The model helps you think. You stay in control. Nothing is committed to a finding folder without an explicit phase being run.
By default, phase targets wrap opencode run --format json with a CodeCome-owned styled renderer. Assistant output, tool calls and tool results render with consistent colors and structure. It also routes plain bash invocations (cat, head, tail, rg, ls, find, tree, rtk …) through the matching styled renderer.
# bypass the styled wrapper entirely CODECOME_USE_WRAPPER=0 # control --thinking on the provider call CODECOME_THINKING=1 # force on CODECOME_THINKING=0 # force off (don't pay for reasoning tokens) # model resolution CODECOME_MODEL=anthropic/claude-opus-4-7 CODECOME_MODEL_VARIANT=high # surfaces CODECOME_RENDER_REASONING=0 # hide on-screen Thinking panels CODECOME_SANDBOX_RENDER=0 # disable structured Sandbox panel CODECOME_BASH_SHIM_RENDER=0 # disable rtk/cat/head/tail/rg routing # budgets CODECOME_BOOTSTRAP_MAX_RETRIES=3 CODECOME_REASONING_MAX_CHARS=4000 # forward extra flags to opencode run OPENCODE_ARGS="--max-tokens 8192"
A handful of make targets cover day-to-day workspace bookkeeping — no LLM call required.
CodeCome is an early PoC. The conventions are stable enough to use; the tooling around them is still moving. Below is what works well today and what is still rough — no marketing.
CodeCome's value is in its methodology, not its UI. The docs explain how each phase works and what its prompts assume.
Install, configure a provider, run your first audit.
README ↗Full phase-by-phase workflow reference.
docs/workflow.md ↗Supported layouts: source trees, submodules, archives, benchmark corpora.
docs/target-setup.md ↗Bootstrap, boundaries, evidence capture, validation environment.
docs/sandbox.md ↗Risk index format and the deep sweep reference.
docs/file-risk-sweeps.md ↗Repo conventions, helper tools, contributor workflow.
docs/development.md ↗All phase prompts — phase-1-recon.md through phase-6-report.md + sweep.md.
prompts/ ↗Pull requests are expected, encouraged, and appreciated. See CONTRIBUTING.md.
CodeCome is small. A patch to a phase prompt, a sandbox template for a new language, or a bug report on a confusing convention are all valuable. We won't accept PRs that turn this into a scanner.
Improve a phase prompt with a diff and a short rationale. Bring a run summary if you can.
Contribute a Dockerfile + scripts for a stack we don't cover yet, under templates/sandboxes/.
Disagree with a phase boundary? Open a discussion before a PR.
CLI ergonomics, schema validation, report generation — all welcome.
Dual-licensed — pick whichever copyleft fits your context. Files under templates/sandboxes/ are an exception: MIT, so they can be copied into user workspaces without imposing copyleft on those projects. See LICENSE, AGPL-LICENSE, templates/sandboxes/LICENSE and NOTICE.