Detection Benchmark
We publish the raw numbers on how well our scanner performs against a labeled corpus of vulnerable and clean code samples. Checked daily and rerun when rules or fixtures change (at least weekly); updates go live with the next deploy.
Last run: Nov 4, 2026 · Corpus size: 649 fixtures (357 vulnerable, 292 clean)
Updated Nov 3, 2026: we now count reviewed false positives on clean test files. Earlier versions of this page showed 100% precision.
The numbers above are for all 223 rules (paid plans). On the free plan's 30 rules, the same corpus scores recall 24.8% (TP 160 / forbidden-rule hits 0 / FN 484). Published precision is the reviewed figure above. Most fixtures target vulnerabilities only the paid rules detect.
Trend over time
90 days · since Jun 15, 2026Recall from the last benchmark run of each day. The corpus grows over time, so a flat or rising line means detection kept pace as new fixtures were added.
Updated Nov 3, 2026: we now count reviewed false positives on clean test files. Earlier versions of this page showed 100% precision.
Corpus: 232 → 649 fixtures. Deterministic scanner only (excludes the optional AI false-positive filter).
How we score
Every fixture in packages/cli/test-fixtures/ is labeled with expectedFindings (rules that must fire, with file and line range) and mustNotFire (rules that must not fire anywhere in the fixture). The runner scans each fixture and counts:
- True positive (TP) — a finding whose rule, file, and line fall inside an expected entry.
- False negative (FN) — an expected entry with no matching finding.
- Forbidden-rule hit — a finding for a rule explicitly listed as mustNotFire. The per-rule column counts these. It is not the precision figure above.
We reviewed by hand, on Nov 3, 2026, each finding on a clean fixture that its label does not expect. Of 396 such findings then, 132 were real false positives. The Nov 4 rule fixes removed 33 of those and did not add any, leaving 99. Of the 339 unexpected findings on the clean fixtures as of Nov 4, 2026, those 99 are the reviewed false positives, which gives 86.3% precision. False positives are counted on clean fixtures only. Recall is computed on every benchmark run.
The runner lives at scripts/benchmark.js in the repository.
About this baseline. The corpus is 649 fixtures, and growing it is ongoing work. These scores are not a claim that the scanner catches everything; they are the floor below which we will not regress. We will keep adding harder cases, and over time the score will become a more demanding indicator.
Held-out set — 7/8 blind and 15/16 after fixes
The corpus above is ours — we wrote the rules and the fixtures, so a high score partly measures how well we test ourselves. This set is different: 16 real vulnerabilities pulled from public, intentionally-vulnerable projects (OWASP NodeGoat, Juice Shop, DVNA) and one public prototype-pollution example that none of our rule authors wrote, with the projects' hint comments stripped. We expect a lower number here — that's the point, and 7 of 8 (a small sample) is the honest one. When the first 16 cases were each first scored, before any rule was changed for them, XploitScan caught 8 of 16. The 8 misses were then studied and the rules changed to catch them, so those 8 are excluded from the blind figure above — a case you have studied cannot measure whether a rule generalises to code you have not. They still appear in the table below. Why we built this →
On the free plan's 30 rules alone, the same cases score 6 of 16 (4 of 8 blind).
The same corpus is scored against other scanners with the same class-level methodology: did the tool flag this class of vulnerability anywhere in the fixture? XploitScan 93.8% · Semgrep 50.0% · Bearer 56.3% (all three scored over the full 16-case corpus. XploitScan's rules were changed after we studied 8 of these cases; Semgrep and Bearer were run unmodified. Before that tuning, XploitScan caught 8 of 16 (50.0%).)
| Vulnerability class | Source | XploitScan | Semgrep | Bearer |
|---|---|---|---|---|
| Command injection | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | missed | ✓ caught |
| Insecure deserialization | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | missed |
| Code injection (eval) | OWASP/NodeGoat — app/routes/contributions.js (Apache-2.0) | ✓ caught | ✓ caught | ✓ caught |
| IDOR / broken access control (BOLA) | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | missed | missed |
| Hardcoded credentials | juice-shop/juice-shop — lib/insecurity.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Open redirect | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Path traversal | juice-shop/juice-shop — routes/fileServer.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| Prototype pollution | kimmobrunfeldt/lodash-merge-pollution-example — index.js (ISC) | ✓ caught | missed | missed |
| SQL injection | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
| SQL injection | juice-shop/juice-shop — routes/login.ts (MIT) | ✓ caught | ✓ caught | ✓ caught |
| SSRF | OWASP/NodeGoat — app/routes/research.js (Apache-2.0) | ✓ caught | missed | missed |
| Server-side template injection | juice-shop/juice-shop — routes/dataErasure.ts (MIT) | ✓ caught | missed | missed |
| Weak password hashing | juice-shop/juice-shop — lib/insecurity.ts, routes/login.ts (MIT) | missed | missed | ✓ caught |
| NoSQL injection | OWASP/NodeGoat — app/data/allocations-dao.js (Apache-2.0) | ✓ caught | missed | missed |
| Cross-site scripting (DOM) | juice-shop/juice-shop — frontend search-result.component.ts (MIT) | ✓ caught | missed | missed |
| XXE | appsecco/dvna — core/appHandler.js (MIT) | ✓ caught | ✓ caught | ✓ caught |
XploitScan's misses here are real coverage gaps we track openly. Corpus + sources in the public repo under test-fixtures/held-out/.
Per-rule scores
| Rule | TP | forbidden-rule hits | FN | Recall | F1 |
|---|---|---|---|---|---|
| ENTROPY | 7 | 0 | 0 | 100.0% | — |
| VC001 | 13 | 0 | 0 | 100.0% | — |
| VC002 | 1 | 0 | 0 | 100.0% | — |
| VC003 | 43 | 0 | 0 | 100.0% | — |
| VC004 | 1 | 0 | 0 | 100.0% | — |
| VC005 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC006 | 9 | 0 | 0 | 100.0% | — |
| VC007 | 5 | 0 | 0 | 100.0% | — |
| VC008 | 2 | 0 | 0 | 100.0% | — |
| VC009 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC010 | 2 | 0 | 0 | 100.0% | — |
| VC011 | 21 | 0 | 0 | 100.0% | — |
| VC012 | 1 | 0 | 0 | 100.0% | — |
| VC013 | 2 | 0 | 0 | 100.0% | — |
| VC014 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC015 | 1 | 0 | 0 | 100.0% | — |
| VC016 | 1 | 0 | 0 | 100.0% | — |
| VC017 | 2 | 0 | 0 | 100.0% | — |
| VC018 | 1 | 0 | 0 | 100.0% | — |
| VC019 | 1 | 0 | 0 | 100.0% | — |
| VC020 | 1 | 0 | 0 | 100.0% | — |
| VC021 | 1 | 0 | 0 | 100.0% | — |
| VC022 | 1 | 0 | 0 | 100.0% | — |
| VC023 | 4 | 0 | 0 | 100.0% | — |
| VC024 | 1 | 0 | 0 | 100.0% | — |
| VC025 | 4 | 0 | 0 | 100.0% | — |
| VC026 | 1 | 0 | 0 | 100.0% | — |
| VC027 | 1 | 0 | 0 | 100.0% | — |
| VC028 | 1 | 0 | 0 | 100.0% | — |
| VC029 | 3 | 0 | 0 | 100.0% | — |
| VC030 | 1 | 0 | 0 | 100.0% | — |
| VC031 | 3 | 0 | 0 | 100.0% | — |
| VC032 | 5 | 0 | 0 | 100.0% | — |
| VC033 | 1 | 0 | 0 | 100.0% | — |
| VC034 | 2 | 0 | 0 | 100.0% | — |
| VC035 | 2 | 0 | 0 | 100.0% | — |
| VC036 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC037 | 4 | 0 | 0 | 100.0% | — |
| VC038 | 2 | 0 | 0 | 100.0% | — |
| VC039 | 2 | 0 | 0 | 100.0% | — |
| VC040 | 1 | 0 | 0 | 100.0% | — |
| VC041 | 6 | 0 | 0 | 100.0% | — |
| VC042 | 10 | 0 | 0 | 100.0% | — |
| VC043 | 1 | 0 | 0 | 100.0% | — |
| VC044 | 6 | 0 | 1 | 85.7% | 92.3% |
| VC045 | 1 | 0 | 0 | 100.0% | — |
| VC046 | 2 | 0 | 0 | 100.0% | — |
| VC047 | 2 | 0 | 0 | 100.0% | — |
| VC048 | 3 | 0 | 0 | 100.0% | — |
| VC049 | 9 | 0 | 0 | 100.0% | — |
| VC050 | 1 | 0 | 0 | 100.0% | — |
| VC051 | 2 | 0 | 0 | 100.0% | — |
| VC052 | 1 | 0 | 0 | 100.0% | — |
| VC053 | 3 | 0 | 0 | 100.0% | — |
| VC054 | 2 | 0 | 0 | 100.0% | — |
| VC055 | 1 | 0 | 0 | 100.0% | — |
| VC056 | 1 | 0 | 0 | 100.0% | — |
| VC057 | 1 | 0 | 0 | 100.0% | — |
| VC058 | 1 | 0 | 0 | 100.0% | — |
| VC059 | 1 | 0 | 0 | 100.0% | — |
| VC060 | 9 | 0 | 0 | 100.0% | — |
| VC061 | 3 | 0 | 0 | 100.0% | — |
| VC062 | 1 | 0 | 0 | 100.0% | — |
| VC063 | 6 | 0 | 0 | 100.0% | — |
| VC064 | 4 | 0 | 0 | 100.0% | — |
| VC065 | 13 | 0 | 0 | 100.0% | — |
| VC066 | 1 | 0 | 0 | 100.0% | — |
| VC067 | 2 | 0 | 0 | 100.0% | — |
| VC068 | 3 | 0 | 0 | 100.0% | — |
| VC069 | 2 | 0 | 0 | 100.0% | — |
| VC070 | 1 | 0 | 0 | 100.0% | — |
| VC071 | 1 | 0 | 0 | 100.0% | — |
| VC072 | 1 | 0 | 0 | 100.0% | — |
| VC073 | 1 | 0 | 0 | 100.0% | — |
| VC074 | 1 | 0 | 0 | 100.0% | — |
| VC075 | 1 | 0 | 0 | 100.0% | — |
| VC076 | 3 | 0 | 0 | 100.0% | — |
| VC077 | 1 | 0 | 0 | 100.0% | — |
| VC078 | 1 | 0 | 0 | 100.0% | — |
| VC079 | 2 | 0 | 0 | 100.0% | — |
| VC080 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC081 | 2 | 0 | 0 | 100.0% | — |
| VC082 | 5 | 0 | 0 | 100.0% | — |
| VC083 | 1 | 0 | 0 | 100.0% | — |
| VC084 | 2 | 0 | 0 | 100.0% | — |
| VC085 | 2 | 0 | 0 | 100.0% | — |
| VC086 | 2 | 0 | 0 | 100.0% | — |
| VC087 | 1 | 0 | 0 | 100.0% | — |
| VC088 | 2 | 0 | 0 | 100.0% | — |
| VC089 | 2 | 0 | 0 | 100.0% | — |
| VC090 | 1 | 0 | 0 | 100.0% | — |
| VC091 | 1 | 0 | 0 | 100.0% | — |
| VC092 | 2 | 0 | 0 | 100.0% | — |
| VC093 | 2 | 0 | 0 | 100.0% | — |
| VC094 | 10 | 0 | 0 | 100.0% | — |
| VC095 | 4 | 0 | 0 | 100.0% | — |
| VC096 | 5 | 0 | 0 | 100.0% | — |
| VC097 | 1 | 0 | 0 | 100.0% | — |
| VC098 | 2 | 0 | 0 | 100.0% | — |
| VC099 | 8 | 0 | 0 | 100.0% | — |
| VC100 | 1 | 0 | 0 | 100.0% | — |
| VC101 | 4 | 0 | 0 | 100.0% | — |
| VC102 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC103 | 4 | 0 | 0 | 100.0% | — |
| VC104 | 4 | 0 | 0 | 100.0% | — |
| VC105 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC106 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC107 | 1 | 0 | 0 | 100.0% | — |
| VC108 | 1 | 0 | 0 | 100.0% | — |
| VC109 | 1 | 0 | 0 | 100.0% | — |
| VC110 | 1 | 0 | 0 | 100.0% | — |
| VC111 | 1 | 0 | 0 | 100.0% | — |
| VC112 | 2 | 0 | 0 | 100.0% | — |
| VC113 | 1 | 0 | 0 | 100.0% | — |
| VC114 | 1 | 0 | 0 | 100.0% | — |
| VC115 | 2 | 0 | 0 | 100.0% | — |
| VC116 | 1 | 0 | 0 | 100.0% | — |
| VC117 | 5 | 0 | 0 | 100.0% | — |
| VC118 | 7 | 0 | 0 | 100.0% | — |
| VC119 | 1 | 0 | 0 | 100.0% | — |
| VC120 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC121 | 4 | 0 | 0 | 100.0% | — |
| VC122 | 3 | 0 | 0 | 100.0% | — |
| VC123 | 11 | 0 | 0 | 100.0% | — |
| VC124 | 12 | 0 | 0 | 100.0% | — |
| VC125 | 8 | 0 | 0 | 100.0% | — |
| VC126 | 1 | 0 | 0 | 100.0% | — |
| VC127 | 8 | 0 | 0 | 100.0% | — |
| VC128 | 2 | 0 | 0 | 100.0% | — |
| VC129 | 8 | 0 | 0 | 100.0% | — |
| VC130 | 3 | 0 | 1 | 75.0% | 85.7% |
| VC131 | 3 | 0 | 0 | 100.0% | — |
| VC132 | 1 | 0 | 0 | 100.0% | — |
| VC133 | 2 | 0 | 0 | 100.0% | — |
| VC134 | 2 | 0 | 0 | 100.0% | — |
| VC135 | 1 | 0 | 0 | 100.0% | — |
| VC136 | 1 | 0 | 0 | 100.0% | — |
| VC137 | 1 | 0 | 0 | 100.0% | — |
| VC138 | 1 | 0 | 0 | 100.0% | — |
| VC139 | 3 | 0 | 0 | 100.0% | — |
| VC140 | 2 | 0 | 0 | 100.0% | — |
| VC141 | 2 | 0 | 0 | 100.0% | — |
| VC142 | 1 | 0 | 0 | 100.0% | — |
| VC143 | 2 | 0 | 0 | 100.0% | — |
| VC144 | 2 | 0 | 0 | 100.0% | — |
| VC145 | 1 | 0 | 0 | 100.0% | — |
| VC146 | 1 | 0 | 0 | 100.0% | — |
| VC147 | 7 | 0 | 0 | 100.0% | — |
| VC148 | 1 | 0 | 0 | 100.0% | — |
| VC149 | 1 | 0 | 0 | 100.0% | — |
| VC150 | 2 | 0 | 0 | 100.0% | — |
| VC151 | 2 | 0 | 0 | 100.0% | — |
| VC152 | 1 | 0 | 0 | 100.0% | — |
| VC153 | 1 | 0 | 0 | 100.0% | — |
| VC154 | 1 | 0 | 0 | 100.0% | — |
| VC155 | 4 | 0 | 0 | 100.0% | — |
| VC156 | 1 | 0 | 0 | 100.0% | — |
| VC157 | 2 | 0 | 0 | 100.0% | — |
| VC158 | 3 | 0 | 0 | 100.0% | — |
| VC159 | 2 | 0 | 0 | 100.0% | — |
| VC160 | 1 | 0 | 0 | 100.0% | — |
| VC161 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC162 | 2 | 0 | 0 | 100.0% | — |
| VC163 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC164 | 1 | 0 | 0 | 100.0% | — |
| VC165 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC166 | 1 | 0 | 0 | 100.0% | — |
| VC167 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC168 | 1 | 0 | 0 | 100.0% | — |
| VC169 | 2 | 0 | 0 | 100.0% | — |
| VC170 | 1 | 0 | 0 | 100.0% | — |
| VC171 | 1 | 0 | 0 | 100.0% | — |
| VC172 | 1 | 0 | 0 | 100.0% | — |
| VC173 | 1 | 0 | 0 | 100.0% | — |
| VC174 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC175 | 2 | 0 | 1 | 66.7% | 80.0% |
| VC176 | 1 | 0 | 1 | 50.0% | 66.7% |
| VC177 | 2 | 0 | 0 | 100.0% | — |
| VC178 | 1 | 0 | 0 | 100.0% | — |
| VC179 | 2 | 0 | 0 | 100.0% | — |
| VC180 | 2 | 0 | 0 | 100.0% | — |
| VC181 | 3 | 0 | 0 | 100.0% | — |
| VC182 | 2 | 0 | 0 | 100.0% | — |
| VC183 | 3 | 0 | 0 | 100.0% | — |
| VC184 | 1 | 0 | 0 | 100.0% | — |
| VC185 | 1 | 0 | 0 | 100.0% | — |
| VC186 | 2 | 0 | 0 | 100.0% | — |
| VC187 | 6 | 0 | 0 | 100.0% | — |
| VC188 | 3 | 0 | 0 | 100.0% | — |
| VC189 | 1 | 0 | 0 | 100.0% | — |
| VC190 | 1 | 0 | 0 | 100.0% | — |
| VC191 | 1 | 0 | 0 | 100.0% | — |
| VC192 | 1 | 0 | 0 | 100.0% | — |
| VC193 | 2 | 0 | 0 | 100.0% | — |
| VC194 | 1 | 0 | 0 | 100.0% | — |
| VC195 | 2 | 0 | 0 | 100.0% | — |
| VC196 | 3 | 0 | 0 | 100.0% | — |
| VC197 | 3 | 0 | 0 | 100.0% | — |
| VC198 | 1 | 0 | 0 | 100.0% | — |
| VC199 | 1 | 0 | 0 | 100.0% | — |
| VC200 | 1 | 0 | 0 | 100.0% | — |
| VC201 | 2 | 0 | 0 | 100.0% | — |
| VC202 | 9 | 0 | 0 | 100.0% | — |
| VC203 | 2 | 0 | 0 | 100.0% | — |
| VC204 | 1 | 0 | 0 | 100.0% | — |
| VC205 | 1 | 0 | 0 | 100.0% | — |
| VC206 | 1 | 0 | 0 | 100.0% | — |
| VC207 | 1 | 0 | 0 | 100.0% | — |
| VC208 | 1 | 0 | 0 | 100.0% | — |
| VC209 | 1 | 0 | 0 | 100.0% | — |
| VC210 | 1 | 0 | 0 | 100.0% | — |
| VC211 | 1 | 0 | 0 | 100.0% | — |
| VC212 | 1 | 0 | 0 | 100.0% | — |
| VC213 | 1 | 0 | 0 | 100.0% | — |
| VC214 | 1 | 0 | 0 | 100.0% | — |
| VC215 | 5 | 0 | 0 | 100.0% | — |
| VC216 | 7 | 0 | 0 | 100.0% | — |
| VC217 | 4 | 0 | 0 | 100.0% | — |
| VC218 | 2 | 0 | 0 | 100.0% | — |
| VC219 | 4 | 0 | 0 | 100.0% | — |
| VC220 | 7 | 0 | 0 | 100.0% | — |
| VC221 | 8 | 0 | 0 | 100.0% | — |
| VC222 | 4 | 0 | 0 | 100.0% | — |
| VC223 | 6 | 0 | 0 | 100.0% | — |
Only rules with at least one ground-truth entry in the corpus appear here.
Head-to-head vs free scanners
| VC Rule | XploitScan | Semgrep | Sem? | Bearer | Bear? |
|---|---|---|---|---|---|
| ENTROPY | 100.0% | 0.0% | ✗ | 14.3% | ✓ |
| VC001 | 100.0% | 0.0% | ✗ | 38.5% | ✓ |
| VC002 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC003 | 100.0% | 0.0% | ✗ | 7.0% | ✓ |
| VC004 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC005 | 75.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC006 | 100.0% | 11.1% | ✓ | 44.4% | ✓ |
| VC007 | 100.0% | 40.0% | ✓ | 60.0% | ✓ |
| VC008 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC009 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC010 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC011 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC012 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC013 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC014 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC015 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC016 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC017 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC018 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC019 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC020 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC021 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC022 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC023 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC024 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC025 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC026 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC027 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC028 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC029 | 100.0% | 33.3% | ✓ | 100.0% | ✓ |
| VC030 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC031 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC032 | 100.0% | 60.0% | ✓ | 60.0% | ✓ |
| VC033 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC034 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC035 | 100.0% | 50.0% | ✓ | 100.0% | ✓ |
| VC036 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC037 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC038 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC039 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC040 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC041 | 100.0% | 16.7% | ✓ | 83.3% | ✓ |
| VC042 | 100.0% | 0.0% | ✗ | 10.0% | ✓ |
| VC043 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC044 | 85.7% | 14.3% | ✓ | 100.0% | ✓ |
| VC045 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC046 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC047 | 100.0% | 50.0% | ✓ | 50.0% | ✓ |
| VC048 | 100.0% | 0.0% | ✗ | 33.3% | ✓ |
| VC049 | 100.0% | 0.0% | ✗ | 33.3% | ✓ |
| VC050 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC051 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC052 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC053 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC054 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC055 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC056 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC057 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC058 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC059 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC060 | 100.0% | 11.1% | ✓ | 33.3% | ✓ |
| VC061 | 100.0% | 66.7% | ✓ | 66.7% | ✓ |
| VC062 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC063 | 100.0% | 0.0% | ✗ | 83.3% | ✓ |
| VC064 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC065 | 100.0% | 0.0% | ✗ | 23.1% | ✓ |
| VC066 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC067 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC068 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC069 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC070 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC071 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC072 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC073 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC074 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC075 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC076 | 100.0% | 33.3% | ✓ | 0.0% | ✗ |
| VC077 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC078 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC079 | 100.0% | 100.0% | ✓ | 50.0% | ✓ |
| VC080 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC081 | 100.0% | 50.0% | ✓ | 100.0% | ✓ |
| VC082 | 100.0% | 40.0% | ✓ | 40.0% | ✓ |
| VC083 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC084 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC085 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC086 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC087 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC088 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC089 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC090 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC091 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC092 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC093 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC094 | 100.0% | 50.0% | ✓ | 80.0% | ✓ |
| VC095 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC096 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC097 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC098 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC099 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC100 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC101 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC102 | 66.7% | 0.0% | ✗ | 0.0% | ✗ |
| VC103 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC104 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC105 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC106 | 75.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC107 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC108 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC109 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC110 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC111 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC112 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC113 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC114 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC115 | 100.0% | 50.0% | ✓ | 0.0% | ✗ |
| VC116 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC117 | 100.0% | 60.0% | ✓ | 80.0% | ✓ |
| VC118 | 100.0% | 28.6% | ✓ | 85.7% | ✓ |
| VC119 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC120 | 66.7% | 33.3% | ✓ | 0.0% | ✗ |
| VC121 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC122 | 100.0% | 0.0% | ✗ | 66.7% | ✓ |
| VC123 | 100.0% | 36.4% | ✓ | 0.0% | ✗ |
| VC124 | 100.0% | 8.3% | ✓ | 8.3% | ✓ |
| VC125 | 100.0% | 0.0% | ✗ | 75.0% | ✓ |
| VC126 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC127 | 100.0% | 0.0% | ✗ | 75.0% | ✓ |
| VC128 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC129 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC130 | 75.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC131 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC132 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC133 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC134 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC135 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC136 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC137 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC138 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC139 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC140 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC141 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC142 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC143 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC144 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC145 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC146 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC147 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC148 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC149 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC150 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC151 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC152 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC153 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC154 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC155 | 100.0% | 0.0% | ✗ | 25.0% | ✓ |
| VC156 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC157 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC158 | 100.0% | 0.0% | ✗ | 66.7% | ✓ |
| VC159 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC160 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC161 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC162 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC163 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC164 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC165 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC166 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC167 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC168 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC169 | 100.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC170 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC171 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC172 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC173 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC174 | 50.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC175 | 66.7% | 0.0% | ✗ | 33.3% | ✓ |
| VC176 | 50.0% | 0.0% | ✗ | 50.0% | ✓ |
| VC177 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC178 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC179 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC180 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC181 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC182 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC183 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC184 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC185 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC186 | 100.0% | 50.0% | ✓ | 0.0% | ✗ |
| VC187 | 100.0% | 66.7% | ✓ | 0.0% | ✗ |
| VC188 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC189 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC190 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC191 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC192 | 100.0% | 100.0% | ✓ | 0.0% | ✗ |
| VC193 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC194 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC195 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC196 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC197 | 100.0% | 33.3% | ✓ | 33.3% | ✓ |
| VC198 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC199 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC200 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC201 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC202 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC203 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC204 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC205 | 100.0% | 0.0% | ✗ | 100.0% | ✓ |
| VC206 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC207 | 100.0% | 100.0% | ✓ | 100.0% | ✓ |
| VC208 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC209 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC210 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC211 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC212 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC213 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC214 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC215 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC216 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC217 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC218 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC219 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC220 | 100.0% | 0.0% | ✗ | 0.0% | ✗ |
| VC221 | 100.0% | 12.5% | ✓ | 12.5% | ✓ |
| VC222 | 100.0% | 25.0% | ✓ | 0.0% | ✗ |
| VC223 | 100.0% | 16.7% | ✓ | 50.0% | ✓ |
Methodology. All scanners run against the same 649-fixture labeled corpus. A VC rule counts as "covered" by another scanner if any of that scanner's rules fires within ±10 lines of our expected range in the correct file. We don't require rule-ID equivalence — the question is capability to detect the class of vulnerability, not taxonomy alignment.
Semgrep. Version 1.86.0 with community rulesets p/security-audit, p/owasp-top-ten, p/javascript, p/typescript, p/react. Semgrep's Pro rules — included free for up to 10 contributors with a Semgrep account — would likely score higher; we ran the open-source CLI with registry rulesets because it needs no account.
XploitScan. Scored with the full 223-rule catalogue — what the CLI, GitHub Action, MCP server and VS Code extension run with a paid plan's API key. Without one, each runs 30 of those rules and would score lower here. We say so because Semgrep and Bearer are scored as free, no-account runs, and a comparison that is free on one side only is not one.
Bearer. 1.50.0, SAST scanner mode, Bearer's built-in security ruleset. Free to use under the Elastic License 2.0 (source-available, not OSI open source); no account required. Bearer describes itself as a SAST tool covering both security and privacy risks.
FP counting. Both Semgrep and Bearer use their own rule taxonomies, so we can't per-rule attribute FPs against our mustNotFire list the way we can with our own scanner. Any finding from those scanners on a clean fixture counts as an FP — a stricter interpretation that puts the third-party scanners at a precision disadvantage we acknowledge. We reviewed by hand, on Nov 3, 2026, each finding on a clean fixture that its label does not expect. Of 396 such findings then, 132 were real false positives. The Nov 4 rule fixes removed 33 of those and did not add any, leaving 99. Of the 339 unexpected findings on the clean fixtures as of Nov 4, 2026, those 99 are the reviewed false positives, which gives 86.3% precision. False positives are counted on clean fixtures only. Recall is computed on every benchmark run.
The Vibe Security Index — the same scanner, pointed at real repositories
Everything above answers one question: is this scanner accurate? That is a question about the tool, and it can only be answered against code where the right answer is already known. The Vibe Security Index answers a different one — what does the code AI app builders actually ship look like? — by running this same deterministic rule set over public repositories generated by Lovable, v0, Bolt and Base44.
The two belong together. A vendor publishing a survey of other people's code is worth exactly the published, checkable accuracy of the tool that produced it, misses included. This page is what earns the right to publish that one.
Aggregate only: the Index never names a repository, an owner, or a URL. See what it found →
Spot a gap?
The corpus is open and the runner is in the repo. If you have a real-world vulnerability pattern that our scanner misses, open a PR with a fixture at xploitscan-benchmark/test-fixtures/ or email admin@xploitscan.com. See the disclosure policy for anything you need to keep private.