Skip to content
Back to Leaderboard/SonarQube (CE)
OSSJavaLast updated: 2023-09-05
Manual CVE_R
Based on S_M-C after manual review

5.5%

9 / 165 CVEs

OWASP Benchmark F1

27.0%

OWASP Benchmark v1.2 (Java)

Composite score
Manual 60 / F1 30 / (1-FP)10

15.8

Approx.FP: 55.6%

Real-world detections by scenario
Counts (bar) plus percentages (line) per benchmark scenario.
Approximate false positives
#Dvul and #Dvul&Dpatch for S_F-C / S_M-C.

SF-C

66.7%

#Dvul = 12, #Dvul&Dpatch = 8

Dvul12
Dvul&Dpatch8
Rate66.7%

SM-C

55.6%

#Dvul = 9, #Dvul&Dpatch = 5

Dvul9
Dvul&Dpatch5
Rate55.6%
Claimed vs measured coverage
Over-claim is computed with the Manual (S_M-C) column.

Over-claim

(#Supported - #Detected) / #Supported

90.8%

Speed notes
Qualitative tier plus study observations.

Tier: mid

Roughly 5-15 min depending on project size

  • Run time jumps sharply once projects pass ~50k LoC (all tools).

Manual definition: research teams manually inspected S_M-C detections and removed patch-only hits. Over-claim is evaluated against that Manual baseline.