What did JFrog find in the SQLite CVE reports?

JFrog Security Research investigated six SQLite CVEs published through a GitHub advisory repository and concluded that all six were fabricated. The advisories cited functions that did not exist in the versions they named, referenced line numbers that exceeded the actual length of the source files, and shipped proof-of-concept payloads that produced no crash and no memory error when run under AddressSanitizer instrumentation. None of the six appeared on SQLite's official advisory page, and the version metadata in the reports contradicted the actual code changes in those releases.

How many advisories from that account were fake?

Of 55 total advisories published by the account, JFrog assessed 54 as fabricated. One contained legitimate vulnerability information. That ratio — roughly 98 percent noise — is what makes this a pipeline problem rather than an isolated bad report.

Were the advisories generated by an LLM?

JFrog ran AI-detection analysis and reported that combining the advisories into a single file triggered AI-generated content warnings, using GPTZero, and concluded that the advisories appear to be AI generated. Detector output is probabilistic and should not be treated as proof of authorship on its own. The stronger evidence is technical: functions that do not exist and line numbers past the end of the file are the characteristic failure mode of text generated without access to the code it describes.

Why did these reports reach a public vulnerability database?

Because the verification step that would have caught them was reduced. NIST scaled back manual analysis of incoming submissions in February 2024. Since then, unverified reports can propagate into public vulnerability data without anyone attempting to reproduce the claimed bug. The bottleneck was never the writing of advisories; it was the reviewing of them, and generation costs collapsed while review capacity did not.

What harm does a fabricated CVE actually cause?

Three kinds. Maintainers — often unpaid volunteers — spend hours investigating a bug that does not exist, which is a direct tax on the scarcest resource in open source. Automated remediation tools generate patches and version bumps for phantom vulnerabilities, propagating churn downstream into every dependent project. And the signal quality of the vulnerability database itself degrades, which matters because the entire software supply chain treats that database as ground truth for what needs patching.

How can a maintainer tell a fabricated advisory from a real one?

Check the code before you check the prose. Confirm the cited function exists in the named version. Confirm the line number is within the file. Run the proof-of-concept under AddressSanitizer or Valgrind and see whether anything actually happens. Check whether the project's own security page lists it. A real report survives all four checks trivially; a fabricated one usually fails the first.

Does this mean AI-assisted vulnerability research is worthless?

No, and that is the wrong conclusion to draw. Machine-assisted analysis has produced genuine findings in real codebases. The distinction is whether a claim was reproduced before it was published. A finding that was triaged, tested against the actual source, and confirmed with a working proof-of-concept is valuable regardless of what tool surfaced it. A finding that was written and submitted without ever being run is noise regardless of what wrote it. The problem is the missing reproduction step, not the tooling.

54 of 55 SQLite Advisories Were Fabricated

Six SQLite CVEs were published as critical. JFrog Security Research tested them. All six were fabricated.

The details are worse than the headline. The advisories cited functions that do not exist in the versions they named. They referenced line numbers past the end of the source files they pointed at. The proof-of-concept payloads, run under AddressSanitizer, produced no crash and no memory error. None of the six appeared on SQLite’s own advisory page, and the version metadata contradicted the actual code changes in those releases.

Then JFrog looked at the rest of the account. Of 55 advisories, 54 were fabricated. One was real.

The post reached the fourth slot on Hacker News with 705 points and 352 comments, and the discussion mostly went where you would expect — toward the question of who wrote them. That is the less interesting question. The interesting question is why they were published at all.

Executive summary

JFrog tested six critical SQLite CVEs and found all six fabricated: nonexistent functions, out-of-range line numbers, proof-of-concepts that do nothing under AddressSanitizer.
Across the same account, 54 of 55 advisories were assessed as fabricated. One was legitimate.
AI-detection analysis flagged the combined advisories as machine-generated. Detector output is probabilistic — the technical evidence is far stronger than the detector evidence.
The structural cause is that NIST reduced manual analysis of submissions in February 2024. Generation got cheap; review did not scale.
The harm is not to SQLite. It is to maintainer attention, to automated remediation tooling, and to the trustworthiness of vulnerability data that the whole supply chain depends on.
This is a denial-of-service attack on maintainer time that requires no attacker and no malice to work.

What a fabricated advisory looks like from the inside

The failure modes JFrog documented are specific, and they are worth learning to recognise because they generalise well beyond this one account.

A cited function that does not exist in the named version is the clearest tell. Real vulnerability research starts from the code — you find the bug by reading the function, so the function is necessarily there. Text describing a bug in a function that was never in that release was not written by anyone looking at that release.

A line number past the end of the file is the same tell in a different form. It is what you get when a plausible-sounding location is produced rather than looked up.

The proof-of-concept is the decisive one. A memory-safety CVE makes a falsifiable prediction: run this input, and the process will do something it should not. AddressSanitizer exists precisely to make that prediction easy to check. JFrog ran the payloads under instrumentation and nothing happened. That is not an ambiguous result. Every one of these advisories could have been refuted in minutes by anyone who tried.

Nobody tried. That is the finding.

The bottleneck was always review

It is tempting to read this as a story about generative text, and partly it is. But the mechanism is older and duller than that.

Publishing a vulnerability advisory has always been cheap relative to verifying one. Writing a paragraph asserting a heap overflow takes minutes. Confirming a heap overflow requires the source, a build, a harness, an instrumented run, and someone who knows what they are looking at. That asymmetry has existed for as long as the CVE system has, and it was held in check by a human review layer that read submissions before they became public data.

That layer thinned. NIST scaled back manual analysis of incoming submissions in February 2024, and the queue-shaped consequence was predictable: things go in, fewer things get looked at, and the ones that were never going to survive scrutiny stop encountering scrutiny.

What changed after that is only the cost of the input side. When producing a fluent, technically-shaped, entirely wrong advisory dropped to near-zero effort, a pipeline already running without a working filter got flooded. The generator is the proximate cause. The missing filter is the actual one — and the filter was missing for two years before anyone noticed the flood.

Why this costs more than it looks like it costs

SQLite is fine. Richard Hipp’s project has one of the most rigorous test suites in existence and is not meaningfully endangered by six imaginary bugs.

The cost lands somewhere else, in three places.

Maintainer attention. A critical CVE against your project is not something you can ignore, and you cannot know it is fake until you have investigated it. Every fabricated advisory converts directly into hours of unpaid volunteer time spent proving a negative. Open-source maintenance is bottlenecked on exactly one resource, and this consumes it at scale. There is no attacker required for this to function as a denial-of-service attack — carelessness distributed widely enough is sufficient.

Automated remediation. Most organisations no longer read CVEs; their tooling does. Dependency scanners ingest vulnerability data and open pull requests, bump versions, and fail builds. Feed phantom vulnerabilities into that machinery and it dutifully generates real churn — patches for bugs that were never there, version pins that solve nothing, CI failures that block shipping. The fabrication happens once; the downstream cost is multiplied by every dependent project.

The data itself. This is the durable damage. The vulnerability database works because it is treated as ground truth. Its entire utility rests on the assumption that an entry means something. Once maintainers learn that most advisories against their project are noise, the rational response is to triage less aggressively — and the day that becomes normal is the day a real critical advisory sits unread for a week. Degrading the signal is worse than any individual false report, because it is what makes future true reports ineffective.

What to do about it, concretely

For maintainers receiving an advisory, the check order is: code, then prose.

Does the cited function exist in the named version? One grep. This alone catches most of it.
Is the line number inside the file? One wc -l. Catches most of the rest.
Does the proof-of-concept do anything? Build with -fsanitize=address, run the payload, watch. A real memory-safety bug announces itself immediately; a fabricated one produces a clean exit.
Does the project’s own security page list it? Absence is not proof, but combined with the above it settles the question.

For the ecosystem, the requirement is more uncomfortable, because it costs money: reproduction should be a precondition of publication, not a courtesy extended afterward. An advisory that has not been reproduced by anyone other than its author is a claim, not a finding, and publishing claims as findings is what produced this.

There is an obvious objection — reproduction does not scale, which is why the review layer thinned in the first place. That objection is correct and it is not a reason to keep publishing unverified data. It is an argument that the current model is underfunded, which was already true before anyone could generate a thousand plausible advisories in an afternoon.

The part that should not be over-read

It would be easy to leave this with “AI-generated security research is garbage,” and that conclusion is both convenient and wrong.

Machine-assisted analysis has surfaced genuine bugs in real codebases, including in heavily-audited ones. The tooling is not the variable that determines whether a report is worth reading. The variable is whether anyone ran it before publishing it.

A finding that was triaged against actual source, tested with a working proof-of-concept, and confirmed to reproduce is valuable no matter what surfaced it. A finding that was written and submitted without ever being executed is noise no matter what wrote it. Those two things have always been different, and the only new development is how cheap it became to produce the second kind at volume.

The detector result, incidentally, deserves the same skepticism. JFrog reported that GPTZero flagged the combined advisories, and that is a reasonable data point to include. It is not proof of authorship — these detectors have well-documented false-positive behaviour on technical writing. The case here does not need it. Functions that do not exist and line numbers past the end of the file are not probabilistic evidence.

The verdict

Fifty-four fabricated advisories reached public vulnerability data because nothing in the path between submission and publication attempted to reproduce a single one of them. That gap opened in February 2024 and was survivable only while writing a convincing fake advisory remained expensive.

It is not expensive anymore. Nothing about the pipeline has changed to account for that, and the next account to try this will not be caught by a security vendor deciding to spend a week on it.

If you maintain something people file CVEs against, add the grep and the sanitizer run to your triage checklist this week. It takes ten minutes and it is currently the only filter that reliably works.