- False and junk bug reports, written by AI tools, are on the rise
- Reading them all hits maintainer time and energy, report warns
- One maintainer called the alerts “AI slop”
Security report triage worker Seth Larson has revealed many open source project maintainers are being hit by “low-quality, spammy, and LLM-hallucinated security reports.”
The AI-generated reports, often inaccurate and misleading, demand time and effort to review, which is taking away from the already limited time open source software developers and maintainers typically have given that they contribute on a volunteer basis.
Larson added maintainers are typically discouraged from sharing their experiences or asking for help due to the security-sensitive nature of reports, making the unreliable security reports even more time-consuming.
OSS maintainers are being hit hard
Maintainers of open source projects like Curl and Python have faced “an uptick” in such reports recently, revealed Larson, who points to Curl maintainer Daniel Stenberg’s post of a similar nature.
Responding to a recent bug report, Stenberg criticized the reported for submitting an AI-generated vulnerability claim without verification, adding that this sort of behavior adds to the already stretched workload of developers.
Stenberg, who is a maintainer for Curl, said: “We receive AI slop like this regularly and at volume. You contribute to unnecessary load of curl maintainers and I refuse to take that lightly and I am determined to act swiftly against it… You submitted what seems to be an obvious AI slop ‘report’ where you say there is a security problem, probably because an AI tricked you into believing this.”
While the problem of false reports like this is nothing new, artificial intelligence has seemingly worsened it.
AI-generated bug reports are already proving to be draining on maintainers’ time and energy, but Larson said that continued false reports could discourage developers from wanting to contribute to open source projects altogether.
To address this issue, Larson is calling on bug reports to verify their submissions manually before reporting, and to avoid using AI for vulnerability detection in the first place. Reporters who can provide actionable solutions rather than simply highlighting vague issues can also prove their worth to maintainers.
For maintainers, Larson says they should not respond to suspected AI-generated reports to same themselves time, and ask reporters to justify their claims if in doubt.