Comment on As Ubuntu embraces AI, Debian discusses banning all AI-generated code
irotsoma@piefed.blahaj.zone 1 day agoThere are “signatures” that can be detected in code just like universities detect it in student work, online stores detect it in reviews they compensate people for, and some employers detect it in resumes and cover letters.
That said it doesn’t detect it all, and it does rely on the developer to be honest. But it’s also a good indicator when reviewers post questions to the developer and they can’t answer them because they don’t understand the code and things like that. Also, if the developer did not follow coding standards, which LLMs can sometimes have trouble with. And there’s always been malicious code submissions in open source software, so there are other ways.
Anyway, it’s more of a policy discussion at this point, and why LLM generated code is bad. I use them for manipulating data, figuring out syntax, and getting examples, but there are always problems with anything they generate, so I never would allow one to write any production bound code. I see use in the technology, but it’s not creative and it doesn’t understand what it’s writing so it can’t write code with consequences in mind. So it can write Jr level code, but not solid code, at least not without being properly trained on only well written code, and that proper training would be too expensive for an LLM company to actually do it.
Wildmimic@anarchist.nexus 1 day ago
This is an approach with a with a time limit that is nearly exhausted. These detection methods already suffer from abysmal false negative AND false positive rates, and the further LLM progresses, the worse these detection algorithms will become at actually identifying anything written by a coding agent. The point where they become completely usekess is approaching fast - mainly because these tools are great for adversarial training - every detection tool that is actually working provides everything needed to make sure that future models (or even the current one slightly updated).
The best policy isn’t worth the paper it is written on if you can’t detect violations.
irotsoma@piefed.blahaj.zone 1 day ago
Yeah and when teachers or employers use it to reject things outright, it’s really dumb. But if it’s presented as a probability to the user rather than a binary like most LLM results are presented to users, then it is useful. If something has a high probability of being LLM generated then it can cast suspicion and investigation can happen. Unlike how things like facial recognition are used to prove guilt without any investigation when the probability of a match is low, but no better matches were found and the police aren’t told it’s low probability because then the product would look useless in solving crimes for them which is what they want in a product, not an investigation tool. That’s not what they were sold.
Point being. It’s a tool. It’s a metric. And used as such it can be helpful if all of a developer’s contributions get flagged as high probability, then looking more closely at the code can be warranted. If only LLMs themselves were configured to tell the user how probable that their answer is correct, it wouldn’t even be an issue needing investigation.
Wildmimic@anarchist.nexus 1 day ago
You did not understand. The “percentage based” thing is as good as dead. You could just flip a coin and it would have the same accuracy. There will be no tool able to distinguish between human code and agent code going forward, percentage based out not.