I did a cursory search on this, and couldn’t find anything, however, is there or could there be an llm trained only on text or data which is in the public domain or is otherwise openly licensed?
Apertus. Or at least so it claims, I haven't checked if their data sources (listed in this paper) corroborate their claims about them.
CameronDev@programming.dev 5 hours ago
Nvidia Nemotron has a published dataset, so if they had copyright data in it it would be very easy to determine.
huggingface.co/…/nemotron-pre-training-datasets
False@lemmy.world 1 hour ago
They’re also some of the better open weight models
CameronDev@programming.dev 51 minutes ago
Ive never tried them tbh, have you? Do they work for code?
ell1e@leminal.space 5 hours ago
huggingface.co/datasets/…/Nemotron-CC-Code-v1 They’re using Common Crawl and there’s no mention of filtering for public domain licenses in this text. Pretty safe to say it’s probably a bad dataset. (In my opinion, I’m not a lawyer. This isn’t legal advice.)
CameronDev@programming.dev 4 hours ago
If that dataset has copyrighted material, nvidia committed a clear copyright violation by publishing it publically. I’m happy to trust that nvidias lawyers have vetted it properly.