Given that FOSS licenses are premised on copyright, yes, the same ails would still exist: 1) AI washing of licenses (including transforming one license into another), and 2) the vagueness of whether LLM outputs can be copyright, which threatens the validity of a FOSS license upon that output.

The first ail can be seen even without LLMs: the BSD variants have gone through great pains to remove GPL-licensed code from their base repositories. This basically involves reimplementing utilities and functionality from scratch, using only the ideas that are in common with the equivalent GPL code, but never copying that code directly. This is properly considered a reimplementation, which can then be licensed permissively (eg MIT license).

If an LLM were to train on GPL code but the output were licensed with MIT, then that could be a GPL violation because GPL mandates that remixes continue to keep the GPL license.

Maybe you could avoid this fate by limiting the LLM to only train on permissively licenses code. So that it would be permissive licenses going in, and permissive licenses coming out. No GPL problems here. But that brings us to ail #2.

Some jurisdictions have rules against granting copyright for computer-generated works, in the same vein as works generated by non-humans (eg a macaque). If this LLM fell into this situation, then the output is not copyrightable. And if there is no copyright, a license like MIT or GPL simply cannot apply, because its terms couldn’t be enforced.

Well, to be clear, the copyright parts of those licenses would be unenforceable. Some parts of the license may still be enforced under a contracts claim. But in any case, the things we refer to as “FOSS licenses” cannot attach to uncopyrightable works (with the possible exception of the CC0 license, which is essential the absence of any license).