should

when it comes to legality

This needs clarification. Are you asking about the legal status of Character AI’s chatbot, and how its output would be treated w.r.t. to intellectual property rights? Or about the ethical or moral questions raised by machine-generated content, and whether society or law should adapt to answer those questions?

The former is an objective inquiry, which can be answered based on the current laws for a given jurisdiction. The latter is an open-ended, subjective question for which there is no settled consensus, let alone a firm answer one way or another.

I decline to answer the latter, but I think there’s only one answer for the objective law question. IANAL, but existing fanfiction does not imbue its author with rights over characters from another author, at least in the USA. But fanfiction authors do retain copyright over their own contributions.

So if an author writes about the 1920s Mickey Mouse character (now in public domain) but set in a gay space communist utopia, the plot of that novel would be the author’s intellectual property. But not the character itself, which remains public domain. However, character development that happens would be the author’s property, insofar as such traits didn’t exist before.

What aspects of this situation do you envision would require different treatment just because it’s the output from a chatbot? Barring specific language in a Terms of Use agreement that transfers ownership to the parent company of Character AI chatbot, machines – and gorillas – are not eligible to own intellectual property. The author would be the human being which set into motion the conditions for the machine to produce a particular output.

In conventional writing, an author does not relinquish ownership to Xerox Corporation just because the final manuscript was printed using a Xerox-made printer. But just because an author uses a machine to help produce a work, that will not excuse plagiarism or intellectual property violations, which will accrue against the human being commiting that act.

(I express no opinion on whether intellectual property is still a net positive for society, or not)