These are not hallucinations. The image generator system prompt has been intensely altered to mix all races and genders. The model is probably not inaccurate up until it being misused. The misuse could be at any level of interaction, so it’s very misleading to base it on such an example
Comment on AI or DEI?
stoneparchment@possumpat.io 9 months ago-
it’s true that this would mislead children, but the model could hallucinate about literally anything. Especially at this stage, no one-- children or adults-- should be uncritically accepting what the model states as fact. That said, I agree LLMs need to improve their factual accuracy
-
Although it is highly debated, some scholars suggest Queen Charlotte might have had African ancestry, or that she would be considered a POC by today’s standards. Of course, she reigned in the 17-1800s, but it isn’t entirely outlandish to have a “Queen of Color”, if we aren’t requesting a specific queen or a specific race
-
People of color did live in England in the middle ages? Like not diverse in the way we conceive now, but here are a few papers discussing the racial diversity at the time. It was surely less intermingled than today, but it’s not like these images are impossible
*Other things are anachronistic or fantastical about these images, such as clothing. Are we worried about children getting the wrong impression of history in that sense?
- Of course increasing visibility and representation of all kinds marginalized people is important. I, myself, am disabled, so I care about that representation too-- thanks for pointing it out. I do kinda feel like people would be groaning if the model had produced a Queen with a visible disability, though… I would be delighted to be wrong on this front :)
KeenFlame@feddit.nu 9 months ago
ji59@kbin.social 9 months ago
I know that POC lived in England and it was possible to meet someone like that. But I would prefer if the model gave most possible, most general answers. If I ask for an image of a car I would like to give me four-wheeled red or gray or green car, not three-wheeled pink car just because there exist some car like that.
stoneparchment@possumpat.io 9 months ago
That’s valid! I agree. I think in this case it would be reasonable for the model to give multiple (or like, at least one, jeez) images with white queens. I don’t disagree with anyone in that sense. I just also don’t think it’s worth pitching a fit when the dumbass model that has been trained to show more racial diversity produces (frankly comical) hallucinations.
The ethos of the trainers is a good one. Attempting to counter the (demonstrated, measurable) bias of many models toward whiteness is a good choice. I prefer that the trainers choose to address the bias even if it makes the model make silly mistakes like this. That’s all.