In a study published in the journal Judgment and Decision Making, 1,682 adults were asked to read one of six short stories, three of which were written by humans and three by ChatGPT. Each AI story had a similar theme to one of the human-authored works.
The team told participants whether their story was written by a human or AI, but this information was not always correct. The researchers then asked participants to rate how absorbing and engaging they found the story, and its quality.
Participants who read an AI-generated story rated it as more absorbing and of higher quality than those who read a story written by a human. However, participants gave higher ratings to stories they had been told were written by people.
Dr Deena Skolnick Weisberg, a senior author of the research from Villanova University in Pennsylvania, said: “AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly.”
However, Weisberg, who is herself a creative writer, said that did not mean writing should be left to AI, noting that novels produced by tech were probably going to be different from those written by humans.
“We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn’t mean that there’s no longer space for us to appreciate the process of human creativity,” she said.
Look, man, I have been in writing groups and classes with people of all ages and skill levels. I have read some truly horrendous stories. Is the infinite word predictive machine going to write a better story than the worst, least skillful writer? Easily. Is it going to produce a better story than some of the more skillful writers? Absolutely not.
Worth noting also, I’m not saying someone who is noted for selling a lot of books, or even someone in an MFA program for writing. I am saying, LLMs are unlikely to produce a better work than me, and I am just some guy who is pretty skilled and practiced at writing. Maybe the one or two most skilled in a group of hobbyists who meet and share feedback and try to get better.
AI is always going to shoot for the absolute middle in terms of quality, and can definitionally produce nothing new or innovative.
ds12@beehaw.org 1 week ago
Reading the journal paper this article referenced, I was a little disappointed that there weren’t more variables accounted for regarding the participants and the texts. With the participants, factors I see commonly accounted for are levels of education, socioeconomic indicators (say income), or how many books the participants have read in the past month or something similar.
Regarding the text, some form of analysis to then account for or attempt to equalise the AI-generated texts so that they’re similar to the author-written ones would have been nice - like average sentence lengths or average syllables per word.
However, this is still a data point (and a scary one) that it can be very hard to distinguish AI-generated content from human-generated ones…
maegul@lemmy.ml 1 week ago
To be honest, this moment was always coming or is yet to come for anti-AI folks.
The real issue isn’t whether it’s good or bad. The reason it’s such a hot thing is because generally it’s good enough to get people interested. And that’s very unlikely to go away.
Too interested? Sure.
But machines that are better than people at something people value about themselves. That’s where we’re heading if we’re not there already.
Which unlocks a massive question: What kind of world do you want to live in and how much are you willing to do to maintain it? Just about everything else is no longer a given. Not our value or importance, not our position at the top of the food chain, not our democracy or wealth, or the continuous growth of capitalism, or even our continued existence.
We’re basically at the beginning of an AI apocalypse movie. You can’t presume that you know the ending. You have to figure out how you want it to end. And not just for you, but your children and grandchildren children. Cuz it’s all at stake now.
themachinestops@lemmy.dbzer0.com 1 week ago
I read The machine stops short story and I really hope we don’t head that direction, but it seems we are.
p03locke@lemmy.dbzer0.com 1 week ago
There’s only two choices:
That’s it. Those are the choices. Most are sucking in that copeium for the second option.
XLE@piefed.social 1 week ago
I think you’re taking the doomer narrative a bit too seriously, especially with that final paragraph. The prominent pushers of said narrative usually have something they want to sell, or a real-world issue to distract you from. Some (Yudkowsky) from their cults, some (Altman, Amodei) to enable monopoly or drive profits. Short stories do not an AI apocalypse make.
Sources need to be vetted and held to account when they are wrong or make unfalsifiable statements.
spit_evil_olive_tips@beehaw.org 1 week ago
I think it’s even worse than that. quoting from the study’s abstract:
hadn’t heard of Profilic before, they describe themselves as:
“Prolific connects researchers, AI developers and product teams with real people who take part in studies online. It’s built for collecting high-quality, human data quickly and reliably, from training AI models to running behavioral research. Every participant is vetted, active and fairly paid, so you get trustworthy results without the usual noise.”
so the common theme is that 100% of the study participants have opted-in to this “help evaluate AI as a paid side-hustle” thing. that is going to introduce a huge selection bias that is definitely not representative of the population as a whole.
BaroqueInMind@piefed.social 1 day ago
In this economy, I’m not going to sit down and do evaluation work for you to sell to another company unless you fucking compensate me for my time. Who is willing to help do this study for free?
ds12@beehaw.org 1 week ago
Oof, I didn’t even catch or look up the Prolific bit!
Onomatopoeia@lemmy.cafe 1 week ago
AI text can be very hard or nearly impossible to deteft from human generated, at times, depending in the model used.
It’s rather surprising.
ByteSorcerer@beehaw.org 1 week ago
People are also way worse at distinguishing them than they think they are. And it’s the same story with images.