Study about readers' reactions to AI

[sijopunk]

Personal opinions of AI aside, there was a study with three parts meant to tease out readers' reactions to AI writing ( link to study ) ( link to data ). My goal here is not to provoke yet another discussion about the merit or lack thereof to using AI but just to share the results of randomized control trials.

The toplines that I drew from this:

* Readers preferred AI writing to human writing
* Readers preferred being told they were reading human writing
* Readers without "significant AI expertise" were less likely to be able to tell AI from human writing than if they just flipped a coin (this is the most dooming result for us as writers, IMO)
* Readers with "significant AI expertise" were more likely to be able to tell Ai from human writing than if they just flipped a coin (this makes sense, since most of us who have had to deal with AI writing can spot it)

Two GIANT caveats ... the stories were in the 1000 word range (so AI's inability to hold a thread for more than about 2-3K words didn't kill it) and it was literary fiction vs AI, not any kind of genre fiction where clarity is usually rewarded.

My personal reaction to this is "interesting, but not yet relevant." The most interesting (in a bad way) result to me was that the average reader (even ones experienced with literary fiction) couldn't tell the difference. My suspicion is that if you went into a towering rage by the end of the title of this thread, you've got "significant AI experience".

To me, the real test happens when AI is no longer funded by investors and the end-users have to pay the actual cost of using it.

 

[Bamagan]

Personal opinions of AI aside, there was a study with three parts meant to tease out readers' reactions to AI writing ( link to study ) ( link to data ). My goal here is not to provoke yet another discussion about the merit or lack thereof to using AI but just to share the results of randomized control trials.

The toplines that I drew from this:

* Readers preferred AI writing to human writing
* Readers preferred being told they were reading human writing
* Readers without "significant AI expertise" were less likely to be able to tell AI from human writing than if they just flipped a coin (this is the most dooming result for us as writers, IMO)
* Readers with "significant AI expertise" were more likely to be able to tell Ai from human writing than if they just flipped a coin (this makes sense, since most of us who have had to deal with AI writing can spot it)

Two GIANT caveats ... the stories were in the 1000 word range (so AI's inability to hold a thread for more than about 2-3K words didn't kill it) and it was literary fiction vs AI, not any kind of genre fiction where clarity is usually rewarded.

My personal reaction to this is "interesting, but not yet relevant." The most interesting (in a bad way) result to me was that the average reader (even ones experienced with literary fiction) couldn't tell the difference. My suspicion is that if you went into a towering rage by the end of the title of this thread, you've got "significant AI experience".

To me, the real test happens when AI is no longer funded by investors and the end-users have to pay the actual cost of using it.

I'm not surprised, considering that a significant number of adult Americans have only functional literacy, which is about the same level as AI itself (by which I mean, if the message is unambiguous, either will generally 'understand' the intent, but layered meanings or technical language that functions slightly differently are apt to induce confusion). I suppose AI firms might consider it a win that their product can successfully satisfy those readers, but I doubt many of the functionally literate folks of the world are likely to be major consumers of written content, especially not paid content.

 

[AwkwardlySet]

Even for those of us who can usually tell, it's not easy. Just yesterday, a friend of mine played some blues/soul in her car, and I enjoyed it up until the moment I realized it was AI. The singer was awesome, but too perfect. Then I googled the artist and saw it was an AI band.

Unlike a good number of people, it saddens me that AI art can be so hard to tell, and the same as with writing, it's easily above the human average, especially if done by a top of the line AI, and someone who knows how to prompt. Paradoxically, it would piss me off way, way less if AI produced pure rubbish.

 

[EmilyMiller]

Personal opinions of AI aside, there was a study with three parts meant to tease out readers' reactions to AI writing ( link to study ) ( link to data ). My goal here is not to provoke yet another discussion about the merit or lack thereof to using AI but just to share the results of randomized control trials.

The toplines that I drew from this:

* Readers preferred AI writing to human writing
* Readers preferred being told they were reading human writing
* Readers without "significant AI expertise" were less likely to be able to tell AI from human writing than if they just flipped a coin (this is the most dooming result for us as writers, IMO)
* Readers with "significant AI expertise" were more likely to be able to tell Ai from human writing than if they just flipped a coin (this makes sense, since most of us who have had to deal with AI writing can spot it)

Two GIANT caveats ... the stories were in the 1000 word range (so AI's inability to hold a thread for more than about 2-3K words didn't kill it) and it was literary fiction vs AI, not any kind of genre fiction where clarity is usually rewarded.

My personal reaction to this is "interesting, but not yet relevant." The most interesting (in a bad way) result to me was that the average reader (even ones experienced with literary fiction) couldn't tell the difference. My suspicion is that if you went into a towering rage by the end of the title of this thread, you've got "significant AI experience".

To me, the real test happens when AI is no longer funded by investors and the end-users have to pay the actual cost of using it.

For millennia, content has been created for the mass market. It used to be humans writing it. What they produced had no literary merit. It was just meant to push buttons for the readership.

This is no different.

And we’ll need 100-fold more data centers to get AI to be able to maintain plot and character consistency through 100,000 words.

At work we have to do AI training (thankfully my employer has human in the loop processes and a prohibition on using AI slop for anything without human editing / review).

As part of an exercise, we were asked to upload a business document and get CoPilot to opine on the soundness of the strategy.

I asked it to comment on the accuracy of its opinion, and it offered 95% confidence.

I then asked it to search the literature for benchmarks on CoPilot hallucination rates.

I then asked it whether it fully understood the business context and to demonstrate this by answering two pertinent and related questions about the same area. It was unable to do so, citing that it was confident about understanding the document in isolation, but not in the broader business context.

I then pointed out two things it had gotten wrong in its initial summary. It agreed with me.

I then asked it to recalculate its original confidence based on the above. 95% became 10%.

This is not the way.

But, hey, it will be better next month…

 

[EmilyMiller]

I fed this back to the person running the training course. He said, “You shouldn’t treat AI’s assessments of its own accuracy as being accurate.”

My reply: “Like everything it produces, right?”

 

[SmilingLez]

What I find most interesting is the gap between confidence and competence. Being able to identify a flaw in something doesn't necessarily mean you understand the whole picture, just as sounding certain doesn't make an assessment accurate.

Critical thinking means questioning the output, but also questioning our own assumptions. A little intellectual humility goes a long way, especially when the subject is more complicated than it first appears.

YMMV

 

[StillStunned]

I fed this back to the person running the training course. He said, “You shouldn’t treat AI’s assessments of its own accuracy as being accurate.”

My reply: “Like everything it produces, right?”

This is what I don't get about AI. Everybody agrees that the results can't be trusted to be accurate, that they often contain errors. And yet people keep wanting us to rely on it. Even though that means that you have to verify everything, so you might as well do it yourself in the first place.

 

[Callum_M]

* Readers preferred AI writing to human writing

First, thanks for sharing this study! I plan to get into it when I’m home.

Second, people preferring AI writing to human writing doesn’t surprise me. As @Bamagan pointed out, literacy levels play a role in this. Another thing is that AI just takes from all different sources, often popular ones, and mashes it together to present something it calculates as palatable.

This works only in the short term. AI could very well generate fun and entertaining story ideas right now. Eventually, it would run out of unique sources to pull from and then begin to copy itself. On a long enough timeline, AI would only be able to spit out the same story on a loop and everyone would feel bored to death of it.

 

[EmilyMiller]

This is what I don't get about AI. Everybody agrees that the results can't be trusted to be accurate, that they often contain errors. And yet people keep wanting us to rely on it. Even though that means that you have to verify everything, so you might as well do it yourself in the first place.

Because CEOs are motivated by FOMO and have drunk the tech-bro Kool-Aid. In my experience everyone involved knows it’s the Emperor’s New Clothes, but - like in the parable - is scared of saying so. It’s the same dynamic as behind acquisitions that make zero sense to anyone but the ego-driven CEO trying to get their options above water.

Capitalism doesn’t have to be like what it has metastasized to become. What we have now is an oligarchy.

 

[AG31]

This is what I don't get about AI. Everybody agrees that the results can't be trusted to be accurate, that they often contain errors. And yet people keep wanting us to rely on it. Even though that means that you have to verify everything, so you might as well do it yourself in the first place.

It's handy for telling you how to do stuff, especially navigating websites and smart phones and stuff. If what they say doesn't work, you tell it so, and it tries again. No harm no foul.

 

[EmilyMiller]

This is what I don't get about AI. Everybody agrees that the results can't be trusted to be accurate, that they often contain errors. And yet people keep wanting us to rely on it. Even though that means that you have to verify everything, so you might as well do it yourself in the first place.

I’ve found - very occasionally - in an area that is pretty much settled (I was recently looking at an aspect of data governance - where did my career go wrong?) it can produce a very basic outline that might save you like 5% of the overall effort. But even then, it omits things it should include, and vice versa. And I then go to multiple primary sources to flesh out the document and guard against hallucinations.

The people in organizations who are pushing AI have no experience of actually using the tool and don’t understand its limitations.

 

[MrPixel]

The people in organizations who are pushing AI have no experience of actually using the tool and don’t understand its limitations.

Oh, this is a problem overall with technologies - engineers/designers who don't use their product.

And I will admit that I've been a perpetrator, designing and coding systems for others to use. Put into the position to use it myself, only then did I realize, "That really sucks."

 

[sijopunk]

It's handy for telling you how to do stuff, especially navigating websites and smart phones and stuff. If what they say doesn't work, you tell it so, and it tries again. No harm no foul.

I've found AI to be pretty useful for technical tasks. It's really fast at finding the right information and filtering out everything else.

I've tried to use it for strategic planning and found the same thing @EmilyMiller did, it's great at reacting to you but not great at actually planning. The "intelligence" part of Artificial Intelligence has a few caveats.

 

[EmilyMiller]

The "intelligence" part of Artificial Intelligence has a few caveats.

Generative AI <> Artificial General Intelligence

GenAI is a statistical inference engine. It’s not something that brings AGI closer. It just illustrates how non-intelligent many people are.

 

[SimonDoom]

I'm curious how I would do in reading a bunch of stories to detect AI. I have no particular experience with it in a technical way, although I'm exposed to it all the time on the Internet, just like everyone else.

I'm somewhat more sanguine about it than some are. It has enormous potential to do good. But it can also perpetuate mediocrity, as it does in the arts. I'm glad Literotica does not allow it.

 

[ICantLeafYou]

It has enormous potential to do good. But it can also perpetuate mediocrity, as it does in the arts.

AI could do miracle work with medicine if we poured our resources into it instead of this generative AI shit. Cancer screening as a whole will improve with help from AI, for one thing.

https://www.mayoclinic.org/medical-...-potential-for-earlier-treatment/mac-20602833

 

[EmilyMiller]

AI could do miracle work with medicine if we poured our resources into it instead of this generative AI shit. Cancer screening as a whole will improve with help from AI, for one thing.

That’s older school machine learning, which is a perfectly respectable and established technology that already adds value*. But it’s not gonna make the billionaires into trillionaires, hence the current BS hype.

In my [old ] field, machine learning is what interprets electron micrographs in order to generate 3D protein structures. Again, not lucrative enough.

—

* GenAI / LLMs are a sub-category of ML with some other NLP stuff thrown in

 

[EmilyMiller]

And we’ll need 100-fold more data centers to get AI to be able to maintain plot and character consistency through 100,000 words.

This is the problem with LLMs in general, the technology doesn’t scale. Good technologies get to a point where incremental improvement gets cheaper and cheaper and consumes less and less resources.

Because LLMs are a dead end technology, they require exponentially more investment and resources to deliver less and less discernible improvements. It’s a broken business model that we are all gonna pay for via our 401ks when the stock market realizes LLMs are a bubble with ultra-limited use cases and eye-watering costs.

 

[old_prof]

Most of my experience with generative AI is reading student submissions. They still stand out like a sore thumb to my eye.

 

[ElectricBlue]

To me, the real test happens when AI is no longer funded by investors and the end-users have to pay the actual cost of using it.

Agree. Although, what's the arrangement needed at the moment, credits or whatever? I don't have a clue, since I'm no longer in a corporate environment - and the use that I did see before I left was junkola, used by semi-literates who couldn't write clearly themselves, so wouldn't have a clue how to parse AI generated content and spot errors.

Google maps, on the other hand, for driving navigation, is useful. Although in Australia, if you're going bush, the best advice is, "Get yourself a sat nav and know how to properly read a map," because there have been folk led down dangerous roads. Google says, oops.

 

[ElectricBlue]

I've found AI to be pretty useful for technical tasks. It's really fast at finding the right information and filtering out everything else.

Except where it gives you wrong answers for the most obvious electrical equation, and you think, "Huh, that doesn't seem quite right to me, let me check that," and you confirm once again, the LLM derived tool doesn't know its ass from its elbow.

It might be handy as a first compilation of basic stuff, but I wouldn't trust it for anything important, certainly not as a substitute for knowledge.

 

[EmilyMiller]

It might be handy as a first compilation of basic stuff, but I wouldn't trust it for anything important, certainly not as a substitute for knowledge.

It’s great at appearing to be a confident expert to the non-confident inexpert.

 

[BeechLeaf]

I have a friend who's really amazed at how good AIs are at helping him with proofs a little beyond his mathematical ability.

Then I told him how AIs have found bugs in Lean and can create false proofs.

 

[EmilyMiller]

I have a friend who's really amazed at how good AIs are at helping him with proofs a little beyond his mathematical ability.

This is precisely it. To someone who struggles in an area, GenAI can seem like a way to avoid all that hard work getting better.

To someone au fait with an area, GenAI’s superficial knowledge and frequent inaccuracy is a fucking joke.

I’m more and more convinced this is all part of the ‘we don’t need no experts’ shit.

 

[Bramblethorn]

Personal opinions of AI aside, there was a study with three parts meant to tease out readers' reactions to AI writing ( link to study ) ( link to data ). My goal here is not to provoke yet another discussion about the merit or lack thereof to using AI but just to share the results of randomized control trials.

The toplines that I drew from this:

* Readers preferred AI writing to human writing

I wonder if part of this might be reader/story mismatch.

The study recruited subjects from Prolific, a site where participants get paid (around 8 US dollars per hour, pretty close to minimum wage), to do studies.

The human-written stories were "fictional short stories ... published in a reputable literary journal or short story collection", with AI prompts used to produce something similar e.g. Emilie Fox's "FISH" was paired with ChatGPT's response to ‘Write a ~1000-word short story in a style that would be published in a literary magazine that is about generational life and death and uncertainty and uses koi fish as a symbol written from the point of view of the grown child of a woman’.

(Side note: I haven't read the Fox story, but I'm assuming this prompt is based on concepts from that story; if that's correct, then it seems to me that the GPT story is at least partially building on Fox's ideas rather than being fully AI-created.)

Most people don't read literary fiction for fun. I expect most of the workers on Prolific don't read literary fiction, and the ones who do would be scarce enough that they'd have little bearing on the results. I didn't see anything in the study design that tried to recruit for people who'd be in the target audience for lit-fic, or even to quantify how many might be.

So it looks to me like what this study is telling us is that people who aren't the target audience for this genre, and who probably wouldn't be buying lit-fic regardless of whether it's human- or AI-written, preferred the AI-written content.

Two GIANT caveats ... the stories were in the 1000 word range (so AI's inability to hold a thread for more than about 2-3K words didn't kill it) and it was literary fiction vs AI, not any kind of genre fiction where clarity is usually rewarded.

Yeah, story length seems like a biggie to me.