Fake positives in AI detection tools, and the fear of being human.

[SkyBubble]

One it’s “mostly” new people who seem to be flagged, even though in this very thread Millie mentioned being flagged.

Mostly, but not exclusively. I've been here 19 years and I've been flagged for 100% human-written stories.

 

[AwkwardMD]

Hey ya'll! I worked out every other day for the past two weeks and worked out 10 times! Hope that helps!

 

[SkyBubble]

I also said she would have an exceptional understanding of stories from her experience and because of the number of stories a day she reviews. I do not think I could do her job; I doubt that many of us could.

Give the number of submissions to this site (estimated around 200-250 daily), I don't think Laurel can possibly review them all. That's why she uses an automated system.

 

[ShelbyDawn57]

I just went to an AI detector and entered

To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them.
To die—to sleep, No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to: 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep, perchance to dream—ay, there's the rub:

It told me it was 71.08% AI-generated.

With all the rumors and theories of Willie The Bard not being real, surprised, I'm not...

 

[ShelbyDawn57]

I just went to an AI detector and entered

To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them.
To die—to sleep, No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to: 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep, perchance to dream—ay, there's the rub:

It told me it was 71.08% AI-generated.

FWIW, I checked Tomorrow and tomorrow and tomorrow...
It got 77.33% AI

As I mentioned previously, Shakespeare was obviously a fraud...


---
Tomorrow, and tomorrow, and tomorrow,
Creeps in this petty pace from day to day,
To the last syllable of recorded time;
And all our yesterdays have lighted fools
The way to dusty death. Out, out, brief candle!
Life's but a walking shadow, a poor player,
That struts and frets his hour upon the stage,
And then is heard no more. It is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.

 

[NotWise]

I just went to an AI detector and entered

To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them.
To die—to sleep, No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to: 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep, perchance to dream—ay, there's the rub:

It told me it was 71.08% AI-generated.

I pasted your text into GPTZero, and it said Human; 3% AI.

I edited it to more common punctuation and capitalization and it came back entirely Human; 0% AI.

 

[AwkwardMD]

I pasted your text into GPTZero, and it said Human; 3% AI.

I edited it to more common punctuation and capitalization and it came back entirely Human; 0% AI.

I was expecting you to pay this off with "by simply doing this one weird trick"

You'll make a million dollars.

 

[WCSGarland]

All detectors are not created equal I guess.

 

[NotWise]

FWIW, I checked Tomorrow and tomorrow and tomorrow...
It got 77.33% AI

As I mentioned previously, Shakespeare was obviously a fraud...


---
Tomorrow, and tomorrow, and tomorrow,
Creeps in this petty pace from day to day,
To the last syllable of recorded time;
And all our yesterdays have lighted fools
The way to dusty death. Out, out, brief candle!
Life's but a walking shadow, a poor player,
That struts and frets his hour upon the stage,
And then is heard no more. It is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.

It would be good to know what AI detector's y'all are using. They don't work very well.

I pasted your text to GPTZero, and it came back probably human, 23% AI.

I edited capitalization and punctuation and it came back Human, 4% AI.

I'm not promoting GPTZero. It's just one of the first ones that pops up on a Google search, but it's better than whatever you used.

 

[NotWise]

I was expecting you to pay this off with "by simply doing this one weird trick"

You'll make a million dollars.

Are capitalization and punctuation things that you look at as indicating Human .vs. AI? I wouldn't have thought it would make that much difference.

 

[AwkwardMD]

Are capitalization and punctuation things that you look at as indicating Human .vs. AI? I wouldn't have thought it would make that much difference.

Those are largely matters of style, and I do not believe that style is what triggers Lit's AI Detector.

 

[ShelbyDawn57]

It would be good to know what AI detector's y'all are using. They don't work very well.

I pasted your text to GPTZero, and it came back probably human, 23% AI.

I edited capitalization and punctuation and it came back Human, 4% AI.

I'm not promoting GPTZero. It's just one of the first ones that pops up on a Google search, but it's better than whatever you used.

I used GPTZero

 

[NotWise]

I used GPTZero

I wonder how we got such different results from the text you posted.

 

[AwkwardMD]

I wonder how we got such different results from the text you posted.

The easy answer is that its an unreliable system.

Bramblethorn demonstrated this previously, in conversations with AlexBailey, by using attempting to get ChatGPT to describe a specific flag. Multiple repetitions of an identical prompt produced descriptions that sometimes got some details right but just as often invented details. More importantly, as I recall, no two attempts produced the same description.

 

[SilkyVine]

 

[NotWise]

The easy answer is that its an unreliable system.

Bramblethorn demonstrated this previously, in conversations with AlexBailey, by using attempting to get ChatGPT to describe a specific flag. Multiple repetitions of an identical produced descriptions that sometimes got some details right but just as often invented details. More importantly, as I recall, no two attempts produced the same description.

I don't think that's the answer.

I put the text from @ShelbyDawn57's post into ZeroGPT (not GPTZero) and got the same 77.33% AI that he reported. ZeroGPT is the same site that reported large AI% for 1984 and Ready Player One to @Matt4224 way back at the beginning of this thread.

The problem here is that ZeroGPT clearly does not work. GPTZero is a different site, and produces different results. Unfortunately the two sites are very easily confused.

 

[ShelbyDawn57]

I wonder how we got such different results from the text you posted.

I ran it again. Try with the exact text I used. Maybe your version was different somehow. Or, AI is just unreliable.

 

[ShelbyDawn57]

I wonder how we got such different results from the text you posted.

My bad, I used zeroGPT, not GPTZero.

 

[TheLobster]

View attachment 2560708

Still more accurate than some of the stories here.

 

[AwkwardMD]

Please stop attempting to publically disassemble how AI detectors work. No good will come of this.

 

[MillieDynamite]

Every Middle English text I've run through Checkers always has a high AI quotient. But they aren't saying it is 70% written by a machine, they are saying only a 70% chance it is written by a human, and that 30% of the might, I say might, be written by an AI. Let us please put it the right way. The image below is what I got on a piece of Willy Shakespeare I ran through.

 

[Tomh1966]

Using Grammarly beyond spell check has been shown in multiple threads to cause false positives.

 

[NotWise]

Using Grammarly beyond spell check has been shown in multiple threads to cause false positives.

Those would not be false positives. Grammarly is a generative AI.

 

[Tomh1966]

Those would not be false positives. Grammarly is a generative AI.

Point taken but its often not a huge shift as a percent overall words. Still... dont use it.

 

[OverconfidentSarcasm]

Or, AI is just unreliable.

*gasp* The mere THOUGHT is OUTRAGEOUS!