From pending purgatory to AI hell

[MillieDynamite]

You'll notice that all the tools AI shows for detecting AI are AI tools.

 

[AwkwardMD]

Yes, the logic does hold up.

So, to generalize, every submission is given a percentage likelihood of being AI generated. It doesn't have to be 100% likelihood to be flagged. How the cutoff is determined may vary, but they all have one. Anything that scores below that line is not flagged as AI. Anything that reaches or exceeds that score is flagged as AI. Simple, right?

Well, not really.

To make up numbers, let's say that the cutoff is 75%. To fudge a bit, let's pretend we can know the true score (ts) of a story. So, in a perfect world, any story where ts >= 75% is flagged as AI while any story where ts < 75% is not. Now, there's two issues that come into play. First, AI detection tools are not good enough to always calculate the ts, so let's call it the calculated score (cs). Second, each time the story is reviewed, it can get a different cs. If you run a big enough set of stories through it enough times, you can determine the range of precision (rp).

Now, the fun part…

If ts+rp<75%, that story will never be flagged as AI. If ts-rp>=75%, that story will always be flagged as AI. However, where ts is within 75%±rp, that story may or may not be flagged as AI, depending on where within that range of precision it's scored.

So, to apply the logic to Literotica, a lot of stories are never going to be flagged as AI. This is especially true for all of the unedited first drafts that are submitted and seem to be approved in bulk on a daily basis. You're also going to have a pool of stories that will always be flagged, whether because they are AI generated or simply check too many boxes. Then, you have those that could go either way. When they score low, they get published. When they score high, they go in the queue for review. Those stories, if resubmitted, may pull a low score and get published, or they may pull a high score and get stuck in Pending again. The lower the ts, the more likely for it to work. The higher the ts, the more times it will probably take to catch an outlier on the low side, but they still can.

So, once again, the logic fits the current situation quite well.

TLDR this for me. Why would a community of writers as diverse in all facets as the AH members have (to the best of my knowledge) exactly 1 rejection (and even that, likely a false positive) in 18 months?

 

[MillieDynamite]

When you suggest they use non-AI non-human detection, that's misinformation. Even a human reader can't be sure if it's human-written or AI-written. And there is no commercial AI detection tool that doesn't use AI. YOu can't find one, you can't name one, because they don't have any for sale or for free out there.

I will not discuss how Lit's AI detector works. Your only choices are to trust me when I say that I side with Lit, or not. Either is fine, but I push back on misinformation.

 

[MillieDynamite]

@AwkwardMD I also side with Lit's right to determine, by whatever means they wish, what they will or won't publish. And they are under no obligation to tell us how they choose to reject any story. But if we are to figure out what we need to do to fix an issue, we're limited to the tools we can find. And I tell you, you won't find a detector out there that doesn't use AI, free or otherwise.

 

[Writer61]

Grammarly has become a standard part of my writing workflow, picking up typos and grammatical errors. I generally reject any word changes that it suggests.

So far, fingers crossed, I have had no rejections for the use of AI.

 

[AwkwardlySet]

I'm sure you just need to PM Laurel politely and respectfully, and wish her happy holidays, and your problem will be solved.

 

[AwkwardMD]

I'm sure you just need to PM Laurel politely and respectfully, and wish her happy holidays, and your problem will be solved.

I have always suspected that if Millie resubmitted with a note, her story would have gone through. I know you were being snide, but here you are not wrong.

 

[AwkwardlySet]

I have always suspected that if Millie resubmitted with a note, her story would have gone through. I know you were being snide, but here you are not wrong.

That's cause Millie is a level 137 Jedi Mistress. I'm not sure ordinary mortals would have it that easy.

On a more serious note, I've been using Grammarly Basic for two years now for spellcheck and punctuation. Never had an AI rejection. Grammarly might contribute to AI rejection even in cases where people do not use any rewording suggestions, but I suspect that's the perfect grammar and punctuation trigger that AI detectors take into account.

 

[RejectReality]

The reason it got processed was because it was entered in the contest. Those aren't found through the normal day-to-day use of the backend dashboard, and are prioritized. Laurel does a search for the keyphrase the instructions tell you to add. It's a manual search that bypasses the normal dashboard. So the long-known bug in the dashboard doesn't prevent it from being found, if it's been in that void. It likely overrides any other filtering that's in place for AI or anything else. It may still be AI flagged, but if there's some mechanism that bottom-piles them, that would likely be overridden.

The same would hold true for any of the challenges where a keyphrase is requested.

 

[IWroteThis]

This is the part which seems suspect.

Sorry, but it's true. Plus, I didn't even mention how these tools are constantly being refined. What would be the point if it didn't result in different scores for the same text?

TLDR this for me. Why would a community of writers as diverse in all facets as the AH members have (to the best of my knowledge) exactly 1 rejection (and even that, likely a false positive) in 18 months?

I love how you didn't read what I said but still want to argue about what I said. Not…

But, to answer your question anyway, it's probably because the AI-flagged queue is the lowest priority. From what I've read here, most of the movement out of that queue is by resubmitting until the story is not flagged. It's also quite possible that it's ordered by AI score rather than date, so the reviewed stories are more likely to be approved than rejected.

 

[IWroteThis]

The reason it got processed was because it was entered in the contest. Those aren't found through the normal day-to-day use of the backend dashboard, and are prioritized. Laurel does a search for the keyphrase the instructions tell you to add. It's a manual search that bypasses the normal dashboard. So the long-known bug in the dashboard doesn't prevent it from being found, if it's been in that void. It likely overrides any other filtering that's in place for AI or anything else. It may still be AI flagged, but if there's some mechanism that bottom-piles them, that would likely be overridden.

The same would hold true for any of the challenges where a keyphrase is requested.

I ended up deleting my submission because it sat in Pending too long and it would have missed the contest even if it had eventually been approved. It was a later submission, just over a week before the end, but others got stories posted that they submitted later than that.

Including the keyphrase didn't appear to change anything.

 

[MillieDynamite]

I sent them a message that I hadn't used AI, but didn't have time to go through it at this time to find the offending part. (Which I'm certain is some clunky sentences that got by my editor.) When I have the time, I will go through it and find the part that caused the rejection. I wasn't rude, and their reply was friendly. I'm by no means a perfect writer, and at times I have to put in a lot of work to fix what I know is wrong. And let me make this clear, I'm not upset about my rejection. I was upset about all the rejections from so many who claimed they didn't use AI.

I believe some of them did use AI, but I'd never tell them they did.

I have always suspected that if Millie resubmitted with a note, her story would have gone through. I know you were being snide, but here you are not wrong.

 

[AwkwardMD]

I love how you didn't read what I said but still want to argue about what I said. Not…

But, to answer your question anyway, it's probably because the AI-flagged queue is the lowest priority. From what I've read here, most of the movement out of that queue is by resubmitting until the story is not flagged. It's also quite possible that it's ordered by AI score rather than date, so the reviewed stories are more likely to be approved than rejected.

I said your logic didn't hold up because AH members don't experience rejections. In response you hypothesized a workflow out of thin air. Acting like you answered me the first time and that I just didn't read it while then quietly tacking on a completely different answer is kind of disingenuous. I mean, everyone can read the interaction. We all know you didn't answer me.

And you technically still haven't. "AI flagging is the lowest priority" is a wild supposition that, I guess, answers the question "Why not the AH", in isolation, without then addressing why so many non-AH members (such as yourself) show up complaining about rejections. You can't have it both ways where it's aggressive when it applies to you but lax when it comes to the rest of us here. There is a clear bubble around some authors and not others, and I am suggesting that the reason is significant, and further that when there are actual bugs and glitches they affect us all equally.

I'm not arguing that you improperly used generative AI to produce your work. I'm not arguing that Lit's AI Detector isn't capable of producing false positives; it definitely can. It sucks that submissions are subject to a system that is capable of false positives at all. That being said, understanding what I do about how it does what it does and the stance that Lit takes on prioritizing human creativity, I support the method they're using.

The next step for you is to resubmit your story with a note in the Notes to the Admin field stating clearly that you did not use AI in the creation of your story.

 

[AwkwardMD]

I sent them a message that I hadn't used AI, but didn't have time to go through it at this time to find the offending part. (Which I'm certain is some clunky sentences that got by my editor.) When I have the time, I will go through it and find the part that caused the rejection. I wasn't rude, and their reply was friendly. I'm by no means a perfect writer, and at times I have to put in a lot of work to fix what I know is wrong. And let me make this clear, I'm not upset about my rejection. I was upset about all the rejections from so many who claimed they didn't use AI.

I believe some of them did use AI, but I'd never tell them they did.

I do not believe that would be necessary. Not for you. Resubmitting it as is, with a note, is the first step.

 

[AwkwardMD]

When you suggest they use non-AI non-human detection, that's misinformation. Even a human reader can't be sure if it's human-written or AI-written. And there is no commercial AI detection tool that doesn't use AI. YOu can't find one, you can't name one, because they don't have any for sale or for free out there.

I can name one, and you're right that it isn't for sale. If it was, who would be the first in line to buy it if not AI companies?

Lit's AI Detector is home grown. What is true of the AI community at large, whether you mean ChatGPT or GPTZero, does not apply to Lit's AI Detector.

 

[AwkwardMD]

That's cause Millie is a level 137 Jedi Mistress. I'm not sure ordinary mortals would have it that easy.

On a more serious note, I've been using Grammarly Basic for two years now for spellcheck and punctuation. Never had an AI rejection. Grammarly might contribute to AI rejection even in cases where people do not use any rewording suggestions, but I suspect that's the perfect grammar and punctuation trigger that AI detectors take into account.

My best guess here has always been that when you say "I only use Grammarly for X, Y, and Z" and other people say "I only use Grammarly for X, Y, and Z", you mean different things and/or one of you is speaking imprecisely.

Not lying, per se. Just very minor differences in your process that affect elements that Lit's AI Detector is designed to notice.

 

[LetsMisBehave]

For what it is worth, I use Grammarly mainly for spell checking and tell the moderators that I have used it. I am English of a certain age and use English vernacular vocabulary from the 70s, 80s and 90s, which no AI can be buggered to use. The only time I have had a problem with a story is when I mucked up the numbering in aeries, and it took three submissions before I was told that was the problem

 

[IWroteThis]

I said your logic didn't hold up because AH members don't experience rejections. In response you hypothesized a workflow out of thin air. Acting like you answered me the first time and that I just didn't read it while then quietly tacking on a completely different answer is kind of disingenuous. I mean, everyone can read the interaction. We all know you didn't answer me.

And I explained how your attempted justification for claiming the logic didn't hold up actually failed, by explaining how the logic could fit perfectly with the situation we are experiencing. Note that I didn't claim to know exactly how it worked, but simply explained how the logic I explained matched the circumstances we're seeing.

Also, I've reviewed your (albeit edited) comment and found not a single question mark within. So, what question did you allegedly ask that I didn't answer? I mean, true, I didn't answer your question, but that has a lot to do with you not asking it.

And you technically still haven't. "AI flagging is the lowest priority" is a wild supposition that, I guess, answers the question "Why not the AH", in isolation, without then addressing why so many non-AH members (such as yourself) show up complaining about rejections. You can't have it both ways where it's aggressive when it applies to you but lax when it comes to the rest of us here. There is a clear bubble around some authors and not others, and I am suggesting that the reason is significant, and further that when there are actual bugs and glitches they affect us all equally.

Way to misquote me in an attempt to create a strawman argument you thought you could win. Too bad you still couldn't pull it off. I did get a laugh out of you calling me a non-AH member, in AH!

Once again, the inconsistency of AI calculations only has so wide of a margin of error, meaning that not all stories have an equal chance of false positives. What I didn't explicitly mention is that writers tend to display a certain level of consistency in their writing, which translates into all of their stories generally scoring in the same neighborhood. Of course, nobody is perfectly consistent, which would explain why those who generally have no problem may have one story get caught.

I'm not arguing that you improperly used generative AI to produce your work. I'm not arguing that Lit's AI Detector isn't capable of producing false positives; it definitely can. It sucks that submissions are subject to a system that is capable of false positives at all. That being said, understanding what I do about how it does what it does and the stance that Lit takes on prioritizing human creativity, I support the method they're using.

No, you're just arguing, with no substance, for the sake of arguing.

The next step for you is to resubmit your story with a note in the Notes to the Admin field stating clearly that you did not use AI in the creation of your story.

No, that was several steps ago, but thanks for the advice. I pulled it back to pending and resubmitted it with that note added. A bit over 15 days later, I deleted it and resubmitted it new with that note. That was nearly a month ago.

I've only submitted three stories since the Pending issue got really bad, so I don't know if the two that got held up were the outliers or if the one that got approved was. I'm finishing up my next one, so I'll soon have another data point.

I would mention that I'm a high-functioning Autistic and how AI detectors have a higher false-positive rate with things written by those on the Spectrum, but I already got lambasted once for daring to speak that truth in AH, so I won't do it again.

 

[MillieDynamite]

I'm certain something in my writing flagged it. I used tricks, repeating something three times in three paragraphs spaced three paragraphs apart (learned from a study on Agatha Christie's works), which works on the subconscious to keep you turning the pages, and nowadays it's flagged as AI writing. There were other things I'd copied from past authors as well. I learned to write by reading and studying the writing I read, including my father's work. And whatever it was that triggered it, I felt I needed to figure it out and stop doing it.

I can name one, and you're right that it isn't for sale. If it was, who would be the first in line to buy it if not AI companies?

Lit's AI Detector is home grown. What is true of the AI community at large, whether you mean ChatGPT or GPTZero, does not apply to Lit's AI Detector.

 

[MillieDynamite]

@AwkwardMD, I don't know if that would've been all I needed to do. My average time from upload to live has been 72 hours since a few months after I started here, when I came back in 2021.

 

[MillieDynamite]

Sometimes, not often, Grammarly suggests something is better. Even so, I don't use it; I rework my sentence until it sounds right to me.

My best guess here has always been that when you say "I only use Grammarly for X, Y, and Z" and other people say "I only use Grammarly for X, Y, and Z", you mean different things and/or one of you is speaking imprecisely.

Not lying, per se. Just very minor differences in your process that affect elements that Lit's AI Detector is designed to notice.

 

[AwkwardMD]

And I explained how your attempted justification for claiming the logic didn't hold up actually failed, by explaining how the logic could fit perfectly with the situation we are experiencing. Note that I didn't claim to know exactly how it worked, but simply explained how the logic I explained matched the circumstances we're seeing.

Also, I've reviewed your (albeit edited) comment and found not a single question mark within. So, what question did you allegedly ask that I didn't answer? I mean, true, I didn't answer your question, but that has a lot to do with you not asking it.

Way to misquote me in an attempt to create a strawman argument you thought you could win. Too bad you still couldn't pull it off. I did get a laugh out of you calling me a non-AH member, in AH!

I actually had not realized I wasn't still talking to the OP, which explains some of the inconsistencies I was struggling to follow. My apologies for this. Today has been a day.

No, you're just arguing, with no substance, for the sake of arguing.

I can appreciate how, from your perspective, this is true. I can appreciate how my unwillingness to provide evidence for understanding Lit's AI Detector is frustrating, confusing, and at times conflicting. It is a difficult thing to talk about without explaining how it works, as doing so runs the risk of exposing the inner workings of a black box to bad actors. I try all the same.

That being said, I do not enjoy having this entire conversation from scratch with every person who comes with a chip on their shoulder and a bachelors in computer science, or a green belt in six sigma. I do not argue for the sake of arguing. I am trying to fill a gap, as I see it, and your mileage may vary on how effective I (or any one non-Literotica admin) can be.

No, that was several steps ago, but thanks for the advice. I pulled it back to pending and resubmitted it with that note added. A bit over 15 days later, I deleted it and resubmitted it new with that note. That was nearly a month ago.

You don't pull a rejected story back to pending. That's not how that works. Rejected stories return to Draft status. Conflating the Pending bug with AI rejections only makes this conversation much harder to have.

I've only submitted three stories since the Pending issue got really bad, so I don't know if the two that got held up were the outliers or if the one that got approved was. I'm finishing up my next one, so I'll soon have another data point.

I would mention that I'm a high-functioning Autistic and how AI detectors have a higher false-positive rate with things written by those on the Spectrum, but I already got lambasted once for daring to speak that truth in AH, so I won't do it again.

This isn't how it works. It might be how it works out in the larger world of AI detection, but Lit's home grown system does not care how spicy your neuro is. It's looking for something very specific.

 

[AwkwardMD]

@AwkwardMD, I don't know if that would've been all I needed to do.

Only one way to know for sure.

 

[AwkwardMD]

Sometimes, not often, Grammarly suggests something is better. Even so, I don't use it; I rework my sentence until it sounds right to me.

Good for you. Stick to your guns. Nobody else can write what you will write, and the world will be richer for it.

 

[MillieDynamite]

Yup. But don't think I want to find out. I've felt like one of the blessed ones, anywhere, outside of my own adopted kinfolk.

Only one way to know for sure.