Regression To The Mean

[caleb35]

I suspect this is true, and it's much more true than many want to acknowledge.

I see people complain about their story getting "knocked down" from 4.93 to 4.85, and then they reveal that their story has 20 votes, and I think, you must be kidding.

A score simply doesn't mean much until you have enough votes. Until then, the sample size is so small that you cannot say with any confidence that it represents anything. This is especially true during contests, because the sweeps make things especially unpredictable.

I had a story that followed the pattern MelissaBaby alludes to. It lucked out in reaching its maximum, 4.92, right when the contest closed, and after sweeps had cleared out some nasty votes. I placed in the contest, so that story will forever have a blue "W" next to its title.

But when it placed, it only had something like 27 total votes. Now, two years later, it has 187 votes and a score of 4.49. That's a huge drop. Did it get maliciously downvoted? Maybe, but it seems much more likely that the 27 votes when it placed simply were not representative of the readers' sentiments as a whole, and that the newer number better represents what everybody really thinks of it. I'm likely to be stuck with a score of around 4.49 for this story in perpetuity, because with 187 votes it's not likely to change too much.

I agree completely but I think there's a related issue of not getting enough votes. You're right, 20 votes is way too small of a sample size -- but if that's all the votes that you got on that work, not much you can do about it. Looking at my 11 stories (and one essay) posted since 2021, the number of votes I get has been generally trending; it's even more pronounced if I throw in my one additional story from 2018.

 

[darwin1859]

Well, speaking for the other side, regression to the mean is all well and good until a story’s been up for months, has several hundred votes and has maintained a steady following a score of 4.75 with little or no change, All of a sudden, the score starts falling like a rock. One or two votes a day and the score keeps falling, say to 4.35 before it levels off again. That’s not regression: that’s enemy action. It exists.

How many days did the score drop? Do you recall how many people had rated it when this started?

Because I took some guesses and did an analysis. If you had 4.75 after 100 votes and then a troll came and 1-bombed you once each day for 12 days, your score would drop to 4.35. [So, that scenario is certainly consistent with "enemy action."] But if you had 1000 votes initially, even if trolls dropped two 1's a day, you'd still be at 4.61 on day 20. [So, maybe that scenario is inconsistent with enemy action, unless this happened over a couple months.]

But if it leveled off there and there were more votes, then I don't think it's enemy action (or at least not solely trolls).

BTW, what story are we talking about? Because virtually all of yours are "hot?" (Well done, you!) If your rating later went up over time, I would be more inclined to agree that someone was screwing with you for a period. It's certainly apparent from the bile in some comments that some readers are at least a bit unhinged...

 

[TarnishedPenny]

How many days did the score drop? Do you recall how many people had rated it when this started?

Because I took some guesses and did an analysis. If you had 4.75 after 100 votes and then a troll came and 1-bombed you once each day for 12 days, your score would drop to 4.35. [So, that scenario is certainly consistent with "enemy action."] But if you had 1000 votes initially, even if trolls dropped two 1's a day, you'd still be at 4.61 on day 20. [So, maybe that scenario is inconsistent with enemy action, unless this happened over a couple months.]

But if it leveled off there and there were more votes, then I don't think it's enemy action (or at least not solely trolls).

BTW, what story are we talking about? Because virtually all of yours are "hot?" (Well done, you!) If your rating later went up over time, I would be more inclined to agree that someone was screwing with you for a period. It's certainly apparent from the bile in some comments that some readers are at least a bit unhinged...

DM to follow, thanks.

 

[Chase_Storm]

Well, speaking for the other side, regression to the mean is all well and good until a story’s been up for months, has several hundred votes and has maintained a steady following a score of 4.75 with little or no change, All of a sudden, the score starts falling like a rock. One or two votes a day and the score keeps falling, say to 4.35 before it levels off again. That’s not regression: that’s enemy action. It exists.

Don't let anyone on this site tell you this isn't a real thing. Of course, there will be some standard regression. The longer a story is out there, the more eyeballs are on it, and a larger audience will have more varying taste than those who read it first (likely people who are fans of your work).

But when you go to sleep and a story is at one point, and wake up the next day to find it at another, and do the math and determine there were three 1 star votes overnight, that's not regression. That's intentional.

The good news is that if your story is of high quality, in time you'll see the opposite - progression. It's just sad that we need to go through this as authors contributing work for free.

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

 

[YmaOHyd]

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

I have, yes. If I click into a story and the first few paragraphs have really bad spelling and grammar errors (not the sorts of nonstandard grammar we talk about here. I mean stuff like "Im pissedoff",he said) I'll rate it one star. That stuff shouldn't be published at all.

I'll also hit the one-star for something that, in my judgement, is evading the letter of the law. There was a story that has a female character who exists just to be had-sex-with. No personality, agency, life outside that scene. It's emphasized multiple times that she's 18 (mostly by calling her "the 18 year old") but her body is described in ways that would be more appropriate for a child.

 

[SimonDoom]

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

I can't recall ever having given a story a 1 vote, because like you I typically click out of a story right away if I don't like it. I have a policy of not voting on a story if I don't read it until the end. I almost never finish 1-star stories to the end so I never vote on them.

But there are plenty of stories that I consider 1-star stories. I don't think anyone should feel wrong to give a vote they think a story genuinely deserves. If you really don't like it, give it the vote you think it deserves. You are conveying useful information to prospective readers when you do so. The system isn't helped by readers being timid about giving a story the bad score they think it deserves.

 

[TarnishedPenny]

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

I cannot recall ever giving a 1 or a 2 and would be hard-pressed to think of a story I gave a 3 for. Life's too short to be wasting my time with bad prose; if it's not good, I walk away without finishing or scoring.

As to 1-star votes being flags, Laurel has gone on record as saying that 1s are votes, too. It's hard to argue with the philosophy and the sweeps do tend to catch up.

Edit - shaky hands on too small a screen. Second edit - 'cause I forgot, too.

 

[Chase_Storm]

I have, yes. If I click into a story and the first few paragraphs have really bad spelling and grammar errors (not the sorts of nonstandard grammar we talk about here. I mean stuff like "Im pissedoff",he said) I'll rate it one star. That stuff shouldn't be published at all.

I'll also hit the one-star for something that, in my judgement, is evading the letter of the law. There was a story that has a female character who exists just to be had-sex-with. No personality, agency, life outside that scene. It's emphasized multiple times that she's 18 (mostly by calling her "the 18 year old") but her body is described in ways that would be more appropriate for a child.

That's fair. I usually click out of the stuff that I find tough to read due to spelling, grammar, punctuation, etc. But I'm with you on the other part, for sure.

 

[alohadave]

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

Yes, I'll 1 star bad stories. Usually they need to be egregiously bad, either in story or in SPaG.

Though even bad stories sometimes will get a pass if I can tell what they were trying to do, but didn't quite make it there.

So no, not all 1 star votes are trolls.

 

[Britva415]

I don't 1-star stories I read all the way to the end. I don't 1-star every story I nope-out of early. I don't 1-star stories because I don't like where it went or what the content turns out to be (after checking tags, category, title, description and any preface for clues).

I don't even 1-star stories just for being written in 2nd person voice

I 1-star stories which I quit reading because the writing quality is such absolute shit that I can't abide continuing.

 

[Off_base]

I've always considered my votes to be recommendations for other readers. I'm not experienced enough to give other authors a score on their writing skills.
I've never read anything that made me vote less than a 3, and even that has only been a couple times. I read a lot based on recommendations though, so I don't find a lot of truly bad stories.

 

[Bramblethorn]

Don't let anyone on this site tell you this isn't a real thing.

Is anybody actually saying that though?

I've seen countless iterations of downvoting discussions in this forum, and I can't recall anybody ever attempting to argue that there's no such thing as malicious downvoting. Only that not everything people assume to be malicious downvoting actually is.

One other thought/question: Do you ever 1 star a story yourself? I can't personally imagine reading through an entire story, getting to the end, and then awarding it 1 star. If I really hate something, I know it pretty fast, and I'm clicking out and moving on. Point being, I wonder if 1 star votes in and of themselves should be a red flag to the powers that be?

If somebody knows pretty fast that a story is awful - and I agree with you that it's often possible to make that assessment early - why should there be an onus on them to trudge through the whole thing before rating it according to their opinion? Clicking through to one-star it is perfectly valid at that point.

 

[IWroteThis]

It seems like a lot of folks here would be happier if stars 1-4 were greyed out.
At least, until they realized that everybody else would get that HOT icon on their perfect 5* story as well…

I've always considered my votes to be recommendations for other readers.

And that is the true purpose of a rating system, whether on stories here on Literotica or some product for sale on Amazon. Of course, it only works when people give honest ratings consistently, good or bad.

 

[SimonDoom]

If somebody knows pretty fast that a story is awful - and I agree with you that it's often possible to make that assessment early - why should there be an onus on them to trudge through the whole thing before rating it according to their opinion? Clicking through to one-star it is perfectly valid at that point.

I don't do it this way in practice, because I follow the "I vote only if I finish the story" rule, but I have to admit there's no strict logical reason to do that. As long as you've read enough of the story that you are confident in your assessment of it, then your vote serves the purpose a vote is supposed to serve--getting incorporated into the overall score that presumably conveys information to prospective readers.

 

[iwatchus]

One of the problems is the second role we are trying to use the ratings for.

The readers need the ratings as a crude tool. They care very much between a 3.0 and a 4.5 but most don't really care about 4.3 vs 4.4. For readers (or at least for me when I was solely a reader), I just want to know is this well enough written to not turn me off. I was always more sensitive to writing than exactly matching my kink, so I wanted ratings to reflect that.

To be honest, as an author, I don't care much between a 4.3 and a 4.4, I'm horrified by either. Many of us who worry about ratings here are worried between a 4.75 and a 4.9. I resent that right now, I story that hits 100 votes at 4.9 will end up with a worse score than one that hits 100 votes at 4.84. Maybe it will eventually catch up, my oldest story is just over 9 months old. My oldest good story is more like 7 months old, so it still needs time to mature. I get that. (Note that in a mega category, like T/I or LW, the votes reach thousands so quickly that it's a whole different game.)

But as a reader, do I care about the difference between actual 4.75 and actual 4.9? No, probably not. Either of them is good enough for me and the subtleties of the style ad kink will dominate any real differences in quality.

As much as we want to understand how the good stories shake out against each other, the ratings will never be optimized for that. Their job is to help readers find stories, not bragging rights by us.

That said, I still hate whoever the fuck feels the need to continually 1-bomb the top lists and the prize winners. Hate them with every cell in my body. With the fire of that volcano Millie showed the other day. I mean they really annoy me. Whether they are trying to defend the positions or they just resent successful stories. I'm not sure which possibility is worse.

 

[Bramblethorn]

It seems like a lot of folks here would be happier if stars 1-4 were greyed out.
At least, until they realized that everybody else would get that HOT icon on their perfect 5* story as well…

Also we should add a 6th star option for when a story is extra special.

And then grey out 5-star votes because only a troll would give the minimum score.

 

[TarnishedPenny]

Well, I still think @EmilyMiller had a good idea. Leave the system essentially unchanged, but not count the top and bottom 5%. All of a sudden, the trolls and the vote-padders wouldn't have the same effect.

 

[EmilyMiller]

Well, I still think @EmilyMiller had a good idea. Leave the system essentially unchanged, but not count the top and bottom 5%. All of a sudden, the trolls and the vote-padders wouldn't have the same effect.

It’s not a novel idea. It’s the basis of most approaches to similar issues. And it’s not hard to implement.

 

[Bramblethorn]

Well, I still think @EmilyMiller had a good idea. Leave the system essentially unchanged, but not count the top and bottom 5%. All of a sudden, the trolls and the vote-padders wouldn't have the same effect.

Wouldn't they?

Scenario #1: we have a story with 100 votes: 5 x 2-star, 50 x 4-star, 45 x 5-star. Then somebody comes along and pads it with 20 more 5-star votes.

Under the current system: the score is initially 4.35, increasing to 4.46 after those 20 votes are added, an increase of .11.

Under this proposed system: the score is initially 4.44, increasing to 4.55 after those 20 votes are added, an increase of .11.

Scenario #2: same start but instead of padding it, somebody drops five more 1s on it, leaving it with 5 x 1-star, 5 x 2-star, 50 x 4-star, 45 x 5-star.

Current system: score drops from 4.35 to 4.19, a drop of .16.

Proposed system: score drops from 4.44 to 4.32, a drop of .12.

Scenario #3: story initially has 5 x 1-star, 50 x 4-star, 45 x 5-star; then somebody drops 5 more 1-stars on it.

Current system: score drops from 4.30 to 4.14 (-.16).

Proposed system: score drops from 4.44 to 4.26 (-.18).

Depending on the scenario, excluding the top and bottom 5% might slightly reduce the impact of trolls and vote-padders (Scenario #2). Or it might do nothing (Scenario #1). Or it might slightly increase it (Scenario #3). It doesn't seem worth the added complexity, although it would push the average score up by about .1 which I guess would be popular with authors here.

The fundamental difficulty here is that even if you set your outliering percentage high enough to exclude all the troll votes this way, they push the next-lowest votes from their previous "excluded" to "included" status, which still hits the score pretty hard. (Meanwhile on stories which aren't being trolled, legitimate low votes would be unfairly excluded, which makes the scores less informative to readers.)

 

[SimonDoom]

Well, I still think @EmilyMiller had a good idea. Leave the system essentially unchanged, but not count the top and bottom 5%. All of a sudden, the trolls and the vote-padders wouldn't have the same effect.

I just don't see objective evidence that there is a problem to correct, and, as Bramblethorn explains, there's reason to believe "trolls" will find ways to game any alternative system that is adopted.

Added to that is the obvious problem that your proposal and Emily's necessarily will mean that SOME legitimate 1 votes and 5 votes will be eliminated. That's a problem. Why should any legitimate voter's vote be discounted just because there is some undefinable number of "illegitimate" extreme votes that do not, objectively speaking, create a problem that demands correcting?

I look at your body of work, and Emily's, and Awkwardlyset's, and those of some of the other authors who've pressed to change the system, and I don't see a problem. By any objective standard you are doing very well in terms of votes, views, and scores. You have all received many, many favorable comments. Your average scores are higher than mine. I know I've received many 1 votes. I don't care. I think the answer is to just keep writing and not to care too much when the scores of stories here and there get hit with strategic or reflexive downvotes. Focus on the big picture. What's always puzzled me is that some of those who press most ardently for change are the very authors whose "big pictures" are great.

Nobody denies that strategic voting occurs. The existing sweep system attempts to deal with it. Based on what I've seen, in the aggregate that system seems to work OK. While it may be an annoyance for particular stories here and there, in the bigger picture it isn't preventing authors like you and others from getting a lot of recognition and appreciation for your work. Do any of us have a right to expect anything more than that?

In nine years of these discussions I have never seen somebody propose a solution that is better than the current system, and in the absence of clear evidence of an objective problem, and in the face of the site's evident intention NOT to do anything that makes voting more difficult for readers (an intention I fully endorse, and which I suspect most authors don't care about) the proposals for change would seem to be a lot of wasted effort.

 

[RejectReality]

Nothing that will hamper trolls in any significant way at the point of casting the vote will not also severely discourage your average reader from bothering to vote at all. Absent all the "can't be bothered" votes, the trolls ( who are far more motivated ) have an easier time tanking scores with lower vote totals.

Using any math other than a simple average is going to draw cries of witchcraft. People don't even really understand how the averages work because half the time when you do the math on their complaints of multiple 1-bombs, you realize it couldn't have possibly been. Throw anything else at them and you're inviting a torch-wielding mob that believes the site is cheating them.

And the trolls? They'll just keep bombing until the score gets to where they want it regardless of how you manipulate it with math. The only math they're doing is >my favorite or >4.50. You might get a few more bombs removed in the next sweep because they have to work harder and leave more traces, but they'll just be right back to smear their feces all over it a day later.

The one and only way to counter trolls is to keep producing quality work that increases your following and dilutes the troll bombs with rising vote totals.

( And avoiding LW like it's radioactive plague )