Regression To The Mean

[caleb35]

I suspect this is true, and it's much more true than many want to acknowledge.

I see people complain about their story getting "knocked down" from 4.93 to 4.85, and then they reveal that their story has 20 votes, and I think, you must be kidding.

A score simply doesn't mean much until you have enough votes. Until then, the sample size is so small that you cannot say with any confidence that it represents anything. This is especially true during contests, because the sweeps make things especially unpredictable.

I had a story that followed the pattern MelissaBaby alludes to. It lucked out in reaching its maximum, 4.92, right when the contest closed, and after sweeps had cleared out some nasty votes. I placed in the contest, so that story will forever have a blue "W" next to its title.

But when it placed, it only had something like 27 total votes. Now, two years later, it has 187 votes and a score of 4.49. That's a huge drop. Did it get maliciously downvoted? Maybe, but it seems much more likely that the 27 votes when it placed simply were not representative of the readers' sentiments as a whole, and that the newer number better represents what everybody really thinks of it. I'm likely to be stuck with a score of around 4.49 for this story in perpetuity, because with 187 votes it's not likely to change too much.

I agree completely but I think there's a related issue of not getting enough votes. You're right, 20 votes is way too small of a sample size -- but if that's all the votes that you got on that work, not much you can do about it. Looking at my 11 stories (and one essay) posted since 2021, the number of votes I get has been generally trending; it's even more pronounced if I throw in my one additional story from 2018.

 

[darwin1859]

Well, speaking for the other side, regression to the mean is all well and good until a story’s been up for months, has several hundred votes and has maintained a steady following a score of 4.75 with little or no change, All of a sudden, the score starts falling like a rock. One or two votes a day and the score keeps falling, say to 4.35 before it levels off again. That’s not regression: that’s enemy action. It exists.

How many days did the score drop? Do you recall how many people had rated it when this started?

Because I took some guesses and did an analysis. If you had 4.75 after 100 votes and then a troll came and 1-bombed you once each day for 12 days, your score would drop to 4.35. [So, that scenario is certainly consistent with "enemy action."] But if you had 1000 votes initially, even if trolls dropped two 1's a day, you'd still be at 4.61 on day 20. [So, maybe that scenario is inconsistent with enemy action, unless this happened over a couple months.]

But if it leveled off there and there were more votes, then I don't think it's enemy action (or at least not solely trolls).

BTW, what story are we talking about? Because virtually all of yours are "hot?" (Well done, you!) If your rating later went up over time, I would be more inclined to agree that someone was screwing with you for a period. It's certainly apparent from the bile in some comments that some readers are at least a bit unhinged...

 

[TarnishedPenny]

How many days did the score drop? Do you recall how many people had rated it when this started?

Because I took some guesses and did an analysis. If you had 4.75 after 100 votes and then a troll came and 1-bombed you once each day for 12 days, your score would drop to 4.35. [So, that scenario is certainly consistent with "enemy action."] But if you had 1000 votes initially, even if trolls dropped two 1's a day, you'd still be at 4.61 on day 20. [So, maybe that scenario is inconsistent with enemy action, unless this happened over a couple months.]

But if it leveled off there and there were more votes, then I don't think it's enemy action (or at least not solely trolls).

BTW, what story are we talking about? Because virtually all of yours are "hot?" (Well done, you!) If your rating later went up over time, I would be more inclined to agree that someone was screwing with you for a period. It's certainly apparent from the bile in some comments that some readers are at least a bit unhinged...

DM to follow, thanks.