Regression To The Mean

[sirhugs]

Opps, my bad!

but when she's bad, she's really really good...

 

[ShelbyDawn57]

Do you check those citations? Because they are known to make them up.

The ones I did check were valid, but I wasn't looking for depth at that moment, so...

 

[Bramblethorn]

Before assuming that your highly-rated story is being vindictively down-voted, be aware of this common fallacy:

Regression Fallacy

I suspect that's often the "reason" that highly-rated story scores tend to drop. And having a high rating means that the regression will happen quite fast because more people look at high-rated stories than low-rated ones.

A few years back I posted some simulations of how story scores can drift over time, but it looks like the attachments got stripped at some point, maybe in the forum migration. Just as you say, sometimes what looks like a successful story being bombed is just a had a run of luck in the early votes, regressing towards its long-term mean.

I'm not denying that bombing does occur too, but I think people are often too quick to assume bombing as the explanation for a drop without considering other possibilities.

 

[SkyBubble]

I can't help wondering what you consider a better candidate for that accolade.

For lists and things like that, Wikipedia is pretty good. For any contentious topic, it's awful.

 

[ruddygore]

Ah, Wikipedia, the most accurate website on the entire internet!

It's not a bad place to start, but you have to stay aware that it has volunteer editors that may have an axe to grind on the subject.

 

[MillieDynamite]

Again, it was a joke.

It's not a bad place to start, but you have to stay aware that it has volunteer editors that may have an axe to grind on the subject.

 

[TheWorldBuilder]

All the bits I wrote were accurate. Unfortunately, there's no way to prevent future Wiki-editors from messing around with them.

 

[Kelliezgirl]

A few years back I posted some simulations of how story scores can drift over time, but it looks like the attachments got stripped at some point, maybe in the forum migration. Just as you say, sometimes what looks like a successful story being bombed is just a had a run of luck in the early votes, regressing towards its long-term mean.

I'm not denying that bombing does occur too, but I think people are often too quick to assume bombing as the explanation for a drop without considering other possibilities.

I've had a story start out in a rather disappointing manner, and slowly climbed up to a rather respectable score. I'm not going to complain that I got "5 bombed".

 

[Britva415]

Do you check those citations? Because they are known to make them up.

They don't get away with it for long, though, because people do check them.

I'm not saying nothing ever gets by, but sourced statements is a feature, not a bug.

 

[alohadave]

They don't get away with it for long, though, because people do check them.

I'm not saying nothing ever gets by, but sourced statements is a feature, not a bug.

This was in reference to the AI providing sources, not Wikipedia.

 

[TheRedChamber]

I'm guilty of this myself. For a story to hold an average of 4.5 it needs to get seven 5 votes to every one 1 vote. So, I sit there hitting refresh going "it's going UP, it's going UP, it's going UP, it's going UP, it's going UP, UP, UP, damn it it's back where it started..."

 

[Statius]

Again, it was a joke.

There's no joking on LitE allowed. All statements MUST go through hair- splitting arguments to the nth degree, for days and days, until the next joke or generalized statement is made. It's a rule. I looked it up on Wikipedia so it's true.

 

[TheWorldBuilder]

The contents of Wikipedia do NOT need to be true, it is more important that they are verifiable. Look it up, it's in their policies.

 

[DeMont]

I actually think that people tend to naturally temper their opinions based on other people's opinions - so a person's voting score will be "pulled" towards its existing average -- people prefer to agree with the consensus. And obviously there's the "friends, family and followers" effect for new stories.

@nice90sguy,
The group convener of a writer's group I was in, somewhere around 2004 I believe, summed up such a phenomenon as "the sheeple effect", (Sheeple of course being people who followed the trend) It was their opinion, when it came to entering regional writing contests, that the people judging the contests came to 'know and love' certain authors while looking down on others regardless of the piece of work before them. This contest situation, it would seem, might be seen in the same light.

The only thing that I can suggest as a means of countering these effects is to have an "Established Panel" of judges who could be relied upon to render (more or less) fair and reasonable verdicts free of personal bias. This would alleviate the twenty person vote up (4.92 or whatever) rating which after the fact could be seen to be a much lower vote score than it actually took to win the contest.

The only other thing to do would be to hold the contest open until each entry had received a pre-determined number of views, say fifty. When the limit of views had been reached close the entry to further voting. Naturally, the problem there would be what if a story didn't get fifty views?

A conundrum either way you look at it I think.
Respectfully,
D.

 

[DeMont]

However, while I didn't put my usual , one might assume I was joking, as smart-assed remarks from me are almost always jokes.

@MillieDynamite,
Didn't I tell you that sense of humour was going to get you in trouble one day? Oh, I didn't, I'm sorry, my bad, I was going to, honestly, really...!
Respectfully, always (no joke!)
D.

 

[darwin1859]

Before assuming that your highly-rated story is being vindictively down-voted, be aware of this common fallacy:

Regression Fallacy

I suspect that's often the "reason" that highly-rated story scores tend to drop. And having a high rating means that the regression will happen quite fast because more people look at high-rated stories than low-rated ones.

I missed this discussion when you posted, but I wanted to mention that I thought exactly the same (regression to the mean) but then I did an analysis of one author's self-reported numbers that convinced me that I was wrong. There's some wiggle room in my conclusion, because the standard deviation of ratings is not reported/missing and has to be imputed. And I am sure that regression to the mean plays a role. But I convinced myself that it's more than just regression to the mean. I think there are different populations who find the story (similar to what others have said on this thread).

https://forum.literotica.com/threads/ratings-scheme.1639500/page-2#post-101461097

 

[ColtonWrites]

I've also found that, if you ask, both XAi and ChatGPT will provide citations and footnotes for questions you ask them. In my experience, XAI seems to be more thorough.

Watch out that it will straight-up hallucinate those at times.

I only assume vindictiveness when I get private feedback that makes it obvious the complaint is with the gender identity or sexuality of the characters. Or in one case, I got a 1-star rating on a story that was 65k words long within 10 minutes of it going live on the site. That felt kinda personal, lol.

 

[TarnishedPenny]

Well, speaking for the other side, regression to the mean is all well and good until a story’s been up for months, has several hundred votes and has maintained a steady following a score of 4.75 with little or no change, All of a sudden, the score starts falling like a rock. One or two votes a day and the score keeps falling, say to 4.35 before it levels off again. That’s not regression: that’s enemy action. It exists.

 

[SamanthaBehgs]

Before assuming that your highly-rated story is being vindictively down-voted, be aware of this common fallacy:

Regression Fallacy

I suspect that's often the "reason" that highly-rated story scores tend to drop. And having a high rating means that the regression will happen quite fast because more people look at high-rated stories than low-rated ones.

As someone who reads this and then says "oh look - he's writing about me" despite the fact that I know you're probably not, I can also say I nod my head along to this and say "yea, that's a really good explanation for what otherwise feels like a pattern but may very well not be." And I think it's something that I only partially accounted for in my own estimation of what was going on (I was more looking at my own total number of scores rather than thinking of regression to the mean).


The trouble is this:

I missed this discussion when you posted, but I wanted to mention that I thought exactly the same (regression to the mean) but then I did an analysis of one author's self-reported numbers that convinced me that I was wrong. There's some wiggle room in my conclusion, because the standard deviation of ratings is not reported/missing and has to be imputed. And I am sure that regression to the mean plays a role. But I convinced myself that it's more than just regression to the mean. I think there are different populations who find the story (similar to what others have said on this thread).

https://forum.literotica.com/threads/ratings-scheme.1639500/page-2#post-101461097

And the question of, in a contest, some of the variables are going to be different.

For example, some suggest that a follower base will inflate the score during the early times of the contest. Is this true universally? Are followers that loyal even if the category or theme of the story doesn't match what the author usually writes? I don't know enough about the *typical* follower to be able to predict their behavior (maybe I'm atypical: I won't read a story that isn't my thing just because I follow an author and I certainly wouldn't rate it).

We also know the greatest visibility for a story outside of a contest will be during the first few days (or however long that category keeps them on the front page for being new). Does being on that page result in inflated scores, or just more of them because you'll have more views faster? I assume this would be the same for contest entries since they still fall in their categories "new stories" page.

Now, as a reader - if I'm reading contest entries, I use the contest page, which seems to intentionally *not* show the story score. When folks here were readers (so before you knew more of the ins and outs of the site) - did you search in other ways? Maybe I was just a very uninformed reader. Is this where the "broader audience" comes into play? How do people know that a contest story is highly rated unless it's because they are using those tag filters? And even outside of contests - before you were an author, did you really search using tags or other search features? Seriously, am I *that* bad of a lit reader?

It also makes me wonder - if you have a large enough follower base to feel confident you'll get the required number of votes to qualify, would that mean that your best move is to submit your entry just before the deadline? Give yourself enough time to get great votes but not enough time to regress to the mean. I'm not sure based on a quick peak at the contest winners for Halloween and Summer (only 2 of the 6 seem to have submitted towards the end of their contests).

Other than the score bombing explanations, did I miss any? I was trying to summarize all of the proposed explanations but there were so many folks going back and forth that I may have missed some.

 

[RejectReality]

No doubt that "liked it, didn't love it" 4s contribute to a lot of score loss. A lot of people don't stop to think that only a 5 increases or maintains your score when it's above 4.0. Absolutely anything else causes a dip. But for the truly high-scoring stuff that makes page 1 of the toplist... That's a hotbed of trolling right there.

 

[RejectReality]

It also makes me wonder - if you have a large enough follower base to feel confident you'll get the required number of votes to qualify, would that mean that your best move is to submit your entry just before the deadline? Give yourself enough time to get great votes but not enough time to regress to the mean. I'm not sure based on a quick peak at the contest winners for Halloween and Summer (only 2 of the 6 seem to have submitted towards the end of their contests).

Other than the score bombing explanations, did I miss any? I was trying to summarize all of the proposed explanations but there were so many folks going back and forth that I may have missed some.

I've done analysis of the winners and when they entered a couple of different times. There was actually a tiny bias toward day-1 entries winning more often. More or less within the margin of error. There's no historical advantage to entering late or even last second.

You have to remember that there's a week ( Give or take. Nobody knows for certain when Laurel's cut-off is, but it looks to be right up until the final sweep. ) after the end of the submission period. Most stories only have a honeymoon period of a couple of days when the majority of votes are going to come in. There's plenty of time for readers who think the story is "meh" to cut it down with 3s and 4s.

 

[SamanthaBehgs]

I've done analysis of the winners and when they entered a couple of different times. There was actually a tiny bias toward day-1 entries winning more often. More or less within the margin of error. There's no historical advantage to entering late or even last second.

You have to remember that there's a week ( Give or take. Nobody knows for certain when Laurel's cut-off is, but it looks to be right up until the final sweep. ) after the end of the submission period. Most stories only have a honeymoon period of a couple of days when the majority of votes are going to come in. There's plenty of time for readers who think the story is "meh" to cut it down with 3s and 4s.

Oh Fascinating! And you're right, I didn't take that full week into consideration there and how much time it would give for regression, it seemed a little shorter than I'd expect. But I love the fact that you've checked into that data and what you found as well. Thanks for sharing it!

 

[calmlikeabomb]

I've had a story start out in a rather disappointing manner, and slowly climbed up to a rather respectable score. I'm not going to complain that I got "5 bombed".

Same. My first story was in the low 3s (and deserved it, it was a throwaway piece just to get over the hump of the fear of hitting publish), but of my other two stories, one of them lucked out and started getting traction.

Because of that one story, my other two stories have also slowly started going up. All hail the Halo effect.

 

[calmlikeabomb]

Well, speaking for the other side, regression to the mean is all well and good until a story’s been up for months, has several hundred votes and has maintained a steady following a score of 4.75 with little or no change, All of a sudden, the score starts falling like a rock. One or two votes a day and the score keeps falling, say to 4.35 before it levels off again. That’s not regression: that’s enemy action. It exists.

Out of curiosity, do you think it happens to a specific category more than others? I know LW is a cesspit at times, but would be interesting to see if that trend exists in other categories too.

 

[MillieDynamite]

I had a story in the Halloween competition at another site in 2024. The voting was blind; no one knew who wrote what during the competition. At the end, when the author's pens were put to the stories, First Contact #1: The Strigoi had a score of 7.79 (out of 10) and was tied for 5th place with two other stories. Within a few days, its vote total doubled, then dropped to 5.34, and it hasn't received another vote since then. Here it has a 4.27 score (out of 5). It wasn't in the contest here because it had to be exclusive to SOL during the competition. You can't compare the two scales, as the other site weights all votes toward the middle with some complicated bell curve.

Out of curiosity, do you think it happens to a specific category more than others? I know LW is a cesspit at times, but would be interesting to see if that trend exists in other categories too.