Score erosion over time

[Fervid]

Over many months, the scores of my stories generally trend downwards. It's only by .02 to .04, but that's enough to reduce readership and take a story much, much lower on the top lists. It’s not obvious to me what drives it. Has anyone else noticed this? Theories?

 

[StrappingMan]

It's the law of averages. I have been part of several sites that have had a rating system over the years and have seen the same thing.

The short answer is if you have a highly rated story all it takes is one four star to lower the average. It takes a lot of five stars to offset that. And don't get me started if someone one stars you.

Anything rated over 4.5 will tend to drop unless a lot of readers keep giving it a top rating.

 

[oggbashan]

My new ones get bashed (not eroded) every time they reach H.

But my older stories? Yes, the scores are generally eroded but sometimes they surprise me by going up considerably.

 

[SimonDoom]

What StrappingMan said. It's reversion to the mean. My stories with low scores tend to rise a little bit over time while my higher-scored stories sometimes fall. Not a lot, but a little.

I think what also happens is that if your story hasn't been swept in a while it may accumulate some bad scores. I sometimes see little unexpected jumps in older stories and I think that's what is making them happen.

 

[NotWise]

I think there are likely to be different reasons for different stories.

For contest stories, the highest score they ever have is often their score when the contest results are announced. The score droops after that. I suspect that the contest-ending sweeps sometimes artificially elevate the ratings.

For contest stories and other stories, the slide may be because your audience isn't homogeneous. You have one group who reads the story when it goes live, and in some categories that group is very giving, and then the group that finds it later through searches, or links on sidebars, etc, is a different group with different standards.

Heterogeneity can also contribute to low scores rising over time -- especially when rated by readers who have become your followers.

In highly rated stories, another cause for the slide may be that some readers gave you multiple five-star ratings or five-star ratings that the site qualifies as "illegitimate" for other reasons. Lit's repeated sweeps then can drop your score by removing those votes.

I imagine there are other reasons, too.

edit: Oh yeah. You may be maliciously attacked, too. The site's sweeps can remove those votes, but they may not correct the rating back to where it was.

 

[MelissaBaby]

Early scores also are propped up by a large percentage of them being submitted by followers who already like your work. It stands to reason that they will give you a higher average score than new readers will.

 

[BobbyBrandt]

I don't follow the scores on my stories daily, but overall, I haven't seen much change in them.

In the last seven years, I have submitted ten stories in sixty-one posts in four different categories. (More on multi-part story scoring later)

All but three of those submissions have consistently remained above the 4.5 score to garner them an 'H', and the three that scored between 4.0 and 4.49 have all remained in that same range. My first story, posted in 2014 remains between 4.85 and 4.9.

Now, what I have seen by breaking a story into multiple parts is that scores start out lower and then grow incrementally as readers become more invested in the story. As an example, part one of my story "Searching" scored a 4.66. The last two parts scored 4.85. The average score for the story remains around 4.79.

If I was hung up on only obtaining the highest scores rather than catering to the requests of my readers, there is no way that I would ever submit anything in multiple parts again. Readers would be voting on the complete work rather than fragments of it, and, at least in my case, the score would hopefully be higher.

 

[KeithD]

I think the ratings on my deflate over time, yes. I don't keep track of this, though, so I can't tell for sure. I think the suggestion that one's established fans are going to be in on the early voting makes sense.

 

[KeithD]

What confuses me is the drop in the number of ratings. My story had 80 ratings, of whatever number. Then I log in again and find the number of ratings has dropped to 77. Are people somehow able to remove their ratings, whatever it was? Or are the site owners/moderators removing them for some reason? Any ideas?

No one can remove votes except for the site management, which periodically sweeps away what its formula considers are bogus votes. And, no, they won't reveal how the formula is achieved.

 

[SimonDoom]

What confuses me is the drop in the number of ratings. My story had 80 ratings, of whatever number. Then I log in again and find the number of ratings has dropped to 77. Are people somehow able to remove their ratings, whatever it was? Or are the site owners/moderators removing them for some reason? Any ideas?

It's called a sweep. The Site owners periodically "sweep" stories to delete votes that appear to be illegitimate. Nobody knows exactly how they determine to do this and we are discouraged from speculating so people cannot game the system.

Sweeps happen fairly often, and in particular during contests, right before the contest ends. But they happen other times, too. A very significant sweep happened last summer for about three straight weeks, deleting many votes for older stories. Several of my stories saw significant boosts in scores. One three-year-old story that had never been above 4.5 finally got there.

 

[ElectricBlue]

Anything rated over 4.5 will tend to drop unless a lot of readers keep giving it a top rating.

The top half of my story file, scores-wise, slowly moves up and down relative to itself, but overall, creeps higher. The bottom half generally, but very slowly, creeps higher or stays exactly the same. Which means readers of my back catalogue are appreciative, overall.

 

[BarracudaSwordfish]

The whole sweep thing baffles the hell out of me, as it is probably intended to do.

One story has not had a rating in the last three years but suddenly lost three last week,. There must have been countless sweeps in the intervening period, so I am completely at a loss to explain it.

Is there a possibility that scores may be erased if someone closes their account?

 

[KeithD]

Is there a possibility that scores may be erased if someone closes their account?

It's possible, yes, but again, if we start pinning down how questionable votes are being swept, the trolls who lodge questionable votes will devise workarounds to continue doing what the Web site is trying to negate in support of the authors.

Not long ago, I experienced a big boost in the scores across my account. I suppose it's conceivable that it was because a troll or two had their accounts deleted. A few harassing posters on the discussion board did have their accounts deleted around that time and I have no doubt that they were the type of poster who would down vote my stories because of a discussion board disagreement.

I just don't know. The bottom line is that the ratings almost always go up after a Web site sweep. Why wouldn't the authors be grateful for that?

 

[Fervid]

I think Melissa has a good point about followers. I posted four stories the day I opened my account here and their scores have wandered around randomly. Now I have enough followers to affect a score, and the downward trend is more noticeable.

I originally thought reversion-to-mean made sense, but what mean? If among all potential readers, 67% would score a story 5 and the rest would give it a 4, you would expect the score to oscillate around 4.67, moving both up and down until it had hundreds of votes.

 

[RejectReality]

I think Melissa has a good point about followers. I posted four stories the day I opened my account here and their scores have wandered around randomly. Now I have enough followers to affect a score, and the downward trend is more noticeable.

I originally thought reversion-to-mean made sense, but what mean? If among all potential readers, 67% would score a story 5 and the rest would give it a 4, you would expect the score to oscillate around 4.67, moving both up and down until it had hundreds of votes.

If you're making page 1 of the category toplist, there's another factor: Jealous fans of other writers.

( and yes, probably some jealous writers, though I still contend their cheerleaders far outnumber them, and are less likely to be swept than one person casting a lot of low votes, no matter how savvy. )

You have some scores that look as if they might be in that range. It's almost inevitable that you're going to get bombed down when you score a high place on the toplists. The toplists are also a fairly significant selection criteria, so you're going to attract more "liked it but didn't love it" 4 votes than you might have otherwise.

You get knocked down. Sweeps eventually come along and pop you back up. You get knocked down again. Swept again. Lather, rinse, repeat.

As time goes on, your vote total gets high enough that you stay on the toplist longer, which attracts even more votes, and eventually, the erosion slows to a crawl. Until then, it's a roller-coaster ride for anything that hits the toplist.

 

[oggbashan]

Some of my followers follow me so they know when to post 1s.

 

[NotWise]

I think Melissa has a good point about followers. I posted four stories the day I opened my account here and their scores have wandered around randomly. Now I have enough followers to affect a score, and the downward trend is more noticeable.

I originally thought reversion-to-mean made sense, but what mean? If among all potential readers, 67% would score a story 5 and the rest would give it a 4, you would expect the score to oscillate around 4.67, moving both up and down until it had hundreds of votes.

My followers aren't necessarily early voters. Some get to it when they get to it.

The mean is unknown, and the more votes your story gets, the closer the average approximates the mean.

 

[Bramblethorn]

I usually see a gradual increase over time, though part of that is probably the effect of new postings - if somebody sees one of my new stories and loves it there's a good chance they'll go looking through my other work and upvote those stories as well.