Why Your Story Rating Idea Sucks

[Bramblethorn]

So you just submitted a story, maybe a couple. It's getting good ratings but you know it should be doing better, and you've come up with a great new idea for how Literotica should be rating its stories, instead of the current "vote from one to five stars and average".

Sorry to break it to you, but your great new idea is probably worse than the existing system. Here's some likely reasons why...

1. Your system is basically "most views" in disguise. Since high ratings help stories find new readers, this becomes self-perpetuating: the stories with highest ratings get the most viewers which pushes their ratings even higher, and soon it becomes impossible for a new story to catch up. (Examples: "number of likes", "likes minus dislikes".)

2. Your system is very granular: there are only a few possible scores a story can get. As well as making it impossible to establish a ranking between the top stories - which might not be such a bad thing - this means that any individual vote has either no effect on the score, or a massive effect. (Example: median-based scores.)

3. Your system doesn't acknowledge the fact that even if 95% of readers loved a story, some people might legitimately dislike it, and their dislike should be able to affect the score too. (Example: trimming the top/bottom X% of votes.)

4. Your system significantly reduces the number of legitimate votes coming in. A lot of the story movements that people get het up about here are nothing more than random noise in your scores, and the fewer voters you have, the bigger that noise is going to be. If you scare away a large number of legitimate voters in order to prevent a small percentage of bad-faith voters, you're actually making scores less meaningful. Also, sweeps are probably catching most of those votes anyway. (Example: anything that depends on voters registering.)

5. Your system creates perverse incentives because you didn't stop to think "if I was an asshole who wanted to maximise my score, how would I abuse this?" (Example: scores based on "engagement" that encourage authors to troll readers or to post misleading clickbait blurbs). To be fair, the current system also suffers from this, since reader attrition gives long stories a big advantage, and the hotlist systems create some weird scenarios where a 1-vote can actually improve your standing because a bad vote is better than no vote.

6. Your system is basically "let's make 2 the new 1".

7. Your system is basically "what if it went up to 11?"

8. Your system is computationally expensive and doesn't scale to a site which has half a million stories, probably tens of millions of individual votes, and updates several times a day.

9. Your system doesn't have any way to translate the last 20 years of rankings data for all the stories already on this site, so we need to throw out all those votes and start over. (Actually, this one might not be such a bad thing, but it's a significant change and not to be embarked upon lightly...)

10. Your system is too complicated. Most voters and authors don't understand it and will get angry about it, even if it's otherwise perfect. If this is you, don't be sad, you have a promising future in cricket.

Probably some others that I missed, but 10 seems like a nice round number to start with.

 

[KeithD]

A few years ago the Web site added the actual ratings to the author lists the readers see. I think it would have been best if they'd just dropped the Red H at the same time and let the readers subsequently individually decide whatever cutoff they wanted for what either is hot or worthy of reading.

 

[Jamie_and_Lisa]

So you just submitted a story, maybe a couple. It's getting good ratings but you know it should be doing better, and you've come up with a great new idea for how Literotica should be rating its stories, instead of the current "vote from one to five stars and average"...

7. Your system is basically "what if it went up to 11?" ...

Nigel: The numbers all go to eleven. Look, right across the board, eleven, eleven, eleven and...
Martin: Oh, I see. And most amps go up to ten?
Nigel: Exactly.
Martin: Does that mean it's louder? Is it any louder?
Nigel: Well, it's one louder, isn't it? It's not ten. You see, most blokes, you know, will be playing at ten. You're on ten here, all the way up, all the way up, all the way up, you're on ten on your guitar. Where can you go from there? Where?
Martin: I don't know.
Nigel: Nowhere. Exactly. What we do is, if we need that extra push over the cliff, you know what we do?
Martin: Put it up to eleven.
Nigel: Eleven. Exactly. One louder.

-This is Spinal Tap (1984)

In all seriousness we should use a point system (1 point per character, 1 point per obscure reference, 1 point per reference to an aircraft built before 1965)...

 

[SimonDoom]

Bravo, B. That was very well done and I agree with all of it. Some of it I've thought before but some of it is new to me.

I would add another consideration, which in some cases overlaps with yours:

11. Your system is based upon trying to make authors happy without taking into consideration the impact on readers. Readers far outnumber authors. It is reader traffic that makes this site valuable. There is an excellent chance that any restriction or limit you place upon, or change you make to, the current voting system is going to make voting on stories incrementally more difficult and therefore less appealing and therefore will have a negative impact on the desire of readers to visit this site. No voting system change is good if it does this.


I do have a suggestion for a change to the voting system, which I think would be an improvement without making things too much more difficult, and I think it would result in more voting, not less, and more accurate voting.

Readers should be given the option of four more possible votes: 1.5, 2.5, 3.5, and 4.5. The Red H would remain at 4.5. The advantage of this is that readers can give the story a 4.5, therefore not sabotaging the chance at a 4.5, but their vote doesn't signal that the story is perfect. A conscientous reader wouldn't have to go through the process of thinking, What do I do? I don't like this story well enough to give it a 5 but I don't want to deny it a 4.5? I think voting would increase.

The system could be set up so it looks almost identical to the system we have now, only the five star options would change a bit so there would be nine stars. But as a practical matter it would not be more difficult and I don't think it would scare anyone away.

I suspect scores themselves would not change much -- people would still see votes in the same way. But I think there would be more votes: more 4.5s and fewer 5, but also 4.5s were previously there were no votes. I doubt average votes would change much.

It might slightly disincentivize the giving of 1 bombs.

The addition of new 4.5 votes, if I'm right, could partly offset existing 1 bombs.

There would be no need to change the current 5 point scale, or the red H system.

Under the current system, depending on the category, a red H isn't all that special. A red H means your story is for most categories in the top 25 to 35 %. Now giving a 5 could be a signal that a story is truly in the upper crust -- the top 10% or better. A red H doesn't mean that, now.

I don't think the system suffers from any of the 10 flaws Bramblethorn identified, or the flaw I did.

 

[PennameWombat]

<snip>

10. Your system is too complicated. Most voters and authors don't understand it and will get angry about it, even if it's otherwise perfect. If this is you, don't be said, you have a promising future in cricket.

Probably some others that I missed, but 10 seems like a nice round number to start with.

I take it you meant "sad" in number 10, not "said." But overall, well bowled, old chap.

Especially number 10.

As to Messrs Duckworth and Lewis... Back in 2007 my wife and I lived in the US. Her parents visited us and it happened to overlap with the semis and grand final of that year's Cricket World Cup. The only way to get it was I signed up for a streaming deal so my father-in-law could watch Australia crush lesser competitors. After all, the Aussies invented the sport, right? No one uses sandpaper better, right?

Back to 2007. The final. My late father-in-law was an educated man, a retired Lieutenant Colonel in the Oz Army, a professional surveyor... I (being of inferior native Yankee stock) asked "what is this Duckworth-Lewis they're going on about?"

Needless to say we simply raised many glasses of beer in celebration until I was too dazed to keep asking...

 

[Isabel_Innuende]

Me, when my story rates above 4.5: Hooray! The system works!

Me, when my story rates below 4.5: This website is infested with trolls! Did they even read my story? This is an outrage! Every user should have to take an intelligence test before they rate my story, unless of course they want to rate my story a '5', in which case they have proven their intelligence already.

 

[PennameWombat]

Bravo, B. That was very well done and I agree with all of it. Some of it I've thought before but some of it is new to me.

I would add another consideration, which in some cases overlaps with yours:

11. Your system is based upon trying to make authors happy without taking into consideration the impact on readers. Readers far outnumber authors. It is reader traffic that makes this site valuable. There is an excellent chance that any restriction or limit you place upon, or change you make to, the current voting system is going to make voting on stories incrementally more difficult and therefore less appealing and therefore will have a negative impact on the desire of readers to visit this site. No voting system change is good if it does this.

Upvote this. I try not to write for other authors (although I definitely appreciate when I see one compliment me or 'favorite' a story of mine). Readers need us but without them we're just jabbering to each other.

I do have a suggestion for a change to the voting system, which I think would be an improvement without making things too much more difficult, and I think it would result in more voting, not less, and more accurate voting.

Readers should be given the option of four more possible votes: 1.5, 2.5, 3.5, and 4.5. The Red H would remain at 4.5. The advantage of this is that readers can give the story a 4.5, therefore not sabotaging the chance at a 4.5, but their vote doesn't signal that the story is perfect. A conscientous reader wouldn't have to go through the process of thinking, What do I do? I don't like this story well enough to give it a 5 but I don't want to deny it a 4.5? I think voting would increase.
<snip>

I like this.

 

[gunhilltrain]

So you just submitted a story, maybe a couple. It's getting good ratings but you know it should be doing better, and you've come up with a great new idea for how Literotica should be rating its stories, instead of the current "vote from one to five stars and average".

Sorry to break it to you, but your great new idea is probably worse than the existing system. Here's some likely reasons why...

Maybe I missed something, but who proposed this again? It seems awfully complicated, and the rebuttal is pretty complicated too. Setting interest rates at the Federal Reserve deserves complexity (although I wonder if they don't just wing it a lot of the time).

There is a site with a one-to-ten system, but there is some procedure for dropping low and high scores that I haven't bothered to comprehend. Almost everybody winds up with a 6 or 7 something, unless they opt-out of allowing voting at all - that seems to be at least a quarter of them. I don't really care; I put stuff there that won't fit here, or just for the sake of variety.

 

[lovecraft68]

Great post

Problem is we are reaching the point where no author seems to think there is such a thing as someone who can dislike their story and give it a 1 or 2.

This was perfectly evidenced in the one bomb thread where a poster, tired of their story being 'bombed' down to a horrible 4.66 asked Laurel for a sweep....and she took the time to do it and its now a 4.81.

if you're at a 4.66 why is that a problem? That's a very good score, but now no, no, there's no way I shouldn't be higher, and the site-who has better things to do and should look before sweeping and say, 'what are you complaining about' is catering to the recent endless whining about trolls.

The Karens are here.

The new system will be everyone asking for sweeps no matter what the score and that will make the new score inflated and just as fraudulent as a story that may have been bombed, but as long as the flawed system boosts the author, that's fine.

Doesn't matter if it renders the system useless, as long as its being useless fluffs their ego.

 

[Bramblethorn]

11. Your system is based upon trying to make authors happy without taking into consideration the impact on readers. Readers far outnumber authors. It is reader traffic that makes this site valuable. There is an excellent chance that any restriction or limit you place upon, or change you make to, the current voting system is going to make voting on stories incrementally more difficult and therefore less appealing and therefore will have a negative impact on the desire of readers to visit this site. No voting system change is good if it does this.

Very good point. There's a lot of overlap with #3 here, but still worth making this one explicit. Scores may be balm to our egos but that's not what they're for.

I do have a suggestion for a change to the voting system, which I think would be an improvement without making things too much more difficult, and I think it would result in more voting, not less, and more accurate voting.

I could live with this one. I suspect we'd see some change to the averages, because people are weird and don't respond to multi-point scales in a mathematically consistent manner, but it's relatively minor. I can't speak for others but the 4.5 option would encourage me to vote a lot more often than I currently do, and increasing legitimate votes helps mitigate a lot of other issues.

I take it you meant "sad" in number 10, not "said." But overall, well bowled, old chap.

Whoops, fixed!

Back to 2007. The final. My late father-in-law was an educated man, a retired Lieutenant Colonel in the Oz Army, a professional surveyor... I (being of inferior native Yankee stock) asked "what is this Duckworth-Lewis they're going on about?"

Many years back I saw a presentation from somebody who liked the general idea of Duckworth-Lewis but felt that it was a bit too simple and needed expanding...

 

[Bramblethorn]

Maybe I missed something, but who proposed this again?

It was mentioned in a recent discussion, but I don't want to name names because this isn't about that one poster. This is a perennial topic of discussion here - every few months somebody has a grand idea for improving on the square wheel by replacing it with a triangular wheel with 25% fewer bumps.

 

[Rustyoznail]

It's not just here that scoring systems are continually discussed.

In my engineering work, there's people who like a condition rating of structures of 0-5, 1-5, 1-10, and 1-100. A couple want 1-100 and two decimal places...

The point is, you're never going to have a system that pleases everyone. I think the addition of text under the stars helps give perspective, but does Anon take any notice? Doubt it.

 

[SimonDoom]

The new system will be everyone asking for sweeps no matter what the score and that will make the new score inflated and just as fraudulent as a story that may have been bombed, but as long as the flawed system boosts the author, that's fine.

Doesn't matter if it renders the system useless, as long as its being useless fluffs their ego.

Right. Every system can be gamed. If "asking for a sweep" (I cannot recall ever having done this, but I won't swear that I didn't) is the new norm, then it just means the squeaky wheel gets the grease. Then it's phonier than ever.

 

[RejectReality]

A few years ago the Web site added the actual ratings to the author lists the readers see. I think it would have been best if they'd just dropped the Red H at the same time and let the readers subsequently individually decide whatever cutoff they wanted for what either is hot or worthy of reading.

This is the one of the very few things that has ever been suggested that has no real downside.

Eliminating the H mitigates the angst of any vote less than 5 having an impact you. Readers will quickly adjust, and start paying more attention to the other information presented, instead of the bright red bling — exposing them to a lot of great stories and authors they might have otherwise ignored.

It's an easy target for bad actors. Removing that bar will most likely force them to resort to more aggressive attacks to get the desired effect. That will leave more traces to be detected, and thus a greater chance all their douchebaggery will be undone the next time the Hoover comes around.

It should require little work on the back end to implement, which is always a consideration.

The one and only place it might cause some folks distress is those who have large fanbases who get in early, causing a story to display an H early, and therefore attract a large amount of attention when it's one of a few lonely ones on the new story lists. They're going to lose that perk and have to depend on their title/description game a little more to stand out. Most probably already have good game in that area to have attained the large fanbase in the first place.

 

[davet_1_1]

The Karens are here.

The mental image of a 1-bomb being the lit equivalent of a reader screaming "I WANT TO SPEAK TO YOUR MANAGER!" at Laurel is very amusing.

 

[Zeb_Carter]

Hey, while we're at it, lets just change it to a like or dislike. 1 and 0 for you computerheads out there. So your story either has a thumbs up or a thumbs down. They are additive no average the higher number is what displays. Simple and easy to understand.

For contests, the average would determine the winners.

Now can anyone see the flaw here?

Think about it.

 

[Jamie_and_Lisa]

... every few months somebody has a grand idea for improving on the square wheel by replacing it with a triangular wheel with 25% fewer bumps.

This is almost a perfect definition of an engineer.

 

[RejectReality]

Someone elsewhere suggested the like with no dislike thing, and when asked what to do about existing scores, said to convert each vote into a like.

Still a bad idea, but it does have one redeeming feature. At that moment of transition, every malicious low vote ever cast on a story is instantaneously transformed into approval. You can almost hear the anguished screams as the poo-flingers fall to their knees upon feeling the disturbance in the force. LOL

After that brief, glorious moment, it turns into a system that makes it almost impossible for new people to gain any traction.

 

[HollyLoriAnnalee]

My Only Ratings Idea

I don't know if this has been put out there before, but here is my simple idea:

To weed out trolls, you do this.

If you read a story (or fucking 3 words of one like some of the 'reviews' I've gotten), and you desperately want to give it 1 single fucking star, you HAVE to:

1) Have an account here, and
2) Submit a short explanation to the author why you felt it was the most abysmal thing that you have to give a 1 to.

and possibly, but not definitely

3) That explanation cannot be simply 'it's too long.' I'm sick of that shit. That's not criticism, that's not rating something, that's being a piece of shit. I got an anonymous feedback that said "10 pages. 90-100 pages in Word. Rating of 1." Fuck you, troll, and your shit shouldn't count. If that's their explanation, we have some way of deleting that rating.

I'm pissed, but I think those first two could actually do something. I don't know, it's a little harder, just a little, to get an account and tell someone you just think their piece was worthless for these reasons.

2s through 5s are available for anonymous readers to rate.

What do you guys think? Tell me if it's stupid, but please for the love of all fucking things tell me WHY you think it's stupid.

Holly

 

[SimonDoom]

I don't know if this has been put out there before, but here is my simple idea:

To weed out trolls, you do this.

If you read a story (or fucking 3 words of one like some of the 'reviews' I've gotten), and you desperately want to give it 1 single fucking star, you HAVE to:

1) Have an account here, and
2) Submit a short explanation to the author why you felt it was the most abysmal thing that you have to give a 1 to.

and possibly, but not definitely

3) That explanation cannot be simply 'it's too long.' I'm sick of that shit. That's not criticism, that's not rating something, that's being a piece of shit. I got an anonymous feedback that said "10 pages. 90-100 pages in Word. Rating of 1." Fuck you, troll, and your shit shouldn't count. If that's their explanation, we have some way of deleting that rating.

I'm pissed, but I think those first two could actually do something. I don't know, it's a little harder, just a little, to get an account and tell someone you just think their piece was worthless for these reasons.

2s through 5s are available for anonymous readers to rate.

What do you guys think? Tell me if it's stupid, but please for the love of all fucking things tell me WHY you think it's stupid.

Holly

I'd say it violates Bramblethorn's 3, 4, 5, 6, and 10, and my 11.

 

[HollyLoriAnnalee]

Gotcha

I'd say it violates Bramblethorn's 3, 4, 5, 6, and 10, and my 11.

I get that, yeah. I mean, any idea at all is going to violate one or many of those reasons, and they're absolutely great reasons. But the troll thing, the people who hardly read something and then click 1--I think I'm just reacting viscerally because some of them--all anonymous--want you to know they did it, and they want you to feel their hatred. It sucks.

I think to deal with that, yeah, you would have to break some rules, but I think mine would not affect as much of the whole ratings system as some bigger ideas. I think you would have just a few less 1s, but yeah, so maybe it's not worth it. I think it would be but I see your point.

 

[HollyLoriAnnalee]

And people are perfectly within their rights to rate a story whatever they think, and I am perfectly fine with people giving me a 1, really! You know my outlook--I'm not really worried about votes, I just write what I write, and I know somebody out there will find it and love it. It's just that problem I already wrote about. Yeah. Just wanted to throw that out there too.

 

[nikisixx]

xxxxx

 

[SimonDoom]

And people are perfectly within their rights to rate a story whatever they think, and I am perfectly fine with people giving me a 1, really! You know my outlook--I'm not really worried about votes, I just write what I write, and I know somebody out there will find it and love it. It's just that problem I already wrote about. Yeah. Just wanted to throw that out there too.

More people should have that attitude.

There's nothing inherently wrong or suspect about a 1-vote. Nobody's voting criteria are objectively better than anyone else's. The Site makes no effort to regulate or define what a particular score "means."

If somebody really, really hates your story, even though it's got great grammar and smartly-drawn characters and is based on a cool idea, they're entitled to give it a 1.

You are going to get a lot of flak for your stories -- more than most. It's not because of any lack of skill on your part; it's because you don't stick to category conventions and you include lots of way-out-there content that is going to turn some people off. I'm about 2/3 of the way through your Luau story, and oh my god. It takes an open-minded reader fully to appreciate all the stuff in one of your stories. That's just the way it is. There's really no way fairly to adjust the scoring method to guard against that, nor is there any really objectively sound reason to do so.

 

[nikisixx]

xxxxx