Yet another take on ratings

[HandsInTheDark]

Generally I've decided that ratings don't mean much, and I've said so. But I recently posted a chapter in Non-Erotic, and watched the voting. As befits a later chapter, only readers into the concept are still around, so ratings were generally wonderful: 5,5,5,5,5,5,5,5,5,5,1,4,5,5.

Spot the outlier.

There's nothing unusual about this pattern. I've got a hater who shows up every few days to see if I've posted anything. It's a guaranteed one bomb. I've got a theory as to who he is and why, but that doesn't matter because there's squat I can do about it. Most of you probably know the feeling; it's just how Literotica works.

Normally it doesn't bug me. This time, though, I'm writing in a category where votes are uncommon, so it's going to take awhile for that 1 to get averaged away. It's dragged down the score to 4.64, low enough that people are likely passing it by. Which is the point of an early one bomb, of course. Sooner or later, Manu will get around to sweeping it, but by then the story is off the new list, so damage done.

So I'm back to thinking about ways to deal with this crap.

What I'd like is for Literotica to switch to computing a geometric mean instead of an average. It's not as cheap to compute, and requires a little special programming for large sample sizes because it gets into bigint territory fast. But it does what I think Lit should have been doing all along, showing what most people think about a story, and reducing the impact of outliers. That set of 14 votes yields a 4.84 as a geometric mean - which I think is much closer to what people are generally saying about it.

I think there are a lot of advantages. It would free Manu from bothering with sweeps, except perhaps during contests when ranking is extremely close. Onebomers would eventually figure out that they just don't have much impact, and maybe some would give it up. It also diminishes the effect of your best friend's undeserved 5. Contests would feel less rigged, though a large enough voting bloc, if those exist, will still prevail.

I'll send this on to Manu because he's assured me that Changes Are Coming This Year. I'll post it here so people can kick around the idea.

 

[HandsInTheDark]

"Manu has exceeded their stored private messages quota and can not accept further messages until they clear some space."

I just got the image of Manu taking a few days off around new years, and his inbox being flooded out with sweep requests.

I'm easily amused by self-referential humor, yeah, but ha.

 

[lovecraft68]

The problem I see off the bat is how does one know the one is not legit? I know many aren't, but as much as we hate to admit it, some are.

In your case its one bad vote and it could very well be a troll. But on the chance its not, sometimes I get bugged that few can seem to accept a one as someone really not liking the piece.

It would play havoc with contests as well. As it is the sweeps are out of control. I recall placing in contests in third with a 4.82. Lately 4.87 is left outside as winners are in the 4.9's. I'm sorry I refuse to believe in legit 4.95 scores so if anything I am becoming of the mindset that too many bombs or 2's and 3's are removed as it is.

But in the end, this is kind of a moot discussion. I doubt the scoring and how its monitored is going to change.

 

[Sara_Hawkes]

How?

Quick question from a newbie. How do you see individual votes? All I have access to is the average.

 

[lovecraft68]

Quick question from a newbie. How do you see individual votes? All I have access to is the average.

The only way you can see them is pretty much on a story hardly no one is voting on-as the OP said his was, it was down to the low core audience- and being online long enough to keep checking it.

With smaller numbers you can figure it out as well. If you have a 20 after four votes, then a 21 on five you know number five was a one bomb, etc...

But if you're writing in a category that's going to get hundreds of votes in a day its pretty tough.

My best advice to newbies and longtime authors as well, is don't obsess about the votes and scores focus on your writing. We all want the high scores and little red H;s but the writing our story our way should be more important.

 

[Sara_Hawkes]

My best advice to newbies and longtime authors as well, is don't obsess about the votes and scores focus on your writing. We all want the high scores and little red H;s but the writing our story our way should be more important.

Thanks, and I shall follow your advice.

 

[HandsInTheDark]

The problem I see off the bat is how does one know the one is not legit? I know many aren't, but as much as we hate to admit it, some are.

In your case its one bad vote and it could very well be a troll. But on the chance its not, sometimes I get bugged that few can seem to accept a one as someone really not liking the piece.

It would play havoc with contests as well. As it is the sweeps are out of control. I recall placing in contests in third with a 4.82. Lately 4.87 is left outside as winners are in the 4.9's. I'm sorry I refuse to believe in legit 4.95 scores so if anything I am becoming of the mindset that too many bombs or 2's and 3's are removed as it is.

But in the end, this is kind of a moot discussion. I doubt the scoring and how its monitored is going to change.

That's the beauty of a geometric mean. It doesn't discard any vote, but a vote that's a clear outlier has less impact. The geometric mean of a 1 and a 5 is 3 - what you'd expect, because there's no clear trend. But a single 1 in a sea of fives almost vanishes from the score.

If a story sucks, it should get a lot of low scores - and that's what the geo mean will report. If a story gets a lot of mixed scores with no clear trend, a geometric mean will more or less match an arithmetic one.

I agree that if you get 30 people who are in love with a fetish stroker and pile on the fives, and you get one reader who recognises the premise is flawed, the characters are cardboard, the dialog makes little sense and the plot contains overt contradictions, and gives you a 1, the one vote is going to get lost. But I think that situation is probably rare, and the site has a much larger problems with Haters than it does with unusually savage literary critics waving Strunk and White. (Which is a pity because roaming packs of savage literary critics is a great mental image.)

Contests here are not very meaningful. No offense to the winners, y'all worked hard. But the very existence of people able to vote on competitor's stories means they can never be fair - you can rip someone out of the running by giving their story a 4 these days. I don't have a fix for that. A geometric mean might help, but no algorithm is going to succeed in the face of clever, aggressive rigging. Maybe for contests they can stick to arithmetic means - it's broken, but anyone who cares knows it's broken and it's broken in a way they can understand.

(I don't care because I don't enter contests; someone who cares can probably come up with a better approach.)

 

[HandsInTheDark]

Ok, scratch this thread. Geometric mean isn't a solution. I got my crap mixed up.

 

[oggbashan]

...

My best advice to newbies and longtime authors as well, is don't obsess about the votes and scores focus on your writing. We all want the high scores and little red H;s but the writing our story our way should be more important.

If I cared about high scores and little red Hs I would drastically change my writing.

A quick glance at my list of stories shows that most of my stories are rated below 4.50.

I write a story because I want to. I appreciate the response some of them get, but the rating is unimportant. What really matters to me is that I have finished and posted that particular story, and can move on.

The only rating that is important to me, is my own. Some of my stories are much worse than others, and I know they are.

 

[Nathan_Brazil]

Ok, scratch this thread. Geometric mean isn't a solution. I got my crap mixed up.

LOL! Can you teach a few of the others to do this when their argument gets tanked. Start in the Political forum.

 

[NoJo]

The only rating that is important to me, is my own. Some of my stories are much worse than others, and I know they are.

That's British for "Some of my stories are brilliant, and I know they are"

 

[oggbashan]

That's British for "Some of my stories are brilliant, and I know they are"

Nope. None of my stories are as good as my Muses expect me to write.

And some are so bad that even I don't read them.

 

[jsmiam]

Quick question from a newbie. How do you see individual votes? All I have access to is the average.

If you happen to be online the morning a story goes online, and you stay on your statistics page, obsessively hitting refresh every 3 seconds, you can almost watch the scores unfold in real time. By having what the score was 3 seconds ago and how many votes there were, you can deduce what the ratings submitted in the 3 seconds since were. (Not that any of us ever did that! )

Regarding better ways to score: Amazon probably has a department of 25 people (maybe more) with PhDs in Statistics, and they can't eliminate review fraud there, because it's impossible. To expect little old Lit to solve something that web-giant Amazon can't isn't realistic.

I post somewhere else with a more basic Like/Dislike button, and that's even worse. The Dislike is a Zero-bomb, not a One-Bomb, so a bad vote pulls ratings down even more than a 1-bomb at Lit.

Old thread about the impact of a 1-bomb: http://forum.literotica.com/showthread.php?t=1226676

 

[NOIRTRASH]

I note the AH posters who come and go when the stories go live. Theyre in and out for a few minutes (switching accounts) to vote and vote often.

 

[NaokoSmith]

You can find out individual votes on your stories if you are posting in a low voting category, like Novels and Novellas or Text with Audio. You get so few votes that you can calculate exactly what you got, just by looking every day or two.

If you think your story has been 1-bombed, you can ask for a sweep of that story. You use the Report function in the bottom corner of the story. Pick 'Other' and say 'I think there has been problematic voting on this story'. Once or twice when I was being majorly 1-bombed I did this, mainly in order to make sure that my story would keep its red H at that critical time when it's still in the New story lists and so more people would be likely to read it. My score dramatically improved whenever I did it, although you do run the risk that you may lose some over-enthusiastic 5s as well as 1s.

I have no reason to believe that the 1-bombs were personally directed at me. In one case, friends who read the story said it was too good so being targetted to get it lower than other stories (not in a competition! just in the New story lists ). In the other case, everyone's story in the category regularly gets 1-bombed for no very good reason I can see. I think some people have computer algorithms running that just 1-bomb every story on the site, in the same way that some people have a thing going where they send emails to everyone whose address they can get claiming to be a beautiful Russian with a poorly grandmother who urgently needs an operation.

 

[Sara_Hawkes]

If you happen to be online the morning a story goes online, and you stay on your statistics page, obsessively hitting refresh every 3 seconds, you can almost watch the scores unfold in real time. By having what the score was 3 seconds ago and how many votes there were, you can deduce what the ratings submitted in the 3 seconds since were. (Not that any of us ever did that! )

You can find out individual votes on your stories if you are posting in a low voting category, like Novels and Novellas or Text with Audio. You get so few votes that you can calculate exactly what you got, just by looking every day or two.

If you think your story has been 1-bombed, you can ask for a sweep of that story. You use the Report function in the bottom corner of the story. Pick 'Other' and say 'I think there has been problematic voting on this story'. Once or twice when I was being majorly 1-bombed I did this, mainly in order to make sure that my story would keep its red H at that critical time when it's still in the New story lists and so more people would be likely to read it. My score dramatically improved whenever I did it, although you do run the risk that you may lose some over-enthusiastic 5s as well as 1s.

Thank you. I understand quite a bit more now.

From my own perspective I'll be deliriously happy if my next one gets a higher rating than Sara's Wishlist.

 

[NaokoSmith]

Thank you. I understand quite a bit more now.

From my own perspective I'll be deliriously happy if my next one gets a higher rating than Sara's Wishlist.

I will be happy if I manage to get the next one written and posted .

Now, where did I put those six stories I'm editing, and the FAWC6 stories to review, and the other story I was supposed to review ... oh, bother, it's time to go to sleep now.

http://fontfolly.files.wordpress.com/2013/12/tumblr_m8ueyrizxp1rnvzfwo1_500.jpg

 

[ishtat]

I think that one could argue that a group of 12 5's out of a total of 14 is as much an aberation as the single group of 1.

Statistically it might be more valid to take the mean score for the whole of the particular category then calculate the degree of variation from that mean. It would at least make different categories more reasonably comparable (as a whole) but the individual story's score is still at the mercy of a low incidence of voting.

Alternatively. I recommend that all authors submit a poem or three. The number of reads and votes is so tiny that you'll never complain again. The main difficulty with getting a red H on a poem is getting 10 votes of any score.

 

[oggbashan]

...

Alternatively. I recommend that all authors submit a poem or three. The number of reads and votes is so tiny that you'll never complain again. The main difficulty with getting a red H on a poem is getting 10 votes of any score.

One of my poems has stayed at rating x.xx = NO votes at all for some time.

 

[Sara_Hawkes]

One of my poems has stayed at rating x.xx = NO votes at all for some time.

I SALUTE your honesty. That was too tempting. I just voted.

 

[oggbashan]

I SALUTE your honesty. That was too tempting. I just voted.

Thank you. Your salute is acknowledged.

Unfortunately your vote won't show until that page is updated.

 

[lovecraft68]

I think that one could argue that a group of 12 5's out of a total of 14 is as much an aberation as the single group of 1.

Statistically it might be more valid to take the mean score for the whole of the particular category then calculate the degree of variation from that mean. It would at least make different categories more reasonably comparable (as a whole) but the individual story's score is still at the mercy of a low incidence of voting.

Alternatively. I recommend that all authors submit a poem or three. The number of reads and votes is so tiny that you'll never complain again. The main difficulty with getting a red H on a poem is getting 10 votes of any score.

But nobody wants to hear that

As for poems I have one and have been told it was a huge success as it has lightly over 20 votes.

 

[oggbashan]

...

As for poems I have one and have been told it was a huge success as it has lightly over 20 votes.

Most of mine have less than 10 votes. Now one, unrated since September, has a single vote.

 

[jomar]

While the red H's, etc are fun, you tend to let go over the years and scores become less important, largely because it becomes clearer how easily they can be manipulated.

 

[oggbashan]

While the red H's, etc are fun, you tend to let go over the years and scores become less important, largely because it becomes clearer how easily they can be manipulated.

Ratings on older stories tend to slide slowly downwards - at least they do on mine.