Vote Breakdown on Stories

[Bamagan]

It would be interesting if we could view the total number of votes by rating rather than just the mean. It's not a super useful statistic in isolation no matter how it's presented, but a story with a rating of 3 after ten votes could have a wide variety of possible ways to get there: all threes, 5 ones and 5 fives, and plenty of combos of 1-5. But each of those winds up telling a more nuanced story. The overall rating suggests "Meh," which would be supported by having 10 threes, while the 1-5 split scenario suggests a serious love-hate response, and possible other interpretations if the distribution winds up as a bell curve of some kind. Of course, that might just lead to more people obsessing over their scores, which probably doesn't provide much benefit to anyone. I have a hypothesis that a rating of '2' is the least used of the five choices, though, and would be curious to see if I'm right.

 

[KeithD]

It would be interesting if we could view the total number of votes by rating rather than just the mean.

To my knowledge, we can't.

 

[Djmac1031]

I look at ratings as an average of reader opinions. Nothing more, really.

A rating anything above a 4 tells me the majority of readers liked it.

I'm not gonna obsess over the few who didn't like it, and be happy most did.

 

[Bamagan]

To my knowledge, we can't.

Haha, yeah, that's kind of why I was lamenting!

 

[Bamagan]

I look at ratings as an average of reader opinions. Nothing more, really.

A rating anything above a 4 tells me the majority of readers liked it.

I'm not gonna obsess over the few who didn't like it, and be happy most did.

Yeah, when presented with only the mean/average, the score tells you only what the audience thought collectively. I don't think anyone is helped by obsessing over their scores, good or bad, but a breakdown of how the votes were cast could provide some additional insight into how the audience received it, whether it was a smooth distribution or spikes at either end or whatever. I'm not sure that knowledge could actually help an author identify what specifically was well- or poorly-received about any given work, absent other feedback from comments, but the data nerd in me wishes I could get my hands on more granular details!

 

[mirafrida]

Yes, it would be nice to distinguish between the 'meh' story that got a lot of 4s, and the 'love/hate' story that is all 5s and 1s (though, to be honest, you usually know from other clues).

I have a hypothesis that a rating of '2' is the least used of the five choices, though, and would be curious to see if I'm right.

Having gone through an OCD period of tracking these things, I believe you're right, although '2' and '3' are very close and both much lower than all others.

 

[Djmac1031]

Yeah, when presented with only the mean/average, the score tells you only what the audience thought collectively. I don't think anyone is helped by obsessing over their scores, good or bad, but a breakdown of how the votes were cast could provide some additional insight into how the audience received it, whether it was a smooth distribution or spikes at either end or whatever. I'm not sure that knowledge could actually help an author identify what specifically was well- or poorly-received about any given work, absent other feedback from comments, but the data nerd in me wishes I could get my hands on more granular details!

I get that, I do.

I still think you can make some, admittedly reaching, assumptions on scores as is.

Again, a 4.5 or higher is easy; most readers loved it, the rest liked it well enough.

Low fours are fine; the average reader still enjoyed it.

Under 4 isn't terrible, but gives us some clues. Maybe the story touched on a controversial subject, and so triggered some readers.

Or maybe the writing just wasn't that consistent; too many errors to justify a higher rating.

Low 3's and under 3 scores send up more red flags. Yes, there are trolls and One Bombers, but you still need a LOT of them to truly destroy a score on a GOOD story.

I consider a One Bomb the same as I consider a negative comment in a mix of tons of positive praise.

If someone has constructive criticism and / or a legitimate reason to hate my story, fine. I don't mind criticism.

But if they're just gonna One bomb it and leave no comment, or only comment long enough to say "you suck," well, not much I can do about that except ignore it, especially if the story is doing well with the majority of other readers.

 

[Bamagan]

I would also like to add, I'm not referring exclusively to an author's own works when I say I'd like to see the breakdown of votes. I do pay some attention to the score when I'm looking for something to read, although I principally select based on category and tags (when available). I might hesitate on a story that's got a 3.xx rating, since that might mean it's a grammatical mess or something. But if I could see that the rating has a spike of 1s dragging it down, I would probably be more charitable and assume part of the negativity is just knee-jerk aversion to some kind of fetish, which honestly is probably why I was looking for it in the first place!

 

[Bamagan]

I would also like to add, I'm not referring exclusively to an author's own works when I say I'd like to see the breakdown of votes. I do pay some attention to the score when I'm looking for something to read, although I principally select based on category and tags (when available). I might hesitate on a story that's got a 3.xx rating, since that might mean it's a grammatical mess or something. But if I could see that the rating has a spike of 1s dragging it down, I would probably be more charitable and assume part of the negativity is just knee-jerk aversion to some kind of fetish, which honestly is probably why I was looking for it in the first place!

Or to put it another way, I would let the one-bombs help draw me to the content that the haters hated, perhaps ironically.

 

[Djmac1031]

I would also like to add, I'm not referring exclusively to an author's own works when I say I'd like to see the breakdown of votes. I do pay some attention to the score when I'm looking for something to read, although I principally select based on category and tags (when available). I might hesitate on a story that's got a 3.xx rating, since that might mean it's a grammatical mess or something. But if I could see that the rating has a spike of 1s dragging it down, I would probably be more charitable and assume part of the negativity is just knee-jerk aversion to some kind of fetish, which honestly is probably why I was looking for it in the first place!

My comment above can be applied not just to our own stories, but those we seek out to read.

Do I automatically ignore a story rated under 4? Not at all. I get how it works here.

But if a story is just wallowing in the low threes or under, well, it definitely tells me there's some kind of problem beyond troll attacks.

 

[ronde]

I understand what you're saying, but the real problem here isn't the votes that make up the rating. It's the sample size relative to a very small number of readers. I think most votes come from readers who know the author and his or her work and not from the general population of readers. In my case, I average less than two votes per hundred reads and that's assuming each read means the reader actually read it. In reality, the readers who vote are the readers we write for so the average is a pretty good indication of how they view each story. Even if each vote were known, trying to extrapolate that data to the people who read the story would be the equivalent of sampling two percent of one area of a city for hair color and then extrapolating that data to the US in general.

 

[Bamagan]

My comment above can be applied not just to our own stories, but those we seek out to read.

Do I automatically ignore a story rated under 4? Not at all. I get how it works here.

But if a story is just wallowing in the low threes or under, well, it definitely tells me there's some kind of problem beyond troll attacks.

When a story has hundreds of votes or more, yes, the mean rating is probably a reasonably good indicator of quality. I'm more reluctant to give it much credence on newer stories, though, or ones that might reasonably have very few votes, since extreme votes can sway those scores pretty dramatically until they get drowned out by the vox populi.

 

[Bamagan]

So you’re proposing a system like that of amazon customer reviews or the apple app store or [insert a gazillion examples]? Sure, that’d be nice. I like data — and I do glance at that stuff before pulling the trigger on purchasing something on those websites.
So…why not?
Unless it’s technically too expensive for Lit to implement.

I have no idea how difficult it would be for them to implement some kind of display for it, but they clearly keep granular records of votes since they can do periodic sweeps and purges, so it's not like the underlying architecture would need to be reconfigured.

 

[mirafrida]

I would also like to add, I'm not referring exclusively to an author's own works when I say I'd like to see the breakdown of votes.

You bring up a good point. My main beef with scores is that they purport (or are assumed) to help people figure out what they will enjoy reading - but ultimately obscure or confuse more than they reveal. Seeing the score breakdown would at least give the readers a bit more data than they have now.

(I'm reminded of when I was a Lit newbie, thinking 'well, why don't I start with the best stuff,' and then being confounded by top-stories lists containing very little that I personally wanted to read.)

 

[NotWise]

You can track the rating if you want and back out what the actual votes were. That requires a lot of attention, and frequently requires estimates when votes come in faster than you can record them. Roundoff error limits the ability to back-calculate the individual votes when the story has more than 100 votes.

I've done that for years now with varying success. From what I've seen I can tell you something about what the distributions are likely to look like.

There's no bell curve. Having just five values to pick from and readers with widely varying opinions makes that unlikely.

The "gap" distributed votes (1's and 5's) are likely to be rare. A lot of people like to think this is a common voting pattern, but the closest I've come to that distribution was the story that started me tracking votes to start with. It was mis-categorized; the category readers didn't like it, and other readers seemed to like it. Maybe stories in LW can have that pattern. I don't know.

The most general pattern I've seen is that the number of votes in one category is about equal to the votes in the next higher category times a constant. Given that pattern, a three-star rating means that the vote distribution is flat. A rating below three (I've never seen one) would have more one-star votes than five-star votes, and a rating above three would have more five-star votes than one-star votes.

I think that for every one of my stories, the most common vote is a five-star vote. The most commonly swept vote may also be a five-star vote, but that's hard to prove.

For those who do want to track the individual votes, the site could help by adding another digit to the rating you get when you download the CSV file of your story stats. Popping up the vote count and total number of stars when you mouse over the rating on your works page would also be useful. As it is now, it doesn't pop up anything very useful.

 

[PrincessArianrhod]

I agree it would be a nice feature and certainly more useful than just a mean average. I was thinking the same thing as in the last few hours I watched my latest story drop from a steady 4.95 to 4.52 in just a few votes.

If you don't get 10s of thousands of views and 100s of votes, spiteful voting affects the mean disproportionately. Its pretty demotivating.

 

[Stacnash]

I agree it would be a nice feature and certainly more useful than just a mean average. I was thinking the same thing as in the last few hours I watched my latest story drop from a steady 4.95 to 4.52 in just a few votes.

If you don't get 10s of thousands of views and 100s of votes, spiteful voting affects the mean disproportionately. Its pretty demotivating.

Those dominating the top lists in that category won't have enjoyed you muscling in on their turf with a 4.95.

Don't let it get you down, your stories are brilliant.

 

[Bamagan]

You can track the rating if you want and back out what the actual votes were. That requires a lot of attention, and frequently requires estimates when votes come in faster than you can record them. Roundoff error limits the ability to back-calculate the individual votes when the story has more than 100 votes.

Haha, yeah, that might work okay for one's own stories, although it might be veering close to the 'obsession' we were worrying about above! I'm not hugely concerned about my own scores, although it's always nice to be validated. But the vote tracking and calculating won't help much for assessing another author's work, especially since I don't think it shows us how many votes are included in their rating.

 

[Bamagan]

I agree it would be a nice feature and certainly more useful than just a mean average. I was thinking the same thing as in the last few hours I watched my latest story drop from a steady 4.95 to 4.52 in just a few votes.

If you don't get 10s of thousands of views and 100s of votes, spiteful voting affects the mean disproportionately. Its pretty demotivating.

Small sample size mood swings are the blues personified!

 

[Djmac1031]

There's one story of mine in particular I've mused over what the rating tells me and being honest, I think it's probably my most fair, consistent score.

April Fools Daddy currently sits at a 4.16 rating with 55.6K views and 822 votes.

The comment section sings it's praises. So why not a higher score?

Was it mercilessly attacked by trolls, one bombed repeatedly?

I don't think so.

When I first posted it, it started in the mid fours then suddenly dropped to like a 3.8. Then slowly bit surely crawled it's way back up over 4 to where it is now.

I had one friend / reader suggest the score would dramatically increase after a sweep, but I doubted that.

And I was right, because that never happened.

I don't think my story was attacked by one bombs.

I think ultimately what the rating history tells me is it drew a steady stream of 4 stars.

In other words, good, but not "great."

I can totally live with that. 4.16 is nothing to scoff at or be ashamed of IMO.

 

[NotWise]

Haha, yeah, that might work okay for one's own stories, although it might be veering close to the 'obsession' we were worrying about above! I'm not hugely concerned about my own scores, although it's always nice to be validated. But the vote tracking and calculating won't help much for assessing another author's work, especially since I don't think it shows us how many votes are included in their rating.

You can apply the concept to any score you see.

If you have enough votes, then the voting distributions are usually smooth. Ratings well over three means that the distribution slopes up to five, which is the most common vote. Rating well below three means the distribution slopes up to one, which is the most common vote. Ratings near three are evenly distributed.

You don't get many spikes in the distribution, because there's no standard for rating stories that anyone's agreed to, and because reader's opinions vary widely.

As far as showing the distribution to readers is concerned, I doubt many would even understand it, and even fewer would care. If it were done, most distributions would look a lot like the signal strength bars on your cell phone.

If you only have a few votes, then the distribution can be anything. You get no meaningful information from a small number of votes.

 

[Bamagan]

I can totally live with that. 4.16 is nothing to scoff at or be ashamed of IMO.

Maybe they can add another little icon for 'warm' stories. They already used W, so maybe a little orange-y 'F' for 'warm fuzzies.'

 

[Bamagan]

You can apply the concept to any score you see.

If you have enough votes, then the voting distributions are usually smooth. Ratings well over three means that the distribution slopes up to five, which is the most common vote. Rating well below three means the distribution slopes up to one, which is the most common vote. Ratings near three are evenly distributed.

You don't get many spikes in the distribution, because there's no standard for rating stories that anyone's agreed to, and because reader's opinions vary widely.

As far as showing the distribution to readers is concerned, I doubt many would even understand it, and even fewer would care. If it were done, most distributions would look a lot like the signal strength bars on your cell phone.

If you only have a few votes, then the distribution can be anything. You get no meaningful information from a small number of votes

To respond in reverse order:
It would be nice if they provided readers with the option to see the number and breakdown of votes, because as you mentioned, a small sample size does not lend itself to useful inferences, so a rating of 4.5 on a story with 10 votes is not necessarily a good gauge of quality. Probably it would not be of great value to many readers, who may not be statistically inclined or even prone to critical thinking. But putting it in a mouseover or something similar would be unobtrusive to the general reader and potentially valuable to some. I am not entirely prepared to accept the assurance that there are never or almost never spikes in the distributions, although they're probably a feature that get smoothed out over time, as you and @Djmac1031 have suggested. I assume that you're basing your statement on a careful analysis of your own scores, and possibly some affirmation from similarly inclined authors, but going back to small sample sizes, I'm not sure that your experience(s) as record-keeping author(s) is necessarily representative of all authors. I'm interested to know if the data for the general public bears out your experiences or not, if Literotica ever sees fit to make it available.

 

[Britva415]

It would be interesting if we could view the total number of votes by rating rather than just the mean.

https://www.literotica.com/s/how-to-analyze-your-scores

 

[Djmac1031]

To respond in reverse order:
It would be nice if they provided readers with the option to see the number and breakdown of votes, because as you mentioned, a small sample size does not lend itself to useful inferences, so a rating of 4.5 on a story with 10 votes is not necessarily a good gauge of quality. Probably it would not be of great value to many readers, who may not be statistically inclined or even prone to critical thinking. But putting it in a mouseover or something similar would be unobtrusive to the general reader and potentially valuable to some. I am not entirely prepared to accept the assurance that there are never or almost never spikes in the distributions, although they're probably a feature that get smoothed out over time, as you and @Djmac1031 have suggested. I assume that you're basing your statement on a careful analysis of your own scores, and possibly some affirmation from similarly inclined authors, but going back to small sample sizes, I'm not sure that your experience(s) as record-keeping author(s) is necessarily representative of all authors. I'm interested to know if the data for the general public bears out your experiences or not, if Literotica ever sees fit to make it available.

Oh absolutely. All I can speak from is my own, personal and admittedly limited experience when it comes to MY stories, how they're rated, and the size of my particular audience on any given story.

Which is comparatively low next to many other authors around here.