Revelation about Ratings

[madelinemasoch]

I did the math on some of my ratings as far as what percentage of readers on a particular story actually rated it. It turns out, it was less than 1%. The ratings can't possibly reflect the readership's opinion as a whole if only 0.1% of readers actually rated the story. That final number you see is pretty much meaningless.

It's funny to see how many authors on here actually take ratings as measurements of quality, when they're not, because the vast majority of readers of any given story don't actually rate them. I think it'd be funnier if beside every Red H there was the number indicating the percentage of the story's readership that actually gave it a rating. It'd kind of expose how little the Red H actually means anything. It literally doesn't matter. Imagine if you had a country where only 0.1% of the population voted on anything. Would you take their decisions as reflections of what the people in that country believe?

Let's do all of the math for every story I once had posted on this site, without titles.

1. 21130 reads and 83 votes: 0.3928%
2. 8149 reads and 44 votes: 0.5399%
3. 18526 reads and 48 votes: 0.259%
4. 7815 reads and 32 votes: 0.4%
5. 18334 reads and 21 votes: 0.1145%
6. 9143 reads and 19 votes: 0.2%
7. 23611 reads and 67 votes: 0.2837%
8. 53775 reads and 251 votes: 0.4667%
9. 31816 reads and 128 votes: 0.4%
10. 17844 reads and 67 votes: 0.3754%
11. 17218 reads and 57 votes: 0.331%
12. 10749 reads and 34 votes: 0.3163%
13. 7503 reads and 32 votes: 0.4264%
14. 16079 reads and 40 votes: 0.2487%
15. 9776 reads and 21 votes: 0.2148%
16. 11637 reads and 31 votes: 0.2663%
17. 9736 reads and 28 votes: 0.2875%
18. 8144 reads and 25 votes: 0.3%
19. 9167 reads and 18 votes: 0.1963%
20. 7433 reads and 24 votes: 0.3228%
21. 6376 reads and 14 votes: 0.2195%
22. 7976 reads and 20 votes: 0.25%
23. 7682 reads and 52 votes: 0.6769%
24. 11814 reads and 30 votes: 0.2539%
25. 17269 reads and 234 votes: 1.355%
26. 7808 reads and 43 votes: 0.55%
27. 3281 reads and 21 votes: 0.64%
28. 14429 reads and 81 votes: 0.5613%
29. 8556 reads and 49 votes: 0.5726%
30. 10996 reads and 46 votes: 0.4183%
31. 9238 reads and 33 votes: 0.3572%
32. 3661 reads and 22 votes: 0.6%
33. 6501 reads and 27 votes: 0.4153%
34. 3423 reads and 17 votes: 0.4966%
35. 3942 reads and 14 votes: 0.3551%
36. 2381 reads and 5 votes: 0.2%
37. 35069 reads and 108 votes: 0.3%
The average percentage of readers that voted was 0.3936%

Feel free to do your own stats and post them here.
What I found was that it only gets as high as 1% and that's an outlier. It goes as low as about 0.2% which is more common.

Do you really think what 0.4% of readers think of your story is an accurate reflection of how readers as a whole feel about it? I think not.

 

[lovecraft68]

My first reaction was yours might be lower because your stories are a niche and not a lot of views.

I don't spend a lot of time on stats, but did this quick. My last story, a student/teacher piece (Female teacher)

55.2k views 718 votes which is between 1-2%

The one before that. A Milf story 48780 views 1075 votes between 2-3%

Last incest story came in at 5% but it was exceptionally well received.

The only flaw here is the infamous question of what percentage of views are actual reads? Unless that could be determined for sure, it's impossible to know the true average percentage.

 

[SimonDoom]

It doesn't necessarily follow that because less than 1% of views end up as votes the votes aren't representative. The sample sizes of voter polls are much smaller fractions of the total voting population than that, and they still have great statistical significance in predicting election results. A very small sample relative to the population may have statistical significance, depending on how the sample is selected. In this case, the voting sample is entirely the result of self-selection, which significantly undermines the statistical significance of the final average score as representative of what all readers actually think of the story. If people who like a story are more likely to vote (I have no idea if this is true), then the final average score will tend to overstate what readers actually think of your story.

However, that's not the whole story. We all face, more or less, the same kinds of disparities, so you can say that voting results do matter to some degree for purposes of comparing one story with another. The numbers don't mean anything in and of themselves, but that's true regardless. They only have meaning relative to other numbers derived in the same fashion.

Keep in mind, too, that what you call "reads" are just "views." I estimate that actual reads are less than 25% of views (that estimate could be way, way off). My ratio of views to votes on average is about 90:1. If I'm right than the read to vote ratio is about 23:1 or around 4% voting rate.

 

[TheLobster]

Keep in mind, too, that what you call "reads" are just "views." I estimate that actual reads are less than 25% of views (that estimate could be way, way off). My ratio of views to votes on average is about 90:1. If I'm right than the read to vote ratio is about 23:1 or around 4% voting rate.

This is the most important observation. Story views are merely all the instances of someone opening the first page; you can't actually know how big a percentage of hits eventually translate into people who read through the story.

Absent reliable data, my hunch would be to go with the Sturgeon's law, that ninety percent of everything is worthless (in this case, worthless views that don't lead to any other engagement). Apply that twice, translating views to reads and then reads to votes, and you get the 1% ratio that's close to empirical data observed by the OP.

 

[SimonDoom]

Absent reliable data, my hunch would be to go with the Sturgeon's law, that ninety percent of everything is worthless (in this case, worthless views that don't lead to any other engagement). Apply that twice, translating views to reads and then reads to votes, and you get the 1% ratio that's close to empirical data observed by the OP.

I've long wondered about this. One could apply a "one out of ten" rule here. One out of ten people who click on a story actually read it all the way through, and one out of ten people who read the story vote on it. That calculation rests upon a consistent model of human behavior, so it has some appeal. Of course, it's a guesstimate. I suspect some will object to it because it creates a discouraging picture of how many people actually read our stories. It's consistent, however, with my behavior, because I click on many stories and take a quick look without reading them to the end.

 

[StillStunned]

I think it also depends on the category. My I/T and SF&F stories are mostly 2-5%. Other categories are about 0.5-1%.

 

[TheWritingGroup]

I've long wondered about this. One could apply a "one out of ten" rule here. One out of ten people who click on a story actually read it all the way through, and one out of ten people who read the story vote on it. That calculation rests upon a consistent model of human behavior, so it has some appeal. Of course, it's a guesstimate. I suspect some will object to it because it creates a discouraging picture of how many people actually read our stories. It's consistent, however, with my behavior, because I click on many stories and take a quick look without reading them to the end.

I wish Literotica kept stats for multi-page stories like "What percentage read all the pages including the last one?" That would actually tell us how many people finished those stories.

-Billie

 

[Kumquatqueen]

I've long wondered about this. One could apply a "one out of ten" rule here. One out of ten people who click on a story actually read it all the way through, and one out of ten people who read the story vote on it. That calculation rests upon a consistent model of human behavior, so it has some appeal. Of course, it's a guesstimate. I suspect some will object to it because it creates a discouraging picture of how many people actually read our stories. It's consistent, however, with my behavior, because I click on many stories and take a quick look without reading them to the end.

It seems roughly true for a lot of stories - about 1 in 100 viewers vote, so we can guess that somewhere in between 100% and 1% of views are actual reads of more than the first few paragraphs. 10% would be a logical progression.

Stories where it's obvious what to expect from the title and tagline - or are consistent with expectations - will likely get more actual reads than more unusual stories.

After a certain number of votes, viewers appear to stop voting - possible because they see a rating and choose to up or down-vote if they disagree with it, but if they broadly agree with a rating, they leave it be.

 

[SimonDoom]

After a certain number of votes, viewers appear to stop voting - possible because they see a rating and choose to up or down-vote if they disagree with it, but if they broadly agree with a rating, they leave it be.

One possible explanation is repeat reading. Over time, the number of "repeat" readers relative to the whole number of viewers is likely to increase, but those readers obviously aren't going to vote again.

 

[Agent0069]

I’d argue that “reads” are a nothingburger, while “votes” actually represent people that cared enough to read the story until the end, and then lingered a bit longer to press a single button. Even more valuable are comments, then - personal emails, and if anyone cared enough to fly into your state to buy you a beer/coffee - that’s a divine gift.

 

[Bamagan]

I've had some of the same thoughts as the OP regarding voting.

As mentioned variously above, the main issue is sample bias, moreso than raw totals. Votes to views ratios are generally too low to make reliable inferences about the entire population of readers, but it may give you some legitimate insight into your readers, as a subset of the population that reacted strongly to your story, for good or for ill. But even then, you probably need a hundred votes or more to see the signal amongst the noise.

This still doesn't necessarily mean good things for making apples to apples comparisons between stories, though, because the subsets of the population voting on his story and voting on her story might have very little overlap. Both works might be rated at 4.5 and come by it more-or-less honestly, and yet be oceans apart in terms of technical merit or whatever other criteria one wants to apply. So when it comes to 'dick-measuring' contests, their value is more dubious.

 

[Jorunn]

I vote whether I like my story when I hit the Submit button. That's good enough for me.

 

[Kelliezgirl]

As Simon pointed out, a small sample size doesn't make the sample useless. The real challenge is we don't know if the population is representative or not.

One of the last presidential polls had a sample size of 2700, from a population of 334,914,895.
Or .000806%
Makes the lit voting rate look pretty darn good.



The truly fascinating thing about all these discussions of the rating system is how they always come from the same premise.
I've yet to see anyone make the claim that high ratings on Lit prove they are a great author. Instead what we consistently see is people arguing that their stories are really good despite their low ratings.
The real flaw with the rating system seems to be that it doesn't give some people the result they want.

 

[JGittes]

With my small story sample, the direct correlation between clicks and votes is story length.

Length is the only significant criterion that isn't visible to a reader before clicking to open the story. My stories tend to be long, and the vote/click ratio on even my shorter stories is lower than most I've seen reported. Nonetheless, my longer stories have fewer readers/voters who take the plunge once they know the length.

One author who posts regularly reports that Novels/Novellas doesn't suffer this fate.

 

[RejectReality]

Somewhere between 15-25% of your views are bots and spiders that the filter doesn't catch. ( Number comes from comparisons of the same stories released at nearly the same time on a site that only records downloads from logged-in members ) That number rises the older your story is, because the bot filter has been upgraded to such a point it made a huge change at least twice over the years. Something from the mid 00's would start at closer to 30% for a minimum.

That's just the non-human hits.

 

[madelinemasoch]

Somewhere between 15-25% of your views are bots and spiders that the filter doesn't catch. ( Number comes from comparisons of the same stories released at nearly the same time on a site that only records downloads from logged-in members ) That number rises the older your story is, because the bot filter has been upgraded to such a point it made a huge change at least twice over the years. Something from the mid 00's would start at closer to 30% for a minimum.

That's just the non-human hits.

Are you sure? How do you prove this?

 

[Lifestyle66]

The percentage of votes from the number of views is an interesting metric.

Of my 62 stories, I now average 18,900 views per story with and average of 390 votes per story, or just over 2%. even if I do the individual percentages, and average those it's the same.

That said, the individual stories range from 0.41 to 5.46.

The secret is to engage an emotional audience and provoke them to 1-bomb your stories. And if it hadn't been for the Admins scraping a shitload of 1s from my stories, I'm sure the percentage would be much higher!

Soooo... How many more votes do you want? Just post to Loving Wives!!!!

 

[MostHated]

My first reaction was yours might be lower because your stories are a niche and not a lot of views.

I have experimented with writing in niche categories, and I agree with you 100%

Far fewer views, but much better results with views and ratings. I suppose if ratings are important then writing in those niche areas may be the fastest way to the dopamine release (there's nothing wrong with that.) but I've forgone any realistic views/ratings/comments (here.) a while ago, and that works well for me.

 

[MostHated]

With my small story sample, the direct correlation between clicks and votes is story length.

Length is the only significant criterion that isn't visible to a reader before clicking to open the story. My stories tend to be long, and the vote/click ratio on even my shorter stories is lower than most I've seen reported. Nonetheless, my longer stories have fewer readers/voters who take the plunge once they know the length.

One author who posts regularly reports that Novels/Novellas doesn't suffer this fate.

I'm one of those readers that want more than a single page, and less than 4-5 pages of story text, additionally, I skip stories that I see arrive as new, that are ongoing chapters of older stories. I tend to like my smut condensed into a single story at a time.

 

[RejectReality]

Are you sure? How do you prove this?

Just comparing my statistics across this site and another, where downloads are only recorded when a logged-in member opens the story. Some types of work ( Sci-Fi & Fantasy in particular ) draw nearly identical vote totals on both sites, making them particularly useful barometers, and I have a lot of them for sample size in my other pen names.

It's always possible that the readers there vote more often than readers do here, which would skew it, but nothing I've seen indicates that's the case. Nevertheless, I knock the numbers down a few percentage points ( basically rounding them down to 5s ) to account for that possibility. Underestimating is better than overestimating in this case.

As to the changes in the view numbers here, anyone who's been here since before 2010 can pretty much tell you that there are two distinct points where the recorded views declined, which I attribute to improved filtering. ( because the traffic numbers do not indicate a decline in website visits )

 

[Comshaw]

Because the sample size is small I don't believe that means the vote total is wildly inaccurate. The national polls on politics and other things are usually pretty damned accurate even if they are infinitesimally small compared to the overall population. I would argue it is the same with votes on a story.

And I think you missed one other piece that works into this: the readers that do vote and why they vote.

From experience on other websites that do post what each vote is, I can say you rarely if ever will get a vote other than the top and bottom. Here it's a 1 or a 5. On another site I have posted stories to, it is a 1 or a 10. I have 34 stories posted there and I can count on the fingers of one hand votes other than a 1 or a 10.

The reason I went through that explanation was to say this: You only get 2 kinds of readers that vote (discounting the odd troll or two) on a story, those who utterly love it and those who utterly hate it. The higher a score is on a story the more "love it" readers voted. Even though the sample is small, you can reasonably extrapolate from that (just like a national poll) that most of those who read it liked it too.

Now a couple of questions for anyone willing to answer: If you as an author do not believe the votes mean anything, why keep them active? If you do not believe votes mean anything as to the quality of a story, what metric do you feel should be used to measure that?

Personally, I think the readers are the best and only metric to use. And I can't think of another way to do it other than voting. Can you?



Comshaw

 

[mirafrida]

The weird thing about ratings and erotica, in my mind, is that people's assessments are so much more deeply inflected through their very particularized preferences and fantasies and fetishes. If I think about "media ratings" in a larger sense, ala rotten tomatos or goodreads, well, some people like action more, or superheroes, or romance, or historical drama - and will probably rate higher or lower accordingly. But still, the distinctions are much smaller and less visceral. I believe there a heightened conflation on this site between the "that sucks" low ratings, and "ick" low ratings, and it makes rating less useful as a concept in this context.

I imagine most people who have written here a while, with some seriousness, will occasionally have some commenters who say "you are one of the best writers on the site," or "you do X better than anyone." Not least because we're a good match for them in terms of the "preferred fantasy set." My presumption is that there are usually many more of those people out there who have similar leanings, but whom we HAVEN'T reached yet - and of course we'd all like to. Contrariwise, I, at least, and probably others, would just as soon not reach the people for whom my fantasies are an "ick."

But achieving this is hard to do, and I tend to think ratings are more of a hinderance than help in doing that matching. Certainly there's a fear in my mind that low ratings based on "ick" from ill-targeted readers, will end up deterring casual / drop-in readers who DO happen to be a match for my stuff from trying it.

 

[SimonDoom]

But achieving this is hard to do, and I tend to think ratings are more of a hinderance than help in doing that matching.

I'm curious why you think this. It's tantamount to saying that no information for readers is better than some information. That seems like a strange position to me, unless one believes that ratings have no statistical correlation with quality. I think that's a bizarre thing to think.

I've been a reader far longer than I've been a writer here. I can say confidently that I find story ratings somewhat helpful in choosing stories to read. I would be sorry to see them go. I can't help but think that authors who criticize the ratings system are looking at the system as authors, and they're forgetting that the system is designed for readers, not authors.

 

[Bramblethorn]

It doesn't necessarily follow that because less than 1% of views end up as votes the votes aren't representative. The sample sizes of voter polls are much smaller fractions of the total voting population than that, and they still have great statistical significance in predicting election results. A very small sample relative to the population may have statistical significance, depending on how the sample is selected. In this case, the voting sample is entirely the result of self-selection, which significantly undermines the statistical significance of the final average score as representative of what all readers actually think of the story.

Yup. For something where the exact scores matter, random sampling is really important. On something like a political survey where I want to know who's getting more votes, a truly random sample of 1000 eligible voters will be more accurate than a survey of 1000,000 people chosen by some biased method, e.g. self-selection.

However, that's not the whole story. We all face, more or less, the same kinds of disparities, so you can say that voting results do matter to some degree for purposes of comparing one story with another. The numbers don't mean anything in and of themselves, but that's true regardless. They only have meaning relative to other numbers derived in the same fashion.

Yep, with some qualifiers - I think the biases are likely different between categories, and between long and short stories. But ignoring that:

If I posted a story on Literotica and got a score of 4.40, and I'd never looked at any other story scores, I'd have a very hard time figuring out whether that was good. But if I got 4.40 on one, then 4.50 on the next, and 4.60 on the next, I might speculate that I'm improving.

That's where sample size does matter. As a rough approximation, the random noise in a story score is on the order of 1.00/sqrt(n_votes); differences smaller than that are generally not interpretable.

Say my stories mentioned above each had 100 votes, so the random noise in each score is about 1.00/sqrt(100) = +/- 0.10. I can't reliably say that the 4.50 is doing better than the 4.40 or worse than the 4.60; those differences are small enough to be pure chance. I can probably say that the 4.60 is better than the 4.40 though.

 

[alohadave]

I imagine most people who have written here a while, with some seriousness, will occasionally have some commenters who say "you are one of the best writers on the site," or "you do X better than anyone." Not least because we're a good match for them in terms of the "preferred fantasy set."

As a reader, I have read many stories that may not have been particularly well written, but they hit the kink or genre well and told a good story.

When an author hits just right for me, I'll read as much of their stuff as I can.