Score comparisons between categories

[Actingup]

Righto, following my earlier threads looking at the difference between percentile scores in Loving Wives and between 2025 and 2024 generally, I've done a deep dive into a dozen Literotica story categories and how they scored during 2025. I used the same method described earlier - in brief, this uses a score sorted search retrieval of stories and then a calculation of story percentiles (the 1st, 5th, 10th....90th, 95, and 99th score percentiles in the category). So, if a story has a percentile of 50 (the median), half the stories in the category are above it, and half below. If the story has a percentile of 99, only 1% of the stories in the category are above it. If the story has a percentile of 1... well, you're not reading this post anyway.

So, let's start with a table of results. Although it's only a dozen categories, these 44048 stories cover 66% of the published stories in the past 12 months - if you're interested in other categories, sorry, but you might have to calculate the stats yourself unless you can convince me that another category would be particularly interesting to look at.


For ease of reference, I've marked every score of 4.5 or above in red so that you can see what's 'HOT'. So, for example, you can be hot in Sci-Fi with a 28% percentile or above (everybody gets a prize!), but to be hot in Loving Wives, you need about a 97.5 percentile (he says smugly). Similiarly, gay and lesbian story readers are kinder than reluctance/non-consent readers (who knew!?).

Let's graph that out:



and because nearly everything is clustered together, let's zoom in to the upper range:

Sorry the graphs are rather messy - you can see why I stopped after a dozen categories. There are probably lots of little comments to make about the way that the categories compare, but I'll spare you any more snark about one versus the other - they are simply different.

What can we take from this? Well, to re-state the obvious, our story scores are not an absolute measure of quality, and particularly should not be compared across categories unless you're aware that voter scoring behaviour is not consistent. Even if you exclude the hard arses of LW readers, there's still a difference of nearly 0.3 between the median R/NC and Sci-Fi scores, and that is significant. If you're posting in a hard-marking category, you could use this as a measure of comfort. If you're posting in an easier category, use this to stay humble. And I think that many of us could stand to remember that the 'Hot' rating is not that significant or important. As stated many times, LW readers (for example) will give you masses of reads and comments even if they are scoring your story harder. Scores are not a measure of engagement, and they don't necessarily drive engagement in an absolute sense.

The analysis might also be useful if you publish across these categories and you'd like to look at what readers thought of your stories using a more subtle scoring mechanism. I now have evidence, for example, that my top rated stories in percentile terms are in LW and T/I rather than in Romance (where my top scoring stories are). I know that, as suspected, Group Sex is a little tougher other categories, and I can forgive myself for not being so spectacular there (er, I mean in terms of the stories of course). I also know that my resentment at Erotic Horror readers for not recognising my genius is maybe a bit unfair, since they clearly do reward good stories.

Oh, and I think that this stuff is more meaningful than the much contested top lists, too. If you're in the top few % in your category, you're in the big league without needing to be at the top of the pile.

Comments welcome!

 

[YmaOHyd]

Instant first comment: I think the series effect in sci-fi/fantasy is probably significant. I don't know a good way to test that hypothesis. My impression, though, is that sci-fi/fantasy has generally fewer standalones and generally more series, and series entries filter out non-fans. The #1, #4, #6, #9, #11, #15 and #19 most-discussed series are sci-fi/fantasy, which is quite a bit of the top 20.

 

[iwatchus]

@Actingup Does the data you get include a flag for series?

 

[Actingup]

@Actingup Does the data you get include a flag for series?

No, I'm not adjusting for series, as each episode is scored differently (also, I can't be fagged doing that!). I agree with @YmaOHyd's comment.

 

[NotWise]

It\s a useful way to compare between categories. Would it be possible to construct a scaling factor for scores in different categories that would bring them all about into line?

 

[Actingup]

It\s a useful way to compare between categories. Would it be possible to construct a scaling factor for scores in different categories that would bring them all about into line?

Probably - it wouldn't be quite a linear scaling though. I'll leave that to the mathematicians here. You can do it visually from the graphs above of course - a 4.2 in LW equals a 4.7 in R/NC.

 

[NotWise]

Probably - it wouldn't be quite a linear scaling though. I'll leave that to the mathematicians here. You can do it visually from the graphs above of course - a 4.2 in LW equals a 4.7 in R/NC.

Yeah, a table would probably work better.

edit: and you've already given us the table

 

[EmilyMiller]

but to be hot in Loving Wives, you need about a 97.5 percentile (he says smugly)

I’ll see your 4.53 and raise you 4.64

 

[caleb35]

Just personal anecdote/experience here, but I've found (generally) that my scores in Lesbian category tend to be a tad higher but readership is lower, compared to NonCon/Reluctance, where readership is much higher but scores are the same or a bit lower. Whether that's because NonCon/Reluctance has more 1-star bombers, a pickier readership, or a bit of both, I don't know. Thanks for compiling this data.

 

[Actingup]

I’ll see your 4.53 and raise you 4.64

Well yes, but I've got a 4.65 as well, so nyah nyah nyah! (steadfastly refuses to glance at NTH's story list)

 

[EmilyMiller]

Well yes, but I've got a 4.65 as well, so nyah nyah nyah! (steadfastly refuses to glance at NTH's story list)

Have you got a 3.49 as well to get the full LW experience?

 

[Actingup]

Have you got a 3.49 as well to get the full LW experience?

Sadly, not quite, but note that a 3.49 is still around the 41st percentile, so you're going to have to try harder - perhaps try setting the husband and wife up with a couple of bisexual men?

 

[mirafrida]

I think I've got stories in just about every decile...

 

[Actingup]

I think I've got stories in just about every decile...

Case in point - you post in R/NC and LW, the two toughest categories. The scores are better than they look (and a few of them are >90th percentile).

 

[TheLobster]

I don’t quite get what this data is supposed to tell us. I mean, sure, it does show the scoring percentiles, but it misses so many variables that it’s hard to make any use of it.

I suppose what would be useful is to have something to calibrate your expectations against, so that you know how to interpret a particular score your story gets in a particular category. But this doesn’t offer it, and I don’t think any simple statistical measure like that can.

First, there is the series effect, as the first reply mentions, which not only seriously inflates scores in some categories but also deflates them in others.

Second, relative audience sizes of different categories matter a lot. The bigger the readership, the more likely it is that the outlier scores are outliers purely by chance — i.e., not because they’re particularly good, but because they got lucky and avoided the crowd that’d hammer them back to the mean. Combined with the first effect, it further skews the scores of middle chapters of series.

Third, the impact of followers is hard to overstate, especially for authors that post in multiple categories. It again amplifies the other two factors.

Fourth, we may or may not be aware of the deeper dynamics working inside specific categories. You included LW, which is known to be special with its factions and whatnot; and you have deftly avoided Fetish, which is really 20+ different categories in a trench coat (and very much wants to flash you its feet). But we don’t quite know what’s going on elsewhere either; for example, I suspect T/I shows tendency to get stratified by the type of coupling, with different reader behavior in case of siblings and mom/son (and whether backseat is involved) and other pairings.

I guess my overall point is that while you caution about comparing individual story scores between different categories, you’re simultaneously strongly suggesting that comparing those aggregate, bucketed & graphed scores is actually alright, and useful. Sorry, but I don’t think your analysis is strong enough to make that claim, as per the reservations I touched upon above.

In other words, more research is needed

 

[RocketGrunt]

Agree that the series effect is strong in SF/F especially. Later episodes get less views but engagement per view is much higher. As a series progresses the number of people who read the series and aren't fans dwindles to nothing. On another thread I went into a deep dive about the series that owns the top list in Non-Human and found that Chapter 133 has been rated once for every 11 views, a level of engagement that's several times higher than what you'd get from a stand alone story in any category.

Today, there were 13 posts in Sci-Fi/Fantasy. Of them, ELEVEN mark themselves as being chapter something of a series. Three of them are just chapter 1 or chapter 2, but three of them are double-digit chapters and one of them is a triple digit chapter.

I don't think that the Sci-fi/fantasy readers are any kinder than the other categories, but I'm absolutely certain that readers of chapter 117 are self selected to be interested enough in the story to still be reading after chapter 116.

 

[ElectricBlue]

In other words, more research is needed

I think the key point is that every category has roughly the same curve, but don't compare one category to another.

The data also demonstrates the skew of the Red H. No matter what authors like to think, the Red H pulls in readers (who might not even know how a story gets a Red H).

 

[Actingup]

you have deftly avoided Fetish, which is really 20+ different categories in a trench coat (and very much wants to flash you its feet).

Perhaps I'm worried about the graphs getting caught up in a pissing contest between the factions?

But we don’t quite know what’s going on elsewhere either; for example, I suspect T/I shows tendency to get stratified by the type of coupling, with different reader behavior in case of siblings and mom/son (and whether backseat is involved) and other pairings.

I guess my overall point is that while you caution about comparing individual story scores between different categories, you’re simultaneously strongly suggesting that comparing those aggregate, bucketed & graphed scores is actually alright, and useful. Sorry, but I don’t think your analysis is strong enough to make that claim, as per the reservations I touched upon above.

I'm suggesting the data is useful, and I guess I'm strongly suggesting that the data is more useful than the very coarse 'hot or not' measure. I particularly wanted to lay out the issue with 'hot' more clearly and the extent of the differences between the non-LW categories. But the data doesn't explain everything, and there is also much that would be really interesting to extract but isn't easy to access. More research may indeed be required to tease out some things, although probably not by me.

 

[pink_silk_glove]

I don’t quite get what this data is supposed to tell us. I mean, sure, it does show the scoring percentiles, but it misses so many variables that it’s hard to make any use of it.

It doesn't miss anything that the raw numbers also miss.

Any sort of hot rating should absolutely be tied to a percentile bench mark rather than a hard coded number. Each category should be graded individually by percentile just as you have above to determine hot rating. There are a few of us here who have been saying this for at least a couple of years.

 

[BobbyBrandt]

Agree that the series effect is strong in SF/F especially.

The effect is the same in other categories, such a L/W, which the OP seems pretty focused on at the moment.

 

[RocketGrunt]

The effect is the same in other categories, such a L/W, which the OP seems pretty focused on at the moment.

The effect on a story is the same. The effect on the category is very different.

Sci-Fi/Fantasy had 13 new stories today and EIGHT of them were later chapters of a series.
Loving Wives had 15 new stories today and FOUR of them were later chapters of a series.

I don't see why or how there could be any real filtering effect for Chapter 1, and probably not much of one for Chapter 2, but later chapters are really filtered quite a bit by the sieve of fans continuing to read and people who didn't like the early chapters attritioning away. I don't think it's a bizarre claim to say that a late chapter in a story is overwhelmingly likely to get a Red H, but when that accounts for more than half the stories posted on the daily on the SF/F page, that is sufficient to completely explain the OP's finding that a majority of the chapters on SF/F get Red Hs.

On LW, less than a third of the offerings today are continuations of works that have established readerships. On SF/F, more than half of the offerings today are continuations of works that have established readerships.

 

[TheLobster]

Chapter 1 filters out all the people who only read standalone stories, and if it's an incomplete series then also those who only read complete series.

I haven't elaborated on it much, but when it comes to series there is also a reverse effect to score inflation which happens in the less, let's call it specialized categories. As the series goes on, authors are more likely to trip over some squick that the typical category readers find objectionable and thus cop a lower score.

SF&F is broad enough that this isn't really an issue, but in many other categories this is definitely the case.

 

[Writer61]

My takeaway is that most categories are broadly similar. In seven of the twelve, being in the 50th percentile means you have a score of 4.5 or better (i.e. 'HOT'). In three more, the 60th percentile gets you there. The only significant outlier is LW.