Ratings Scheme

[alohadave]

Again, we are all fully entitled to our own opinions, but I don't think that's a practical blanket statement because AI is just a technology and useful technologies are pretty addictive.

Substitute any useful technology in that sentence and it seems pretty impractical:

We can have our own opinion except when it's contrary to yours, apparently.

But maybe I'm misconstruing your point.

Yes you are.

But AI to run the site? Already happening. I think Laurel would say she must use an AI screener, for example.

Unless you have insider information, you are talking out of your ass. You can speculate about what they are using, but you don't have any more information than anyone else.

And though it seems you are hot on AI, not all automation implies AI. Whatever tools Laurel is using, they most likely predate any publicly available AI implementations.

 

[darwin1859]

Unless you have insider information, you are talking out of your ass. You can speculate about what they are using, but you don't have any more information than anyone else.

And though it seems you are hot on AI, not all automation implies AI. Whatever tools Laurel is using, they most likely predate any publicly available AI implementations.

It's just my ass talking, but let me reassure you that I have a very smart ass.

Ha ha, but seriously that's why I wrote detailed posts with what I see as evidence. I think the evidence I laid out is incontrovertible, and that Laurel uses "AI" to run LE*.

You're correct that I'm conflating GenAI, ML, automation, etc. as "AI" and they're very different. Laurel's almost certainly using a mixture of automation and perhaps simple ML models*. I see no evidence that she's using generative AI*. I used to hate when people conflated, but I think in mid-2025 (a) that's how most people understand that term "AI" (as a broad label for any kind of technology that reduces labor by processing data) and (b) I think lawmakers are eagerly making laws that construe the term very broadly (and not by mistake; very intentionally).

I am not a lawyer, but for example, both the recent NYC AI law and drafts of the EU AI Act seemed to use a term like "AI" to apply when anything like an algorithm makes a decision in place of a human. That's certainly how LE works in reviewing stories*. I don't know if the EU AI Act would apply to accepting or rejecting stories (and the NYC AI law applies only to employee selection devices; it requires an audit bias), but the kinds of "AI" that Laurel uses to run the site* are definitely the technologies these lawmakers are targeting as "AI."

*ass talking

 

[alextasy]

To clarify, I'm not particularly concerned about my personal ratings. What bothers me is the effect of skewed ratings on all the readers. I believe most readers will naturally swing toward the stories with HOT labels first, which I think is achieved with a 4.5 rounded average, or greater. Spurious or spiteful ratings--which may be entered by people who haven't actually read the story, but simply don't like the title or the tags--will act as a despicable form of censorship, driving the ratings down to mislead potential readers, as if any story that has particular words in it can't possibly be good. Good thing those people weren't around in the time of Leo Tolstoy or James Joyce.

My greatest concern is what happens when a story is initially published. A 1-star applied to a story with a rating of 4.60 after fifty (50) votes will hardly move the rating. With only five (5) votes and the same 4.60 rating, a single 1-star vote will kick the rating down to a 4.00. At that point, it will take a solid stream of eight (8) 5-star votes to lift it back into the HOT range. That assumes only 5-star votes. If any of those are lower votes, perhaps a reasonable 4 (I'm sure some people reserve their 5's for only what they consider the most exceptional stories), more 5-star votes will be needed to restore reader interest.

The bigger problem with the situation is, with a reduced readership, it will take even longer to get the votes needed to bring good stories back to the attention of the majority of readers.

There are lots of things that could be easily be done to help, and even to make it more fair and informative (e.g., using the "Rotten Tomatoes" rating system, as Penny Thompson mentioned in one of the posts above). It's unlikely anything will change. The site is hopelessly addicted to the clicks from the anonymous hordes.

Thank you for the energetic discussions. This is what makes truly free nations great.

=Alextasy

 

[darwin1859]

To clarify, I'm not particularly concerned about my personal ratings. What bothers me is the effect of skewed ratings on all the readers. I believe most readers will naturally swing toward the stories with HOT labels first, which I think is achieved with a 4.5 rounded average, or greater. Spurious or spiteful ratings--which may be entered by people who haven't actually read the story, but simply don't like the title or the tags--will act as a despicable form of censorship, driving the ratings down to mislead potential readers, as if any story that has particular words in it can't possibly be good. Good thing those people weren't around in the time of Leo Tolstoy or James Joyce.

My greatest concern is what happens when a story is initially published. A 1-star applied to a story with a rating of 4.60 after fifty (50) votes will hardly move the rating. With only five (5) votes and the same 4.60 rating, a single 1-star vote will kick the rating down to a 4.00. At that point, it will take a solid stream of eight (8) 5-star votes to lift it back into the HOT range. That assumes only 5-star votes. If any of those are lower votes, perhaps a reasonable 4 (I'm sure some people reserve their 5's for only what they consider the most exceptional stories), more 5-star votes will be needed to restore reader interest.

The bigger problem with the situation is, with a reduced readership, it will take even longer to get the votes needed to bring good stories back to the attention of the majority of readers.

There are lots of things that could be easily be done to help, and even to make it more fair and informative (e.g., using the "Rotten Tomatoes" rating system, as Penny Thompson mentioned in one of the posts above). It's unlikely anything will change. The site is hopelessly addicted to the clicks from the anonymous hordes.

Thank you for the energetic discussions. This is what makes truly free nations great.

=Alextasy

Are you really concerned about the ratings per se?

Or are you concerned about authors connecting with "their readers" (meaning the ones that would rate the author's story highly)?

Because it seems like that's more a matter of better search or suggestion algorithms rather than fixing the ratings system (that many people seem to believe is not broken).

 

[alextasy]

The ratings are the primary, built-in search algorithm. That big, red HOT is designed to catch people's eyes and pull them to specific stories that others agree are worthy of their time. It requires no action on the part of the reader, and from everything I hear from other writers and readers, it works.

Except the underlying algorithm for defining what is HOT is broken. At a critical stage in the process, it allows even a single low rating to distort the outcome, and the resulting lower reader count makes it more difficult to make up lost ground.

 

[darwin1859]

The ratings are the primary, built-in search algorithm. That big, red HOT is designed to catch people's eyes and pull them to specific stories that others agree are worthy of their time. It requires no action on the part of the reader, and from everything I hear from other writers and readers, it works.

Are they? I'm willing to believe that's your reality, but I look at the new stories pages in the categories I like, do searches for topics I'm interested in, watch the LFS for stories that sound interesting. Ratings never enter into it for me. I've read many comments about the importance of tagging your stories, which makes me think that many people think tag searches are a common way to find stories. I'm sure that's true for me.

Except the underlying algorithm for defining what is HOT is broken. At a critical stage in the process, it allows even a single low rating to distort the outcome, and the resulting lower reader count makes it more difficult to make up lost ground.

Well, you see it as a single low unfair/erroneous/malicious rating as "distorting" the outcome. How can you be sure that it's not just a single person who legitimately thinks the story is a 1?

I was going to suggest that this is just regression to the mean (I'm sure that's a factor), but I did some calculations and there are documented stories (not yours) where the rating fell outside the confidence interval. For example, dropping from mean=4.97 with N=105 to mean=4.84 with N=130. That's still "Hot" but the author was pissed. To analyze that, you have to estimate/impute the SD. I tried different ways, but no 95% CI calculation for 4.97 includes 4.84. The p-values for a t-test ranged from p < 0.00001 to p < 0.03, depending on the way SD was estimated. So, as you say, some people came along and gave low ratings. If they were 1's, then probably just 4-5 out of 25. That probably indicates a change in the population of readers (from a population that was basically 100% 5's to one that was ~80% 5's). After 235 votes, the story's at 4.87.

So one explanation is (a) 105 "saints" then (b) 25 "saints+trolls" (in a 80/20% mix) then (c) 102 "saints+trolls" (in roughly a 80/20% mix). The author thought it was trolls. This also seems to be your point.

But another possibility that seems more parsimonious to me is that the first 100 voters were draw to the story, liked it, probably wanted the author to keep writing. They gave almost all 5's. But then a broader population of readers found the story (possibly through a different mechanism, like a search--but I have no evidence of this) and they were more mixed in their reactions. So in this explanation, 4.97 was the anomaly, 4.8x is the correct rating.

So, I don't see this as "the rating system is broken and must be fixed so trolls cannot scuttle a good story!" but that it's more of a problem of connecting with your readership (the ones who think the story is a solid 5).

 

[THBGato]

What might be nice is if readers could make and share lists of their favorite works, similar to playlists on some of our favorite video sites.

Oh, you can. Have a look at my profile if you want to see what this looks like (links in signature).

For example, dropping from mean=4.97 with N=105 to mean=4.84 with N=130. That's still "Hot" but the author was pissed.

Ha ha! That was me! I'm over it now
I was pissed less because the score dropped, more because it got knocked out of the top list completely (it was number 1). Being in the top list - especially the top twenty - is one of the ways your story stays on the category homepage, which helps attract more readers. So it's helpful.

That kind of downvoting has since happened to several more of my stories, so I'm kind of used to it now. Haters gonna hate: what can you do?

 

[THBGato]

Sorry, also, I think there's an expectation effect. I think some readers click on a story ranked 4.6, read it and go "wow! That was amazing. Why the lowish score? 5 stars!"

Whereas, if the story is (like mine was) 4.97 and number 1 in category, I suspect sone readers open it going "hmmm, this had better live up to the hype". Then once they finish, they think "yeah, that was pretty good, but it wasn't as good as story X or story Y. I don't see why this is number 1. I'll give it a 4."

Now, I've never done this myself, but I can kind of understand the mindset...

 

[darwin1859]

Sorry, also, I think there's an expectation effect. I think some readers click on a story ranked 4.6, read it and go "wow! That was amazing. Why the lowish score? 5 stars!"

Whereas, if the story is (like mine was) 4.97 and number 1 in category, I suspect sone readers open it going "hmmm, this had better live up to the hype". Then once they finish, they think "yeah, that was pretty good, but it wasn't as good as story X or story Y. I don't see why this is number 1. I'll give it a 4."

Now, I've never done this myself, but I can kind of understand the mindset...

There are probably a lot of things going on, but people's tastes are so varied. I think OP had a point, it would be nice to find stories that were of interest to me with a lot more granularity than "Lesbian" or "Group Sex" or "R/NC" (which includes all manner of power dynamics). In your case, there was clearly a population who thought your story as a 5.0 and another population that thought it was mostly a 5.0 but not as close. I could easily imagine much more diverse reactions.

I think tags help, but they're so hit or miss. I was toying with an idea for a freeuse story and I checked the tags to see how I should spell that word. I thought "freeuse" but tag "free use" is used 3x more often.

 

[TheArsonist]

My greatest concern is what happens when a story is initially published. A 1-star applied to a story with a rating of 4.60 after fifty (50) votes will hardly move the rating. With only five (5) votes and the same 4.60 rating, a single 1-star vote will kick the rating down to a 4.00. At that point, it will take a solid stream of eight (8) 5-star votes to lift it back into the HOT range. That assumes only 5-star votes. If any of those are lower votes, perhaps a reasonable 4 (I'm sure some people reserve their 5's for only what they consider the most exceptional stories), more 5-star votes will be needed to restore reader interest.

This could be solved more easily by showing the number of votes (or, as @Biglibido99 suggested, even a breakdown of the vote distribution, as on Amazon/Goodreads). People are used to taking that into account.

 

[THBGato]

This could be solved more easily by showing the number of votes (or, as @Biglibido99 suggested, even a breakdown of the vote distribution, as on Amazon/Goodreads). People are used to taking that into account.

It shows this on the Hall of Fame for each category.

 

[dirk2024]

I rarely look at the overall new story list. I follow a bunch of authors I've enjoyed in the past. (although lately I've started culling some of them) I see plenty of new stories from those who I follow, specifically. I click on their story and look at various things, its usually too soon to rely on the rating as too few readers have seen it yet. So I look at the tags. If there are any comments I look at those. I look to see how long the story is because, depending on my mood or available time, that can impact my decision on whether to read it or not.

When I do want something else I will usually go to the "Tag" page and search for a specific topic that I find intriging in the moment. Then I depend on story title and the little teaser. If I make it past that, then I do what I wrote above...tags...comments...length.

And then there are the various threads here where authors either advertise their own stories or other people post their reviews of stories they like. That is a good source of good reading as well.