Time for a new rating system

[SimonDoom]

It seems to me that if just the shiny red H was done away with, the rating system in place would provide a more realistic gauge of reader opinion. The arbitrary 4.50 bar would disappear, which would probably de-incentivize some of the games people play—especially over time.

Granted, readers would then have to determine their own acceptable target score to use in selecting a work, but it seems that 'acceptable target' might become more flexible over time. Is a 4.45 story really that much worse than an H story? It's the arbitrary 'goalpost' that implies it is. Also, without that shiny H shining like a beacon, it would require much more effort for anyone wanting to move a score either up or down to find their targets...and even when they do their vote didn't change things much .

Thus, taking away that red 'line in the sand' would be a good thing. It's been said many times that many stories under the magic 4.50 are worthy and decent stories...the red H is working against readers finding those. One could argue that the red H is a disservice to both authors and readers.

Plus, it'd be very simple to implement this revision.

This is the one idea I can think of that would be easy to do and that might yield quick, positive results.

I'm pretty sure the existence of the red H deters some voting. It does for me. There are some stories on the borderline that I don't want to give a "5" to, but I also don't want to be the guy who prevents the story from getting the red H by giving it a 4. So I don't vote. If the red H were done away with, scores might drop, just a little, but they might more accurately reflect the differences between good stories and truly superlative stories.

Whatever need there is for a red H system can be filled by replacing it with better story search functionality, so readers can search for stories more effectively by score. Readers can then determine for themselves what threshold they want to apply.

 

[ElectricBlue]

This is the one idea I can think of that would be easy to do and that might yield quick, positive results.

I'm pretty sure the existence of the red H deters some voting. It does for me. There are some stories on the borderline that I don't want to give a "5" to, but I also don't want to be the guy who prevents the story from getting the red H by giving it a 4. So I don't vote. If the red H were done away with, scores might drop, just a little, but they might more accurately reflect the differences between good stories and truly superlative stories.

Whatever need there is for a red H system can be filled by replacing it with better story search functionality, so readers can search for stories more effectively by score. Readers can then determine for themselves what threshold they want to apply.

I'm inclined to agree - I'll often look at a story and think, "Well, it ain't a five but it isn't a four either," and not vote because of the Red H skew.

But with all this chatter about doing away with the Red H flowers, we need to remember we're predominantly writers not readers, and we don't have a clue how readers use the scoring system to find stories, because they rarely pass by here and tell us. As far as I can tell, most of the folk who post here are writers first, readers second.

 

[JamesMiehoff]

I am reasonably happy with the scoring systems as it is, I would just like a little more transparency. I would like to be able to get the actual counts.

This could be on your authors dashboard or somewhere only authors could look at for their own stories.

But that would certainly give me some idea if the 4.1 rating is a bunch of 4s with a couple of 5s and some 3s, or a bunch of 5s and a bunch of 1s.

It shouldn't be hard to do and it certainly shouldn't violate any privacy laws in the EU. I am just talking about making the raw scores available.

Then you can apply your own mental mathematics on the raw data.

James

 

[eroticstoryspinner]

I am reasonably happy with the scoring systems as it is, I would just like a little more transparency. I would like to be able to get the actual counts.

This could be on your authors dashboard or somewhere only authors could look at for their own stories.

But that would certainly give me some idea if the 4.1 rating is a bunch of 4s with a couple of 5s and some 3s, or a bunch of 5s and a bunch of 1s.

It shouldn't be hard to do and it certainly shouldn't violate any privacy laws in the EU. I am just talking about making the raw scores available.

Then you can apply your own mental mathematics on the raw data.

James

I would not mind seeing the raw scores

 

[KeithD]

I would not mind seeing the raw scores

Presume you mean the distribution of ratings. You do see the raw score. It's on the author's story list accessible by the readers. The Web site went to providing that score a few years ago--when I started suggesting that they then just drop the red H system, easy to do, but not entertained as a simple partial fix of misleading vote jockeying.

 

[JamesMiehoff]

Presume you mean the distribution of ratings. You do see the raw score. It's on the author's story list accessible by the readers. The Web site went to providing that score a few years ago--when I started suggesting that they then just drop the red H system, easy to do, but not entertained as a simple partial fix of misleading vote jockeying.

That is the composite score. What I would like to see is the score as in:
4 - 5 Stars
1 - 4 Stars
0 - 3 Stars
0 - 2 Stars
12 - 1 Stars

Which is shown as 2.12 on the reader's page. If you hover over the number (assuming you are using a browser, the smartphone version doesn't seem to offer this). It will give you the number of ratings. If the number is small enough and your math fu is strong enough, you could come up with a reasonable guess as to the counts. Still not as good as the real counts.

James

 

[LupusDei]

But with all this chatter about doing away with the Red H flowers, we need to remember we're predominantly writers not readers, and we don't have a clue how readers use the scoring system to find stories, because they rarely pass by here and tell us. As far as I can tell, most of the folk who post here are writers first, readers second.

As a reader I personally pay minimal attention to the H or the rating in general for that mater. It isn't a decisive data point in the "to read or not to read" decision. Rating can however influence attitude with with I will go into a story sometimes. As in "interesting, but why exactly it hated?" or "probably not quite what I'm after, but lets see. people seem to have loved it" and I may even be more forgiving to lower rated piece, and set higher expectations for the overrated.

Sure, my sample size is too small to be meaningful, but I have noticed that while some stories are indeed truly not so god (to say it mildly), at least some stories in 3.3-4.0 range are better written and more interesting that majority in 4.0-4.7 range seems to be; the low score a result of hate by a certain group or publication in sub-optimal category, or some such technicality. Well, stories with extremely high rating are usually rather good, unless it is result of cult following of an author or long series. And of course, I talk about settled score of older stories not new publications with just a few votes cast, that may not be meaningful at all.

Generally, I do not expect the score to reflect quality of the story as literature. Although some limited correlation may indeed exist, it isn't strong. While popularity of a story may be of some limited interest on its own right, it isn't in any way predictive of my own opinion: I have long ago learned that I often like what others don't, and hate popular stuff.

 

[LupusDei]

That is the composite score. What I would like to see is the score as in:
4 - 5 Stars
1 - 4 Stars
0 - 3 Stars
0 - 2 Stars
12 - 1 Stars

Which is shown as 2.12 on the reader's page. If you hover over the number (assuming you are using a browser, the smartphone version doesn't seem to offer this). It will give you the number of ratings. If the number is small enough and your math fu is strong enough, you could come up with a reasonable guess as to the counts. Still not as good as the real counts.

James

I can only agree, that would be most helpful. While I do believe most stories would display some combination of mostly 5 and 1 effectively rendering the resulting rating meaningless, a rating dominated by 4 and 3 might perhaps more readily indicate something I probably do not want to waste time with.

 

[TheThird]

Hello, new here.
Question...
Are you necessarily sure its just trolls? I am trying to get a patreon off the ground and was at another site doing a search and came across a big flame war between authors concerning downvotes to block potential views/reads and involving patreon.


I dunno what impact the H has here (versus other things) but I have one story in Sci Fi that's like 2.3 K views and is rated around 3.5 and another in MC that's like 4.3 K and did have a H for a few days (I actually feel the Sci Fi story is better but hey, different strokes). I noticed after it went below 4.5 it would get close and then go back down.


Hell, I'm just excited thousands of people read something I wrote, but I could see where folks could get paranoid about it, especially if the H did translate to more views.

 

[ElectricBlue]

Hello, new here.
Question...
Are you necessarily sure its just trolls? I am trying to get a patreon off the ground and was at another site doing a search and came across a big flame war between authors concerning downvotes to block potential views/reads and involving patreon.

That's lumped into what most of us consider trolling behaviour - it's either misguided individuals with some kind of vendetta against humanity, someone targeting specific writers just because, or writer/reader voting blocs and cliques. These days, it's more likely the former two. When I joined five years ago it was right at the end of some unbelievable "writer's wars" - absolutely toxic behaviour, especially around contests when the cliques would emerge from their various sewers. By comparison, the AH is tea and biscuits with The Queen, now.

 

[eroticstoryspinner]

Presume you mean the distribution of ratings. You do see the raw score. It's on the author's story list accessible by the readers. The Web site went to providing that score a few years ago--when I started suggesting that they then just drop the red H system, easy to do, but not entertained as a simple partial fix of misleading vote jockeying.

Yes, what James posted is what I would like to be able to access

 

[eroticstoryspinner]

That is the composite score. What I would like to see is the score as in:
4 - 5 Stars
1 - 4 Stars
0 - 3 Stars
0 - 2 Stars
12 - 1 Stars

Which is shown as 2.12 on the reader's page. If you hover over the number (assuming you are using a browser, the smartphone version doesn't seem to offer this). It will give you the number of ratings. If the number is small enough and your math fu is strong enough, you could come up with a reasonable guess as to the counts. Still not as good as the real counts.

James

That is exactly what I meant. Thank you

 

[eidetic]

Add The Point Distribution

The current voting/commenting system has its faults, but generally meets my needs as both a reader and a writer. I definitely would like to see the option of displaying the point distribution (no. of 5's, no. of 4's, etc.), much like I can see when I go shopping on Amazon or similar. I could figure out for myself if an average score was being skewed by a few low votes.
The issues surrounding anonymous commenting is a separate can of worms. I don't comment anonymously and have therefore been subject to reprisal voting on my stories by people who took exception to my comments. That sort of makes the case for anonymous commenting as a protective device.
On the other hand, there is no way to respond to / report virulent, hateful comments beyond deleting them, and no way that I know of to associate the comment and the commenter's vote.
Oh, well... my opinion would be to leave it largely intact and add the view of the vote distribution.
Cheers!

 

[NotWise]

I can only agree, that would be most helpful. While I do believe most stories would display some combination of mostly 5 and 1 effectively rendering the resulting rating meaningless, a rating dominated by 4 and 3 might perhaps more readily indicate something I probably do not want to waste time with.

I'd like it if the site provided the distribution of votes. In place of that, I track votes on my stories. That works for up to 100+ votes, and usually involves estimates when multiple votes come in between checks.

I don't have any stories in LW, but for the categories I have stories in, only one comes anywhere near being a combination of 1s and 5s.

For my stories, the typical distribution of votes after sweeps can be described in words as "the number of votes in a set is approximately a constant times the number of votes in the next higher set." Where "set" refers to 1 star, 2 star, 3 star, 4 star or 5 star votes.The inverse would also be descriptive; "the number of votes in a set is approximately a constant times the number of votes in the next lower set."

Given that description, a score over three would have a distribution that increased regularly toward 5*, and a score below three would have a distribution the increased regularly toward 1*. A score of three would be a flat distribution of votes (the constant would be 1.)

For instance, if a story has 30 5*votes and the constant is 1/3, then the distribution is likely to be about:

5* 30
4* 10
3* 3
2* 1
1* 0

That gives a score of 4.57. In cases where a score has been altered by 1* or 2* bombing, you can get an idea of how the rest of the reader population voted from the ratio of 5* votes to 4* votes -- but only if you know the distribution.

A big disclaimer is that all my examples have ratings over 3*. Extension to lower scores is a concept only. The pattern could completely break down for lower scores.

There are obviously going to be exceptions, and some categories my be systematic exceptions.

 

[yukonnights]

In my opinion, the most potent aspect of removing the red H is the fact it takes away the goalpost for intentionally down voting to get a story @ 4.49 or lower. A 4.49 rating and a 4.50 rating becomes meaningless chatter. People are smart enough to read these scores and put that into the mental equation along with the other pertinent story details when selecting a new story to read.

Removing the red H would have little to no effect on people down voting based on content/kinks they personally have a hatred for. A Cuckold story, or a Bisexual Male in a LW story will always be poorly received.

It would take a very determined single actor to have much real impact on the rating...or a clan of like minded folks working together against an author or a specific kink.

In regard to Anonymous Comments; I sometime leave a comment as Anonymous. Never because I say something I'm ashamed of, but to not appear to be hogging or stalking an author—especially on a chapter style story. As has been said many times; many readers don't have a Lit account, to take away those comments would not be good. As is also often said, the number of unique readers is what drives the Lit machine, not the number of accounts. It's really only the rare Anonymous Comment that I delete.

 

[Javahead]

In my opinion, the most potent aspect of removing the red H is the fact it takes away the goalpost for intentionally down voting to get a story @ 4.49 or lower. A 4.49 rating and a 4.50 rating becomes meaningless chatter. People are smart enough to read these scores and put that into the mental equation along with the other pertinent story details when selecting a new story to read.

Removing the red H would have little to no effect on people down voting based on content/kinks they personally have a hatred for. A Cuckold story, or a Bisexual Male in a LW story will always be poorly received.

I agree on this. There are a handful of writers who have a well-merited red H on most of their stories. And many more of us who have the red H on a significant fraction. Then there are some writers - including some of my favorites - who have a handful of red H stories but many more at a solid 4.3 or 4.4.

And generally, especially for works longer than a page or two, I've not noticed a significant drop off in quality between a 4.2 and a 4.7 story. Now once the rating drops below 3.9 or so, it's a sure bet that either the writing sucks or it's a *really* unpopular storyline, but even there I've found occasional gems - well-written, but definitely a love it or hate sort of thing.

I'd be quite happy to lose the red H or re-jigger it to require a LOT of votes (say, 200) to get it, display only the red H (and not the overall point rating), and a significant drop (to, say, 4.4 overall) to lose it once acquired. But just eliminating it altogether would be easier, and would eliminate the easiest ways to game the system.

Now if we could update the search engines to include "all stories above X" as one of the advanced options . . .

 

[RejectReality]

That H is a read target for enough readers to make a significant difference. I've had enough stories dip below the bar and then rise back above it on day 1 to see the difference. The views drop dramatically when that H vanishes, and increase dramatically when you have one.

Readers are using it as a major indicator of what to read.

It's an artificial tipping point that's causing undue anxiety for writers, easy pickings for trolls, and doing a disservice to readers who assume it's a mark of quality — to many, the only such mark.

In addition, it can be manipulated in other ways. One major one is that voting can be turned off on a story once it has attained an H, and the H remains. That particular bug has been exploited by a few people that I know of.

Beyond that, there's no such thing as a scoring system that will please even most writers. You get just as many complaints from a 10 point system. You get just as many complaints from a score breakdown. You get just as many complaints from a multifaceted score ( grammar, characters, enjoyment, etc. )

Changing such a long-established scoring system inevitably favors either old stories or new stories. There's no real way to tweak it to balance the old vs. new systems, and attempts to do so just create new ways for people to complain that the system is rigged against them.

Getting rid of the Hs is the one thing that can have any real benefit without introducing new problems.

 

[KeithD]

That H is a read target for enough readers to make a significant difference. I've had enough stories dip below the bar and then rise back above it on day 1 to see the difference. The views drop dramatically when that H vanishes, and increase dramatically when you have one.

Readers are using it as a major indicator of what to read.

The point is that it's misleading. This is a high cheating zone. People are making these suggestions to take some of the fun out of cheating and creating misleading impressions. Not surprised to see you taking a company "let them eat cake" line, though.

 

[jsmiam]

Mathematically, trolls are trolls, and simple upvote/downvote scoring mechanisms are not better, they are in a way, the equivalent of only offering (in comparative lit terms) a score of either 1 or 5. Another site known for it's sex stories but without a space has that, and the score variances are actually worse and more volatile.

In fact, having 1-bombs probably makes it easier to identify review-fraud, for purges/sweeps of trolls.

Keep in mind: Amazon as potentially the worlds biggest site literally has a small army of PhD statisticians working on review fraud there, and it still happens. And there will be review fraud here too. So while continuous improvement is important everywhere, there is no getting away from the possibility of review fraud.

Either way though, doing the math, there is zero inherent superiority of using yes/no, thumbs up/thumbs down mechanisms. When the math is all said and done, it ends up roughly being the equivalent of only allowing votes of 1 or 5.

 

[RejectReality]

Not surprised that you failed to read the post and jumped to a completely wrong conclusion, either. You do that regularly. If you had read anything beyond the two lines you quoted, you would know that I said it is misleading, creates a whole host of other problems, and needs to go.

The point is that it's misleading. This is a high cheating zone. People are making these suggestions to take some of the fun out of cheating and creating misleading impressions. Not surprised to see you taking a company "let them eat cake" line, though.

 

[Bramblethorn]

Not much is going to change on this Web site, but anything involving payment of any sort just isn't going to happen and is DOA even to talk about. This is a Mom and Pop store. Payments explode the need for more manpower and the owners have shown zero interest in doing that.

Yep. The moment you add payments you attract scammers looking to get that money.

People think the site has a problem now with voting fraud? Wait until "authors" have a financial incentive to steal other people's content and use author payments for money laundering.

When proposing changes, think not only "how could this fix the problems we have?" but also "what new problems would it create?" and in particular, "if I was a criminal how could I exploit this?"

 

[rutger5]

Might as well put in my two cents on this thread since it remains active and I think I have some relevant info. Believe Simon said it first that the rating system is for readers primarily and I concur. The thing is that even if there was a better system that would benefit both readers and writers (for the most part, with the possibility of unintended consequences) what would that actually mean? The same way most writers who post stories on Lit aren’t super active on AH (and related sub forums) I’d guess the same holds true of readers to an even greater degree. They are coming to read stories and some of them vote, favorite, and comment but I’m guessing most pass thru like smoke without leaving a mark. They probably don’t care about this subject unless it affects their ability to read so probably most won’t have an opinion on a subject that some writers seem to obsess over.

Most of the stories I’ve posted here were posted to another site before I came here and these types of threads are hardly unique to Lit and neither are some so called solutions. Going from a five star rating system to a ten star system might help in an incremental way where an honest rater might decide to give a story a nine vote where they wouldn’t want to give a perfect ten (new system) or a perfect five (current system). Odds are if this change was implemented some older stories might get short shrift for though the old story scores could be doubled the thought process would be different when the vote was cast and stories that might have now received a nine may have only received a four at the time, now doubled to eight. I wouldn’t rule out changing the rating system due to this but any time things are judged by different systems it’s impossible to reconcile perfectly.

Saw a suggestion along the lines of eliminating the five star system, with the assumption being that most votes cast are either fives or ones. I can say from experience to those that complain about “one bombs” disrupting their story scores, just try a binary voting system. For the most part you need nineteen thumbs ups to counter one thumb down, or if you really have a lot of votes you need the average of at least eighteen and a half votes to balance the vote and a half. The five star system isn’t perfect but nothing is and any system can (and will) be gamed. If someone wants to vote down a story to “help” their favorites, or they don’t like the writer, or they have an issue with the subject matter the rating system used is irrelevant.

Only allowing logged in, registered members to vote or making readers justify their vote through some process beforehand would only discourage voting so to my mind are nonstarters. There is no incentive for Lit to cause readers to disengage from the site and even if this helped some story ratings at what cost? Though I want my stories to have high ratings I’d rather a story receive a lot more votes and other reactions rather than having a higher rating with fewer votes.

Not that our input is probably wanted but I’d go along with the suggestion mentioned in this thread first by KeithD – namely eliminating the red H. I gather that it wasn’t part of the original rating system but grafted on later and though I’m sure with good intentions I’d argue it has had a negative impact. While each story is different I’ll wager that the average story with a rating of 4.49 isn’t any worse than stories rated 4.51 yet who knows how many readers just skip right past the story without the coveted hot rating. Though I try to not concern myself with my story ratings in general I admit that on Lit if I have a story hovering just below 4.50 there is part of me just hoping it cracks that barrier. On a different site that uses a numerical percentage I don’t care if a story is rated 92.9% rather than 93% but here that tiny amount to get an H can mean so much more.

My preference would be to eliminate the hot rating because it does skew things. But once something is introduced and people become accustomed to it to then remove it is often a bridge too far. Some readers use it as shorthand to find stories and writers that have a lot of H’s won’t want that earned advantage to disappear so what would be the upside for Lit to eliminate it? Writers come and go for a variety of reasons so even if some writers decided to leave over a less than perfect system for the most part things would go on. Like much of life we are just spitting into the wind.

 

[TarnishedPenny]

I'll go with a point which has already been made but needs to be restated. The people complaining (and I use that word carefully and not as an insult) about the present rating system are authors, not readers.

Point two, there are comparatively few writers compared to readers. Fewer still hang out here and only some of those think a change is needed. (Maybe it is, maybe it ain't. My point is that those thinking we need change are a tiny minority in the big scheme of things.)

Point three, the only influence we bring is with our stories, which we provide for free. If every author was upset enough about the present system to 'strike', to demand that Laurel either change the voting system 'or else', then change might happen. If on the other hand only those who are upset about the present system went on strike, it wouldn't make the shadow of a ghost of a difference. Yes, writers are what brings viewers here, but the reality is that we do it because we like it, even with all the frustrations the system throws our way.

The bottom line, I think, is that what we want, what we feel, probably doesn't matter very much. When there is some sort of push for change from the readers, the ones whose visits to this site are attracting advertisers for Laurel and Manu, then there might be some incentive for management to make changes. Until then, I'm not going to lose any sleep over it.

 

[Zeb_Carter]

Consider the math. If a story is at 4.50, it takes 7 5-stars votes to offset a one 1-star vote from a pissed off reader.

Unless the it has over 100 votes... then a 1-star is meaningless. And once it goes over 200 votes, 1-stars are even less meaningful.

 

[KeithD]

. . . eliminating the red H. I gather that it wasn’t part of the original rating system but grafted on later and though I’m sure with good intentions I’d argue it has had a negative impact.

The red H was part of the original rating system. What changed is that until a few years ago, the actual rating of a story wasn't provided on the author's list accessible to the reader. Now it is, so the reader can see a story that has a 4.49 rating (no red H) rather than listing the same as something with a 1.25 rating. Without the red H, the reader could clearly see the precise rating and decide the "hot" issue for her/himself rather than being guided to someone else's very restrictive determination that a story is hot/worth reading or not.

Another issue with the red H is that a lot of the malicious zapping I experience is a concerted effort to keep the story below the 4.50 mark. Without the red H element, the malicious zappers don't have an incentive to do that as much. They have to put a lot more malicious voting effort in to make reading the story not worthwhile.