A Modest Proposal

[Carnevil9]

I am thinking about a new statistic for rating stories.

I find that I am equally thrilled by 5-star and 1-star ratings. In both cases, it means that I have gotten a visceral reaction out of a reader. Being a good or bad reaction isn't always the issue; the strength of the reaction sometimes is what counts to me as an author.

So how about a statistic based on an equation something like: SQRT(Sum(score-3)^2)

For those of you who don't speak FORTRAN, what I mean is that every score has 3 subtracted from it, and is then squared (so negative numbers become positive), and then they are all added up, and then we take the square root of the total. In other words, the STRONGER the reactions, positive or negative, the higher the statistic that results. So we are measuring the intensity of the reaction, rather than the good or bad of it. Lots of 3 scores will get you a very low statistic; lots of 5 scores, or 1 scores, or both, will get you a very high statistic.

This is just a thought experiment. I'm not expecting that it will ever happen, of course. Just floating it out there as a talking point. What do you all think?

 

[TheLobster]

"I posted this 20k story 5 minutes ago and it already got a 3-star! Damn bombers."

 

[RocketGrunt]

Right now a lot of bombers deliberately give things 2 stars in order to make it less likely that their negative reviews get pruned. Trolls adapt to the medium in which they are trolling.

 

[sirhugs]

I prefer my proposals immodest

 

[calmlikeabomb]

Counterpoint: why not go for the Median score? That would be a much better representation of the average Lit reader's feelings.

 

[Z_TheWriter]

I am thinking about a new statistic for rating stories.

I find that I am equally thrilled by 5-star and 1-star ratings. In both cases, it means that I have gotten a visceral reaction out of a reader. Being a good or bad reaction isn't always the issue; the strength of the reaction sometimes is what counts to me as an author.

So how about a statistic based on an equation something like: SQRT(Sum(score-3)^2)

For those of you who don't speak FORTRAN, what I mean is that every score has 3 subtracted from it, and is then squared (so negative numbers become positive), and then they are all added up, and then we take the square root of the total. In other words, the STRONGER the reactions, positive or negative, the higher the statistic that results. So we are measuring the intensity of the reaction, rather than the good or bad of it. Lots of 3 scores will get you a very low statistic; lots of 5 scores, or 1 scores, or both, will get you a very high statistic.

This is just a thought experiment. I'm not expecting that it will ever happen, of course. Just floating it out there as a talking point. What do you all think?

It's an interesting idea. I wouldn't take over and in place of the usual score, but it would be quite an interesting number to have. Unfortunately I don't think we can see the individual scores, so even doing it manually for your own stories out of interest is impossible - or have I missed a part of the control panel?

 

[Z_TheWriter]

Counterpoint: why not go for the Median score? That would be a much better representation of the average Lit reader's feelings.

The median score on a story with 200 5-stars and 201 1-stars would be 1. Obviously an extreme example, but you get my point. So while medians are certainly useful in many statistical cases, I'm not sure it applies here that well.

 

[calmlikeabomb]

The median score on a story with 200 5-stars and 201 1-stars would be 1. Obviously an extreme example, but you get my point. So while medians are certainly useful in many statistical cases, I'm not sure it applies here that well.

Fair point, although I'm assuming something like this would be an extreme edge case? I'm not too familiar with voting patterns here to be honest, except for the fact that sometimes people get review bombed.

 

[Actingup]

I am thinking about a new statistic for rating stories.

I find that I am equally thrilled by 5-star and 1-star ratings. In both cases, it means that I have gotten a visceral reaction out of a reader. Being a good or bad reaction isn't always the issue; the strength of the reaction sometimes is what counts to me as an author.

So how about a statistic based on an equation something like: SQRT(Sum(score-3)^2)

For those of you who don't speak FORTRAN, what I mean is that every score has 3 subtracted from it, and is then squared (so negative numbers become positive), and then they are all added up, and then we take the square root of the total. In other words, the STRONGER the reactions, positive or negative, the higher the statistic that results. So we are measuring the intensity of the reaction, rather than the good or bad of it. Lots of 3 scores will get you a very low statistic; lots of 5 scores, or 1 scores, or both, will get you a very high statistic.

This is just a thought experiment. I'm not expecting that it will ever happen, of course. Just floating it out there as a talking point. What do you all think?

Well, it's probably at least as reasonable as the original proposal of the same name.

(https://www.gutenberg.org/files/1080/1080-h/1080-h.htm)

 

[Actingup]

Counterpoint: why not go for the Median score? That would be a much better representation of the average Lit reader's feelings.

I've previously suggested calculating the percentile rank of the story in each category, since the scores between categories can't be directly compared on their raw value.

 

[ElectricBlue]

Fair point, although I'm assuming something like this would be an extreme edge case? I'm not too familiar with voting patterns here to be honest, except for the fact that sometimes people get review bombed.

Generally speaking, I think most readers vote with a 3, 4, or a 5. If a story is worth less than a 3, I reckon most readers don't bother leaving a score at all. There are one-bombs certainly, but I think the sweep process minimises the long term effect of those.

On that premise, scores skew high, chasing the Red H (4.50 or more), which, despite protestation, definitely drags eyes into stories.

 

[Z_TheWriter]

Fair point, although I'm assuming something like this would be an extreme edge case? I'm not too familiar with voting patterns here to be honest, except for the fact that sometimes people get review bombed.

I don't have that familiarity either. My instinct would be that you'd get the most 5s, 4s, and 1s, because those are the people invested enough in their opinion to bother specifying.

It's also a matter of quantity though. Median is a much more useful tool when you have a sample size large enough. The median height of a country is a much more solid number than of a class. The same is true of averages, but they're less prone to be totally derailed by a single coincidence. Since many stories have quite few ratings, you'd easily get some wildly fluctuating numbers due to new votes that day.

(Like the original proposal though, I would welcome the additional information, just not prominently displayed.)

 

[TheLobster]

Counterpoint: why not go for the Median score? That would be a much better representation of the average Lit reader's feelings.

Not enough granularity. It would put most stories at 4, a relatively large percentage (20ish) of top ones at 5, and then a small number of the worst (or LW) at 3.

The median score on a story with 200 5-stars and 201 1-stars would be 1.

It would be 5. Median is the middle value of a sorted set.

I initially misread it as 200 5* and 1 1*.You’re right.

 

[calmlikeabomb]

Well then I await the day we all have chips in our heads and stats display how many times someone masturbated to your story.

 

[Z_TheWriter]

Well then I await the day we all have chips in our heads and stats display how many times someone masturbated to your story.

I feel like there's a horror/sci-fi story plot-bunny in there for someone.

 

[Bramblethorn]

I don't have that familiarity either. My instinct would be that you'd get the most 5s, 4s, and 1s, because those are the people invested enough in their opinion to bother specifying.

It's also a matter of quantity though. Median is a much more useful tool when you have a sample size large enough. The median height of a country is a much more solid number than of a class. The same is true of averages, but they're less prone to be totally derailed by a single coincidence. Since many stories have quite few ratings, you'd easily get some wildly fluctuating numbers due to new votes that day.

(Like the original proposal though, I would welcome the additional information, just not prominently displayed.)

Median is useful when the data you're analysing takes a wide range of values, but it gets volatile with something like Lit data where only a few scores are possible. A story with 200 5s gets the same median as a story with 101 5s and 99 4s (or 99 1s); throw in a couple more low votes and suddenly the score jumps a whole point. Every single story I've posted here would have the same median, with nothing to distinguish them.

 

[iwatchus]

Generally speaking, I think most readers vote with a 3, 4, or a 5. If a story is worth less than a 3, I reckon most readers don't bother leaving a score at all. There are one-bombs certainly, but I think the sweep process minimises the long term effect of those.

On that premise, scores skew high, chasing the Red H (4.50 or more), which, despite protestation, definitely drags eyes into stories.

The sweeps don't seem to touch the 1-bombs that happen when you are on the leader board. Or when a story wins a prize (those were painful).

 

[Lifestyle66]

I am thinking about a new statistic for rating stories.

I find that I am equally thrilled by 5-star and 1-star ratings. In both cases, it means that I have gotten a visceral reaction out of a reader. Being a good or bad reaction isn't always the issue; the strength of the reaction sometimes is what counts to me as an author.

So how about a statistic based on an equation something like: SQRT(Sum(score-3)^2)

For those of you who don't speak FORTRAN, what I mean is that every score has 3 subtracted from it, and is then squared (so negative numbers become positive), and then they are all added up, and then we take the square root of the total. In other words, the STRONGER the reactions, positive or negative, the higher the statistic that results. So we are measuring the intensity of the reaction, rather than the good or bad of it. Lots of 3 scores will get you a very low statistic; lots of 5 scores, or 1 scores, or both, will get you a very high statistic.

This is just a thought experiment. I'm not expecting that it will ever happen, of course. Just floating it out there as a talking point. What do you all think?

I ran that formula on my list of stories.

It takes those currently rated at 4.5 and comes out with 1.5. So, Red-H stories would be between 1.5 to 2.0. It takes a story rated 1.5 and makes it ... 1.5, with a story rated 1 now a 2. So, it effectively inverts the rating curve of those below 3 to mirror those above 3.

The problem with that suggested rating scheme is that it assumes all stories are written with equal quality, and the rating is due solely to ... what? Content? Triggers? Steamy sex? Story arc? Character development?

We rate stories based on different criteria, and any particular story gets a 1-bomb for various reasons. In over a decade of reading stories on this site, I have given a 1 rating only four or five times. And it's due to an author posting an obviously stupid piece of crap, with no story line, no attempt at quality, no character development, ... basically a first draft of something resembling their rambling thoughts.

So, the new rating suggestion would need to include a "zero" stars to exclude the truly shitty stories, rather than confuse them with a Red-H.

 

[iwatchus]

Not enough granularity. It would put most stories at 4, a relatively large percentage (20ish) of top ones at 5, and then a small number of the worst (or LW) at 3.

Actually, I think 5 would probably be the most common median. By definition, any story with a rating over 4.5 has more than half of their votes being 5. I think it is almost half of stories receive H's by stats someone put up here.

 

[iwatchus]

I find that I am equally thrilled by 5-star and 1-star ratings. In both cases, it means that I have gotten a visceral reaction out of a reader. Being a good or bad reaction isn't always the issue; the strength of the reaction sometimes is what counts to me as an author.

I think too many of the 1's are not actually a reflection of the story, but rather the category or kink. Or it is a reflection of the story's mere existence. When I had my stretch of every new story of mine getting a 1 at 7:45 (given or take a few minutes) on the day it was released, I dare to say that particular voter never read any of those stories. Of the bombs that hit any story that dares to be listed on the all-time lists or win a prize.

 

[Writer61]

Right now a lot of bombers deliberately give things 2 stars in order to make it less likely that their negative reviews get pruned. Trolls adapt to the medium in which they are trolling.

I have had a few of those recently.

 

[Writer61]

Counterpoint: why not go for the Median score? That would be a much better representation of the average Lit reader's feelings.

Not really, most readers don't give a rating.

In my case, just over 1% of readers do so.

 

[cw5523729]

Right now a lot of bombers deliberately give things 2 stars in order to make it less likely that their negative reviews get pruned. Trolls adapt to the medium in which they are trolling.

Oh great, now instead of 1-bombers, we are going to hear about the 2-bombers.

Hopefully eventually it'll upgrade to 4-bombers. I have some stories that could use a boost into the 4s.

 

[Z_TheWriter]

I ran that formula on my list of stories.

It takes those currently rated at 4.5 and comes out with 1.5. So, Red-H stories would be between 1.5 to 2.0. It takes a story rated 1.5 and makes it ... 1.5, with a story rated 1 now a 2. So, it effectively inverts the rating curve of those below 3 to mirror those above 3.

The problem with that suggested rating scheme is that it assumes all stories are written with equal quality, and the rating is due solely to ... what? Content? Triggers? Steamy sex? Story arc? Character development?

We rate stories based on different criteria, and any particular story gets a 1-bomb for various reasons. In over a decade of reading stories on this site, I have given a 1 rating only four or five times. And it's due to an author posting an obviously stupid piece of crap, with no story line, no attempt at quality, no character development, ... basically a first draft of something resembling their rambling thoughts.

So, the new rating suggestion would need to include a "zero" stars to exclude the truly shitty stories, rather than confuse them with a Red-H.

The formula is intended as subtracting 3 from each rating and doing the sum, not from the average score. sqrt((R_1-3)^2+(R_2-3)^2+...) if you will. Like I said previously, we don't have the data to do this ourselves.

-----------------------------Nerd shit----------------------------------------
This took waay longer than it should have, because I have a fever, and it may for the same reason be slightly off - I hope not:

A story with 20 x 5 Stars and 3 x 1 Stars will have the average score of 4.478... So 4.48 rounded.
It will have the "Carnevil-score" of 9.591...

A story with 20 x 5 Stars and 7 x 3 Stars will have the average score of 4.481... So 4.48 rounded.
It will have the Carnevil-score of 8.944... Because fewer people gave an extreme rating, theoretically meaning the story elicits fewer strong reactions.

This will also give a story with an average of 3 distributed across 1 x 2 and 1x 4 a Carnevil-score of 1.4. Whereas for 2 x 3, it will be 0.

-----------------------------End of nerd shit--------------------------------

The fact that stories with similar rating spreads (not averages) will be mirrored around 3 was kind of the point, I believe. To gauge intensity of emotion, not positivity of emotion.

Your other points are correct though, we can't know the nuance of the reasons why people give the scores they do. That holds true regardless though - averages are the same.


Oh and if you feel like asking "Z, why all this ?", the answer is that I'm bored because of previously mentioned fever.

Edit: There should, btw, probably be a "/n" somewhere in that formula to normalize for number of ratings, but I'm not lucid enough right now to tell you if it should inside or outside the sqrt()

 

[calmlikeabomb]

Not really, most readers don't give a rating.

In my case, just over 1% of readers do so.

Which tracks with reading in general. Only 1-3% people ever rate and eBook, or books in general, so the audience here is not far off from general population when it comes to ratings.

Of course in non erotic books, certain categories are outliers where rating ratios sometimes go as high as 5% (easy reading business and finance books for instance, because that's an opportunity for the reviewer to show off how that book changed their life)