Aggressive story rating manipulation

[TiefImWesten]

Hi,

Since December the 23rd, all of my high scored stories are pulled down by daily low votes.

This negative trend cannot be explained with statistics, because they were stable over a longer period of time and now going down by 0.01 points or more each and every day.

All the data can be found here: https://***************/spreadsheets/d/1v3Pn2MFXCVwUhLsccCm07q5FeZbfmLYKSeVcgaItsjQ/edit?usp=sharing

Before that date, the person who attacks the german story scores for years was not interested in mine. But since December the 23rd I seem to be on his hate list as well.
It is frustrating that the hard work of month to write good stories is torpedoed by an individual who seems to have fun to pull down all successful stories. Since the most of us do this just for fun, the feedback from our readers is the only reward that we get. And now it this is taken from us too.
This is a problem of the German part of this site for years. There were long discussions who might this individual and how this affects us.

Is there a way to stop this individual?

@Manu and the other admins, I am eager to support you in any way to come to a solution of this problem of the German stories. I don't know if this is a problem for other languages as well.
Finding this individual is probably impossible without having officials tracing down his/hers IP address. It would be even more difficult if a VPN provider is used to access this site. So I am aware that space of real solutions is very small:

• Only logged in users can vote.
• A captcha is added for the not logged in users that must be solved before a vote is counted. I am aware that this is probably a hurdle for many to leave a vote. But I think it could be explained that this is necessary for fraud protection.
• ???

Best regards and to all a happy new year!
TiW

 

[Escierto]

For some people the joy of lit is torpedoing the scores of the stories they hate. Sometimes it’s entire categories!

 

[HectorBidon]

It certainly looks like you are correct that your scores are being attacked in a malicious way. I can't see how many votes your stories have, but since the votes are going down every day, that must mean that at least one and perhaps more low votes are being submitted every day. I think they must come from different usernames, so that means that your troll is fairly sophisticated. On the English side, the site periodically runs "sweeps" that try to look for and remove this kind of activity. I presume they do the same on the German side, so your scores may recover eventually. You can let Laurel know that your scores are being attacked and request a sweep. As I understand it, you will get a more immediate response by sending her a private message (PM, aka "conversation") rather than by posting here.

Good luck.

 

[postpartem]

It certainly looks like you are correct that your scores are being attacked in a malicious way. I can't see how many votes your stories have, but since the votes are going down every day, that must mean that at least one and perhaps more low votes are being submitted every day. I think they must come from different usernames, so that means that your troll is fairly sophisticated. On the English side, the site periodically runs "sweeps" that try to look for and remove this kind of activity. I presume they do the same on the German side, so your scores may recover eventually. You can let Laurel know that your scores are being attacked and request a sweep. As I understand it, you will get a more immediate response by sending her a private message (PM, aka "conversation") rather than by posting here.

Good luck.

Well, unfortunately, this is not the case. It is not a problem that has only recently occurred. This malicious freak is active for more than a decade on the German section of this site. We have approached Laurel and Manu several times over the years, but never got a direct response.

As I have been posting stories here since 2006 on the English side (different handle) and on the German side since 2010 (also with a different username, I withdrew several times not only but also because of this), I can state that the sweeps are sufficient to weed out fraudulent and malicious votes on the English side. Unfortunately, even the weirdos in Germany are systematic and tech savvy, meaning the gentleman likely employs scripts and bots, as well as VPN.

The hate list the thread starter mentions appears to be the toplist at present and is by no means limited to his stories. He was unusually quiet and/or picky for at least a couple of months, during which time we had something like eighteen or nineteen stories on the (general) toplist with a score of 4.90 or above. Within a week he has reduced that to three stories at present (okay, I disabled the voting on mine, which accounts for some of that).

The sweeps also do not catch all of his votes, for reasons unknown. And they are at best a temporary fix, since he simply resumes his "work" once they have completed.

We certainly appreciate that the German section of this site represents only a fraction of the sum total of the stories posted here and it might sound like we are complaining about a well known phenonemon for which there are sufficient measures in place. But that is simply not the case.

We also appreciate that the suggested measures would necessitate some major reprogramming of the current features, but I would surmise that it could be beneficial to the English section as well, especially for contest stories, which as we all know get some special attention in order to provide the most accurate picture to determine the winners.

My ex-wife acutally won one of those contests. Unfortunately, she didn't keep her German and English stories separate, so our special friend had fun with all of her stories. If I remember correctly, almost all of her stories had an "H" at one point. It looks a tad different now:

Munachi

She and many other good authors have left the site, because of the ongoing problem. It is certainly not limited to this site either, but others have addressed the problem and introduced measures as suggested above. We would certainly appreciate, if this could be taken into consideration.

 

[HectorBidon]

We also appreciate that the suggested measures would necessitate some major reprogramming of the current features . . .

This is no doubt the sticking point for the site developers. Also, doing away with anonymous voting would probably greatly reduce voting overall, which would not be viewed as an improvement. While I agree that "one-bombing" is frustrating and even, as you've pointed out, demoralizing, I'm just not sure that Laurel and Manu are enthusiastic to make such big changes.

I wonder if another approach might be to lobby them to improve the "sweep" algorithm. It seems to me that it ought to be possible to detect and sweep away the type of blatant manipulation seen in the OP's spreadsheet without the need to modify the look or feel of the site. This wouldn't solve the whole problem, but it might make a difference.

Here are two approaches that come to mind:

1. Look for statistical anomalies in the voting pattern. A single one-vote can of course be a legitimate vote. But a block of votes that stands out in a statistically significant way from the story's previous voting history can be a sign of malicious activity.

Consider Ein unerwarteter Segeltörn Teil 01. According to the spreadsheet, its score on 12/23/22 was 4.90. To have such a high score, at least 90% of its votes up to that point must have been fives, and no more than 2.5% could have been ones.

The German Top List shows that on 12/31/22 its score had decreased to 4.84 based on 363 votes. One can guess that the 12/23/22 score was based on 357 votes, and that over the next few days 6 one-votes were received in a row. If those votes had followed the same distribution as the ones before 12/23/22, the chances of getting 6 ones in a row would have been less than 1 in 4 billion. That seems like pretty clear evidence of manipulation.

2. Look specifically for signs of one-bombing:
-- an uptick in the number of votes/day above the story's average steady-state rate, [Did you happen to keep track of the number of votes at each date in your spreadsheet? I would suspect that the numbers did not change very much up until 12/23/22, and then increased simultaneously with the decrease in scores.]
-- most or all of the votes during the uptick are ones,
-- similar suspicious activity is seen simultaneously for other stories by the same author or on the same list.

The existing sweep algorithm is already set up to remove suspicious votes, although my understanding is that it focuses on removing multiple votes from the same IP address. A wider-looking approach might be needed to thwart more tech-savvy miscreants.

(Just some thoughts. I didn't read through the discussion your referenced, and so I apologize if these points are redundant or have already been refuted.)

 

[postpartem]

This is no doubt the sticking point for the site developers. Also, doing away with anonymous voting would probably greatly reduce voting overall, which would not be viewed as an improvement. While I agree that "one-bombing" is frustrating and even, as you've pointed out, demoralizing, I'm just not sure that Laurel and Manu are enthusiastic to make such big changes.

Well, as an author you have the option to allow anonymus feedback in general (not for specific stories, but as a general choice in "My options").

What I have suggested previously was to give the same option for anonymus voting. Then the author can decide, whether he prefers unfiltered, but probably a higher number of votes, or accurate ones. That should not be much of a programming challenge, considering the function is already in place for a different element on the page.

The captcha procedure for anonymus comments wasn't always there either, if I remember correctly. It was a reaction to spam-bots and rightly so. Has it reduced the amount of comments significantly? I don't think so. Someone, who would like to make his or her opinion known, either by commenting or choosing to vote, probably doesn't really mind typing a few extra characters to do so. As long as he is not forced to find all pictures with a bicycle in it, I think we're golden.

And Happy New Year!

 

[kimber22]

While I agree that "one-bombing" is frustrating and even, as you've pointed out, demoralizing, I'm just not sure that...

Ehm, could you please quickly explain what "one-bombing" means?

You used the phrase now several times, and I'm not quite sure what it means...

 

[postpartem]

One-bombing is dragging the score down by selecting the one when voting, usually several times to have a significant and immediate impact on the score. That is the option our downvoter uses when he wants quick results. Other authors have reported that he is sometimes doing the same thing with threes or fours, maybe to avoid creating an obvious pattern. It just takes longer, but has the same effect. At least on my stories before I disabled voting it was one-bombing during the last attacks.

 

[TiefImWesten]

@HectorBidon and @postpartem Thanks for your detailed response. I just update the spread sheet with the number of votes column so that it is easier to reason about the number of down votes.

I like the idea that the author can choose who might vote his, her, or their story. And I could imagine that it is easier to implement than to add a captcha. Perhaps this could be added in a second step.
I wrote to Laurel a detailed message and I hope that I get within the next days some feedback.

So far I just can watch the falling ratings .

 

[TiefImWesten]

The weirdo continues his work:


It is sad that there is no reaction by the site admins.

 

[gunhilltrain]

The weirdo continues his work:

View attachment 2200704
It is sad that there is no reaction by the site admins.

The World Wars were sad. What happens on this site is part of the background noise of our civilization. The site administrators: as somebody said in another context, they care about as much as Frank Perdue worried about the oven-roaster chickens he sold.

Your scores are still very good even though they have been down-voted by somebody. Some sites do not allow anonymous voting (by non-members) but this one does. There seems to be some division among Lit members if that would be a good idea or not.

Nice charts by the way. Do work professionally with them?

 

[RoperTrace]

Hi,

Since December the 23rd, all of my high scored stories are pulled down by daily low votes.

This negative trend cannot be explained with statistics, because they were stable over a longer period of time and now going down by 0.01 points or more each and every day.View attachment 2198489

All the data can be found here: https://***************/spreadsheets/d/1v3Pn2MFXCVwUhLsccCm07q5FeZbfmLYKSeVcgaItsjQ/edit?usp=sharing

Before that date, the person who attacks the german story scores for years was not interested in mine. But since December the 23rd I seem to be on his hate list as well.
It is frustrating that the hard work of month to write good stories is torpedoed by an individual who seems to have fun to pull down all successful stories. Since the most of us do this just for fun, the feedback from our readers is the only reward that we get. And now it this is taken from us too.
This is a problem of the German part of this site for years. There were long discussions who might this individual and how this affects us.

Is there a way to stop this individual?

@Manu and the other admins, I am eager to support you in any way to come to a solution of this problem of the German stories. I don't know if this is a problem for other languages as well.
Finding this individual is probably impossible without having officials tracing down his/hers IP address. It would be even more difficult if a VPN provider is used to access this site. So I am aware that space of real solutions is very small:

• Only logged in users can vote.
• A captcha is added for the not logged in users that must be solved before a vote is counted. I am aware that this is probably a hurdle for many to leave a vote. But I think it could be explained that this is necessary for fraud protection.
• ???

Best regards and to all a happy new year!
TiW

. .

 

[TiefImWesten]

The World Wars were sad. What happens on this site is part of the background noise of our civilization. The site administrators: as somebody said in another context, they care about as much as Frank Perdue worried about the oven-roaster chickens he sold.

Your scores are still very good even though they have been down-voted by somebody. Some sites do not allow anonymous voting (by non-members) but this one does. There seems to be some division among Lit members if that would be a good idea or not.

Nice charts by the way. Do work professionally with them?

No, Well, I am working in IT for 30 years. Creating such charts is not that difficult. I copy manually the values out of the csv reports into the spread sheet. I could automate this, but I am too lazy to do it.

 

[jaF0]

So, how many Bratwursts could you buy with those ratings?




Oh yeah ... none.

 

[NoJo]

I'm a 20 year veteran on Lit.com this coming March. The majority of my stories are 15-19 years old on this site. I've had several stories(6?) with the red H, for most of those years. I didn't become active on this forum until roughly two or three years ago. At that point, all of my high ratings(4.5+) were decimated. I now have one left sitting at 4.55 pts. and slowly dropping. Surprisingly, most if not all of my stories that sat below 4.5, have either improved or stayed relatively close to my pre-forum days.
Nice charts, btw! RT

Similar story for me too, although I was active on the forum shortly after posting my first story in 2002. A spate of one-bombs every so often, occasional sweeps. After a while I ignored the fluctuations. Two of my editor's picks stories ended up losing their hotness. A couple of my "sleepers" built up their score over the decades until they had unassailable H's.

 

[CrassiusNomex]

Hi,

Since December the 23rd, all of my high scored stories are pulled down by daily low votes.

This negative trend cannot be explained with statistics, because they were stable over a longer period of time and now going down by 0.01 points or more each and every day.View attachment 2198489

All the data can be found here: https://***************/spreadsheets/d/1v3Pn2MFXCVwUhLsccCm07q5FeZbfmLYKSeVcgaItsjQ/edit?usp=sharing

Before that date, the person who attacks the german story scores for years was not interested in mine. But since December the 23rd I seem to be on his hate list as well.
It is frustrating that the hard work of month to write good stories is torpedoed by an individual who seems to have fun to pull down all successful stories. Since the most of us do this just for fun, the feedback from our readers is the only reward that we get. And now it this is taken from us too.
This is a problem of the German part of this site for years. There were long discussions who might this individual and how this affects us.

Is there a way to stop this individual?

@Manu and the other admins, I am eager to support you in any way to come to a solution of this problem of the German stories. I don't know if this is a problem for other languages as well.
Finding this individual is probably impossible without having officials tracing down his/hers IP address. It would be even more difficult if a VPN provider is used to access this site. So I am aware that space of real solutions is very small:

• Only logged in users can vote.
• A captcha is added for the not logged in users that must be solved before a vote is counted. I am aware that this is probably a hurdle for many to leave a vote. But I think it could be explained that this is necessary for fraud protection.
• ???

Best regards and to all a happy new year!
TiW

I have experienced this myself.

Only allowing logged in users to vote is not really the way to go.

As a developer I can say that it is definitely possible to include an algorithm that will divide the word count of a story by a reasonable maximum read speed to give the time it would take to read a story.

A slightly high estimate of average silent read speed is 300 words per minute. Double that to 600 wpm. Genuine readers sers don't "speed read", they read for pleasure; they're not reading to gather specific information as quickly as possible.

If a rating is given in a time less than the time it takes to read the story at 600wpm then the reader has not read the story and therefore is in no position to give a valid rating.

The script can subtract the timestamp of the request to access the story from the timestamp of the call to the rating function. That would give the time taken to "read". Yes it could have been accessed for later reading, hours/days/weeks later but that means the rating will be valid.

I would think the sad people abusing the rating system access the story, skip to the last page then drop their 1 bomb.

Give them no response other than, "THANK YOU FOR VOTING", but do not include their rating in that author's stats.

DON'T warn them that their rating is invalid because your systems have detected that they haven't read the story.

Keeping a log of excluded ratings might then help your developers see other patterns regarding the abusers.

Best regards,

CN

 

[HectorBidon]

Prompted by TiefImWesten's original post, I've been recording statistics from the German last-12-months top list since January. My data shows that there has been another large voting attack on German stories that started on 03/12/23 and is still in progress.

Fig. 1 shows the ratings of 5 stories that were attacked. They had stable scores up until 03/12, and then, starting abruptly on that date, their scores all began to drop. In all, over 50 stories on the German last-12-months list showed similar behavior. These included almost every story on the list that had over 100 votes on 03/12. In contrast, none of the stories on the list that had less than 100 votes showed this behavior. Some examples are shown in Fig. 2.




The abrupt and seemingly coordinated nature of the downturns strongly suggest that they are the result of a deliberate downvoting attack rather than normal voting behavior. The motivation for the attack was presumably to remove stories from the all-time top lists. Stories only become eligible for the all-time top lists when they have received 100 votes.

There are three stories that have crossed the 100-vote threshold since 3/12. Their scores began to drop off as soon as they reached 100 votes (Fig. 3).


So all this seems to indicate that whoever is responsible launched a major attack aimed at all the new stories on the all-time top lists on 3/12. Since then they have been launching individual attacks on each new story that crosses the 100-vote threshold. (I should say that I haven't been tracking older stories on the top lists to see if they are being downvoted as well. Nor have I been tracking stories in languages other than German.)

 

[RoperTrace]

Prompted by TiefImWesten's original post, I've been recording statistics from the German last-12-months top list since January. My data shows that there has been another large voting attack on German stories that started on 03/12/23 and is still in progress.

Fig. 1 shows the ratings of 5 stories that were attacked. They had stable scores up until 03/12, and then, starting abruptly on that date, their scores all began to drop. In all, over 50 stories on the German last-12-months list showed similar behavior. These included almost every story on the list that had over 100 votes on 03/12. In contrast, none of the stories on the list that had less than 100 votes showed this behavior. Some examples are shown in Fig. 2.

View attachment 2223764
View attachment 2223765

The abrupt and seemingly coordinated nature of the downturns strongly suggest that they are the result of a deliberate downvoting attack rather than normal voting behavior. The motivation for the attack was presumably to remove stories from the all-time top lists. Stories only become eligible for the all-time top lists when they have received 100 votes.

There are three stories that have crossed the 100-vote threshold since 3/12. Their scores began to drop off as soon as they reached 100 votes (Fig. 3).
View attachment 2223766

So all this seems to indicate that whoever is responsible launched a major attack aimed at all the new stories on the all-time top lists on 3/12. Since then they have been launching individual attacks on each new story that crosses the 100-vote threshold. (I should say that I haven't been tracking older stories on the top lists to see if they are being downvoted as well. Nor have I been tracking stories in languages other than German.)

'What-if, Lit.com does aggressive sweeps more frequently on stories with under 100 votes? I don't have any idea if that's true, but just for example, what if: Lit. has an unwritten rule that they sweep low votes out early in a story's timeline, then after the 100-vote threshold, they let the chips fall where they lie? That would encourage new writers, and newly published stories to receive artificially higher, than normal ratings.
I know my joining the forums happened over a decade or longer after publishing my early stories. I did very little arguing(if any) on the forum, but several of my red-H stories got immediately bombed. I cared then but not anymore. Deciding to stop ratings on my last published story, has been like a breath of fresh air, and my next two chapters soon to be published will get the same. I've only ever entered one contest, and thought it was one of my better stories, and comments told me that also. When I saw several stories coming in at 4.9, I stopped shoveling sand against the tide. With almost 2 million views, I have a good idea where my ratings would fall.

 

[dyetied]

Fascinating stuff. Its a dog-eat-dog world out there. Might jealous authors be responsible? Doing down the competition. I suppose it works the other way too. An author might while away some idle hours punching in 5 star ratings for their own work. Now there's a thought...

 

[RoperTrace]

Prompted by TiefImWesten's original post, I've been recording statistics from the German last-12-months top list since January. My data shows that there has been another large voting attack on German stories that started on 03/12/23 and is still in progress.

Fig. 1 shows the ratings of 5 stories that were attacked. They had stable scores up until 03/12, and then, starting abruptly on that date, their scores all began to drop. In all, over 50 stories on the German last-12-months list showed similar behavior. These included almost every story on the list that had over 100 votes on 03/12. In contrast, none of the stories on the list that had less than 100 votes showed this behavior. Some examples are shown in Fig. 2.

View attachment 2223764
View attachment 2223765

The abrupt and seemingly coordinated nature of the downturns strongly suggest that they are the result of a deliberate downvoting attack rather than normal voting behavior. The motivation for the attack was presumably to remove stories from the all-time top lists. Stories only become eligible for the all-time top lists when they have received 100 votes.

There are three stories that have crossed the 100-vote threshold since 3/12. Their scores began to drop off as soon as they reached 100 votes (Fig. 3).
View attachment 2223766

So all this seems to indicate that whoever is responsible launched a major attack aimed at all the new stories on the all-time top lists on 3/12. Since then they have been launching individual attacks on each new story that crosses the 100-vote threshold. (I should say that I haven't been tracking older stories on the top lists to see if they are being downvoted as well. Nor have I been tracking stories in languages other than German.)

Having taken a more critical look at your graphs, these data points for most of the lines both > <100 votes, do not look statistically, random at all. This is from someone who's gathered and plotted data points, then plotted graphically over a 40-yrs. plus career. For example, some lines(story ratings) have 10 or more ratings of 4.85 in-a-row, then shift one or two .01's and another 10 in-a-row. That just doesn't happen in statistics. Now, if there was a function in the system that auto-rounds scores to the nearest .05, or .10 of a point, then it might happen, but 1/100 of a point, not likely.

 

[HectorBidon]

I don't have any idea if that's true, but just for example, what if: Lit. has an unwritten rule that they sweep low votes out early in a story's timeline, then after the 100-vote threshold, they let the chips fall where they lie?

I'm pretty sure this is not the case. The site doesn't just automatically sweep away low votes. 1-star votes are legitimate votes, as long as they are honestly cast. Several authors who closely track their stories' voting histories have reported receiving 1-star votes. (When the vote count is low enough, you can work out the precise value of each new vote that comes in.)

We don't know exactly how the sweeps are done, but my understanding is that the site tries to identify nefarious repeated votes based on IP addresses, not just vote values.

The voting histories of most of the stories I examined were like the one shown in this post. A lot of people vote in the first week or so after the story is published, then votes come in at a slower rate. The score usually stabilizes fairly quickly. In this case there was small decrement in the vote count on 1/23 that could have been due to a sweep, but there is no indication that there was a continual influx of low votes that were being continually swept away. In this case the dramatic dropoff in the rating didn't start until several weeks after the vote count reached 100. It began on a particular date, 03/12, which was the same date on which the scores of 50 other stories also began to fall. This seems much more likely to be the result of a bad actor than of a site policy.

Having taken a more critical look at your graphs, these data points for most of the lines both > <100 votes, do not look statistically, random at all. This is from someone who's gathered and plotted data points, then plotted graphically over a 40-yrs. plus career. For example, some lines(story ratings) have 10 or more ratings of 4.85 in-a-row, then shift one or two .01's and another 10 in-a-row. That just doesn't happen in statistics. Now, if there was a function in the system that auto-rounds scores to the nearest .05, or .10 of a point, then it might happen, but 1/100 of a point, not likely.

The plots show the ratings reported by the site on successive days. If no new votes came in for a particular story for ten days in a row, then its rating will have remained the same. The site gives the ratings to two significant digits, so when there is a change, it will always be by a multiple of 0.01.

 

[Sunny3429]

Only allowing logged in users to vote is not really the way to go.

As a developer I can say that it is definitely possible to include an algorithm that will divide the word count of a story by a reasonable maximum read speed to give the time it would take to read a story.

Honestly I don't think there is any way to prohibit a story rating manipulation script, if we assume that the attacker has some basic programming skills. Mimicking a human LIT reader's behaviour in code wouldn't be more than a couple of hours of work.

LIT stories are quite short and can be read in say below 30 minutes, and it doesn't take many low votes to kick a story off a toplist, so any statistical approach to counteract is doomed. Every programming language has "sleep()" and "random()" functions. One can use a VPN to change his IP address at will. IMHO whatever algorithm you come up, it would be extremely easy for an attacker to counteract.

It would be of some interest, however, for an author do see the voting habits of someone who downvoted his story, that's why I advocate that votes are allowed for registered users only, and that the voting history of an account should be made public.

Sunny.

(Who suspects his stories got voted off the top-lists at a regular pace, but ... so what?)

 

[TiefImWesten]

Whoever is doing these attacks has not stopped. Here are the graphs.
The data behind the graph with the votes and the number of votes per story is here:
docs dot google dot com/spreadsheets/d/1v3Pn2MFXCVwUhLsccCm07q5FeZbfmLYKSeVcgaItsjQ/edit?usp=sharing

Obvious is the big cleaning action around the Valentines story contest, when suddenly the scores went up.

 

[PastMaster]

Where you you get this data from?

 

[RejectReality]

It's every toplist. Always has been. Always will be. There's nothing you can do to prevent it that wouldn't aversely affect normal user experience. You can keep playing whack-a-mole, making it harder and harder to manipulate, but the bad actors will always find a workaround. Meanwhile, the average person will get frustrated with all the hoops they have to jump through, and will just quit voting. The reduced vote pool then makes every vote cast by an asshole to drag down the score count for more.

Eventually — years down the road — stories that can stand the test of time pick up enough new 5s to consistently stay longer on the first page between sweeps, gaining ever larger cushions, and staying up there longer. If the story is strong enough, it will eventually require extraordinary effort to knock it down, and that inevitably leaves traces that lead to more malicious votes getting swept.

Until then, anything that makes the toplist is going to be on a roller coaster. It's just the nature of things.