Weird

[GroverLang]

I’ve got a story that came of the first page several weeks ago with 4.7. A score with which I am quite happy. A few days ago it received a 5 which moved it to 4.71. That was nice. I’ve just looked again and the rating is back to the previous rating and number of votes. The 5 wasn’t, by the way, me voting for my own story!

This isn’t a complaint just an observation because it seems so strange.

 

[AlinaX]

These things happen.

 

[ElectricBlue]

I call them "not so helpful fives" - those fives that the sweep algorithm detects as spurious. They get stripped from the vote count and the maths get adjusted back to what it was.

 

[TarnishedPenny]

The only way for vote numbers to drop is for the site to remove votes (be they 1s or 5s or anything in between). Sweeps are done to deal with trolls and also with, shall we say, overenthusiastic supporters. The mechanisms are secret so as not to give clues as to how to avoid them.

As AlinaX notes, get used to it. FWIW, it happened to me this morning.

 

[nerfvibrator]

Part of the fluctuations are due to what's known as using a "distributed database".

It's not practical to try to have ONE database handle all of the network traffic.

Instead of everyone having access to the "master" database, there are several copies of that database available on various clouds around the world. If you are routed to a cloud which hasn't been synced with the master recently, you get one set of numbers. If you come back later and get routed to a different cloud, you get a different set of numbers.

The so-called sweeps are a result of the "master" gathering all of the activity from the children copies, removing duplicate votes from same IP addresses, and updating the master.

For things like comments, they are directed immediately to the master database.

At per-determined intervals, a shadow copy of the database is taken offline and updated, so there is a lag between what each child will show. I've even seen the "new stories" page lag when using multiple devices (phone and notebook) connected simultaneous.

There's your geek moment for today.

 

[NotWise]

Part of the fluctuations are due to what's known as using a "distributed database".

It's not practical to try to have ONE database handle all of the network traffic.

Instead of everyone having access to the "master" database, there are several copies of that database available on various clouds around the world. If you are routed to a cloud which hasn't been synced with the master recently, you get one set of numbers. If you come back later and get routed to a different cloud, you get a different set of numbers.

The so-called sweeps are a result of the "master" gathering all of the activity from the children copies, removing duplicate votes from same IP addresses, and updating the master.

For things like comments, they are directed immediately to the master database.

At per-determined intervals, a shadow copy of the database is taken offline and updated, so there is a lag between what each child will show. I've even seen the "new stories" page lag when using multiple devices (phone and notebook) connected simultaneous.

There's your geek moment for today.

This would explain some of the variations I see, but the site does systematically remove illegitimate votes. So when we see a change, we're left to wonder whether it was a sweep or a database adjustment.

 

[SimonDoom]

Part of the fluctuations are due to what's known as using a "distributed database".

It's not practical to try to have ONE database handle all of the network traffic.

Instead of everyone having access to the "master" database, there are several copies of that database available on various clouds around the world. If you are routed to a cloud which hasn't been synced with the master recently, you get one set of numbers. If you come back later and get routed to a different cloud, you get a different set of numbers.

The so-called sweeps are a result of the "master" gathering all of the activity from the children copies, removing duplicate votes from same IP addresses, and updating the master.

For things like comments, they are directed immediately to the master database.

At per-determined intervals, a shadow copy of the database is taken offline and updated, so there is a lag between what each child will show. I've even seen the "new stories" page lag when using multiple devices (phone and notebook) connected simultaneous.

There's your geek moment for today.

Does the existence of these distributed databases explain why the numbers for one's stories on one's control panel can be wildly different from the numbers on various toplists? I assume they must be different databases, else why would the discrepancies exist?

I've seen differences of as much as thousands of views, for intstance, for a story that was on my control panel and on a 12-month most-viewed toplist.

 

[ElectricBlue]

Does the existence of these distributed databases explain why the numbers for one's stories on one's control panel can be wildly different from the numbers on various toplists? I assume they must be different databases, else why would the discrepancies exist?

I've seen differences of as much as thousands of views, for intstance, for a story that was on my control panel and on a 12-month most-viewed toplist.

Snapshots in time, maybe, with very long latencies? I know that different site locations refresh at different rates, and there are always anomalies because of that - but big count differences as you mention, that is very strange. I pay no attention to top lists, so I've not noticed that. Most odd.

 

[GroverLang]

Part of the fluctuations are due to what's known as using a "distributed database".

It's not practical to try to have ONE database handle all of the network traffic.

Instead of everyone having access to the "master" database, there are several copies of that database available on various clouds around the world. If you are routed to a cloud which hasn't been synced with the master recently, you get one set of numbers. If you come back later and get routed to a different cloud, you get a different set of numbers.

The so-called sweeps are a result of the "master" gathering all of the activity from the children copies, removing duplicate votes from same IP addresses, and updating the master.

For things like comments, they are directed immediately to the master database.

At per-determined intervals, a shadow copy of the database is taken offline and updated, so there is a lag between what each child will show. I've even seen the "new stories" page lag when using multiple devices (phone and notebook) connected simultaneous.

There's your geek moment for today.

Thanks very much, nerfvibrator. I didn’t understand a word but that will be down to me being a tech dummy rather than your explanation.

Thank you AlinaX, Electricblue66, tarnishedpenny for confirming what I already know. It was, as I said, merely an observation on my point not a complaint.

Just something for possible discussion about weird things, although perhaps not the same, but which seemed out of the ordinary and not consistent with the norm. I used to enjoy the X-Files so I’m thing the truth is out there.

Incidentally, has anyone written a story (one that’s worth reading, about Mulder and Scully?

 

[Salacious_Scribe]

Incidentally, has anyone written a story (one that’s worth reading, about Mulder and Scully?

Search in the fanfic category, it would surprise me if there weren't any posted.

 

[SyleusSnow]

Snapshots in time, maybe, with very long latencies? I know that different site locations refresh at different rates, and there are always anomalies because of that - but big count differences as you mention, that is very strange. I pay no attention to top lists, so I've not noticed that. Most odd.

The top lists seem to be updated in batch, and independently for "All time", "12 months" and "30 days." I noticed this when the Mature toplist 30 and 12 month sections hadn't updated for 2 weeks, while the All Time in the category was updating every day or so.

Lit's backend must be a veeery interesting collection of legacy code.

 

[SimonDoom]

Thanks very much, nerfvibrator. I didn’t understand a word but that will be down to me being a tech dummy rather than your explanation.

Thank you AlinaX, Electricblue66, tarnishedpenny for confirming what I already know. It was, as I said, merely an observation on my point not a complaint.

Just something for possible discussion about weird things, although perhaps not the same, but which seemed out of the ordinary and not consistent with the norm. I used to enjoy the X-Files so I’m thing the truth is out there.

Incidentally, has anyone written a story (one that’s worth reading, about Mulder and Scully?

My search for "mulder and scully" in the Celebrity and Fanfiction category yielded 21 stories. I haven't read any of them.

 

[Kumquatqueen]

Incidentally, has anyone written a story (one that’s worth reading, about Mulder and Scully?

Try looking on AO3 (Archive of our own). Loads of X-files, possibly more Mulder/Skinner but every combo exists. Filtering for over 1500 words and 50 kudos tends to remove the dross, especially if in reverse date order. If there's bad punctuation or run-on commas in the couple sentences of teaser, it won't get better.

 

[Voboy]

Thanks very much, nerfvibrator. I didn’t understand a word but that will be down to me being a tech dummy rather than your explanation.

He brings up an interesting philosophical point, though.

In a distributed-server universe, can we ever really know our current score? Or is it forever a moving target, like Schroedinger's Cat? And if it is, then is it worth worrying about the tiny little fluctuations?

Obviously, any time we look at our score, it's just a snapshot. But a snapshot of what? From where? More to the point, from WHEN?

If you think about this long enough, it becomes less like X-Files and more like Star Trek.

 

[TxRad]

He brings up an interesting philosophical point, though.

In a distributed-server universe, can we ever really know our current score? Or is it forever a moving target, like Schroedinger's Cat? And if it is, then is it worth worrying about the tiny little fluctuations?

Obviously, any time we look at our score, it's just a snapshot. But a snapshot of what? From where? More to the point, from WHEN?

If you think about this long enough, it becomes less like X-Files and more like Star Trek.

Which leads to paranoia and conspiracy therapies.

 

[jsmiam]

Alternative geeks will have much different explanations.

The description given had elements of truth to it in very high volume settings in the year 1986 (ok, eighties, nineties) perhaps.

Today’s issue, alternate geeks say stick with sweeps of fake votes. Just sayin’

 

[GroverLang]

Another observation to promote discussion or otherwise. I can see the logic of an algorithm considering a 5 or 1 as suspicious but would it consider a 4 or a 2 as a spurious vote?

Would anyone wishing to assist a writer’s rating vote 4 as it would reduce the rating not increase it?

 

[KeithD]

Another observation to promote discussion or otherwise. I can see the logic of an algorithm considering a 5 or 1 as suspicious but would it consider a 4 or a 2 as a spurious vote?

Would anyone wishing to assist a writer’s rating vote 4 as it would reduce the rating not increase it?

The system has apparently caught on to the "2" as being as suspicious as a "1." Not a "4," though, and many of the authors here do seem to take a "4" as a down vote because of where the "hot" is positioned. I would think the most successful "knock it down" vote would be a "3."

 

[ElectricBlue]

Another observation to promote discussion or otherwise. I can see the logic of an algorithm considering a 5 or 1 as suspicious but would it consider a 4 or a 2 as a spurious vote?

Would anyone wishing to assist a writer’s rating vote 4 as it would reduce the rating not increase it?

If it met the parameters of a sweep, whatever they are (don't speculate), I don't see why not. I've seen a trolling tendency towards early two-bombs, but I don't know if they stick around or not.

 

[LexxRuthless]

If it met the parameters of a sweep, whatever they are (don't speculate), I don't see why not. I've seen a trolling tendency towards early two-bombs, but I don't know if they stick around or not.

There have been days recently that I have been on when the new stories roll out on I/T and almost every one of them has a single vote -- a 3 or a 2. My best guess is that this is happening because, despite Laurel cautioning us not to talk about it, some authors keep discussing specific aspects of sweeps on this board almost daily. The dedicated site trolls read the posts here and adjust their strategy accordingly. As a long-time reader on the site, I did not even know there was such a thing as sweeps until I started posting stories and hanging out on this board.

Now I know way too much about how it works, and that is entirely from following the discussions on this board.

What's the first rule of Fight Club? Anyone?

 

[GroverLang]

Despite Laurel cautioning us not to talk about it, some authors keep discussing specific aspects of sweeps on this board almost daily.

What's the first rule of Fight Club? Anyone?

I must not have been looking on the occasions when Laurel has “cautioned us” about discussing sweeps or perhaps I’ve missed the email. I’ve never seen Fight Club.

What’s the point of having a forum if there are restrictions on what can be discussed? I don’t believe trolls will go away just because sweeps aren’t discussed. It’s human nature. The trolls are the ones who believe there is nothing wrong in “real life” in making life miserable for others just because they can. Perhaps they are a bastard of a boss, male or female, who the staff talk about. Stopping the staff talking won’t make the boss change their ways.

 

[ElectricBlue]

I must not have been looking on the occasions when Laurel has “cautioned us” about discussing sweeps or perhaps I’ve missed the email. I’ve never seen Fight Club.

What’s the point of having a forum if there are restrictions on what can be discussed? I don’t believe trolls will go away just because sweeps aren’t discussed.

You'll note, though, that those who have figured out how sweeps probably work (and I suspect there are several) don't let on.

 

[NotWise]

What’s the point of having a forum if there are restrictions on what can be discussed? I don’t believe trolls will go away just because sweeps aren’t discussed. It’s human nature. The trolls are the ones who believe there is nothing wrong in “real life” in making life miserable for others just because they can. Perhaps they are a bastard of a boss, male or female, who the staff talk about. Stopping the staff talking won’t make the boss change their ways.

If the method gets out, then the trolls will use that information to get around the sweeps. They'll become useless. That's the point to not discussing it.

 

[LupusDei]

If the method gets out, then the trolls will use that information to get around the sweeps. They'll become useless. That's the point to not discussing it.

Well, the counterargument could be, as it's said, obscurity is poor security. However, in case of story trolls and one-two bombers, we probably deal with relatively simpletons not skilled and resourceful network warriors, so obscurity does have some sense as part of the strategy. Ideally the fraudulent votes would be detected regardless of the mechanism of sweeps being published, but while that could, in theory be archived for most automated adversaries the problematic individuals likely have the tested resource in abundance. There's just so many ways assumptions about behavior could be made from limited sets of stored data.

 

[LupusDei]

Another observation to promote discussion or otherwise. I can see the logic of an algorithm considering a 5 or 1 as suspicious but would it consider a 4 or a 2 as a spurious vote?

Would anyone wishing to assist a writer’s rating vote 4 as it would reduce the rating not increase it?

To keep it simple, if I will enter direct link to the last page of the story and trigger the voting script within seconds, that vote will likely be removed regardless of its value. What exact additional steps are required for the vote to stick is dependent of how artful code interprets scarcely saved data captured with each vote.

Giving hints about the minimal set would be possibly counterproductive for the algorithm's accuracy, most probably leading to adjustments that would remove more legitimate votes.