ATTN Lit Citizens: A Suggestion - Just for a Few Days
- Thread starter lavender
- Start date
Dillinger
Guerrilla Ontologist
- Joined
- Sep 19, 2000
- Posts
- 26,152
I think that's a great idea lavy...
And here's something else that might help - if we voluntarily turned off our AV's for a while that might free things up a bit more and improve the server response as well...
But - let's at least keep those LONG threads quiescent for a while...
I'll start - going to turn my AV off now... (actually KM started and turned her AV off a while back, but for other reasons...)
And here's something else that might help - if we voluntarily turned off our AV's for a while that might free things up a bit more and improve the server response as well...
But - let's at least keep those LONG threads quiescent for a while...
I'll start - going to turn my AV off now... (actually KM started and turned her AV off a while back, but for other reasons...)
Tim1
Famous Comic
- Joined
- Nov 25, 2001
- Posts
- 1,929
lemme ask
Mine has been slow for a while...but lemme ask, is it due to too much traffic or the fact that their server is too slow and over loaded? Maybe a combo of both? I wonder if our hosts have plans to improve, if in fact it has to do with their server....
Just a dummy wondering....
Mine has been slow for a while...but lemme ask, is it due to too much traffic or the fact that their server is too slow and over loaded? Maybe a combo of both? I wonder if our hosts have plans to improve, if in fact it has to do with their server....
Just a dummy wondering....
modest mouse
Meating People is Easy
- Joined
- Oct 21, 2001
- Posts
- 8,363
Each page is accessed, whether its a new thread or an all time one. It makes no difference. One page is one page.
Removing your own av isnt as beneficial to you as simply turning off your ability to see avatars. Each page you load will not 'hit' the server for everyones avatar and save more bandwidth.
Removing your own av isnt as beneficial to you as simply turning off your ability to see avatars. Each page you load will not 'hit' the server for everyones avatar and save more bandwidth.
CelestialBody
Starlet of India
- Joined
- Sep 30, 2000
- Posts
- 7,904
Cherry
~spankable~
- Joined
- Sep 6, 2000
- Posts
- 5,972
Yes, but MM...
You are thinking from a single viewer's response time
(turning off avs to view)
whereas ...
the first two suggestions would help improve the actual problem of site server/disk contention.
(everyone turning off av's would relieve the contention of finding them on disk to display ... and the postings on large threads would relieve contention on the server of loading, as well)
You are thinking from a single viewer's response time
(turning off avs to view)
whereas ...
the first two suggestions would help improve the actual problem of site server/disk contention.
(everyone turning off av's would relieve the contention of finding them on disk to display ... and the postings on large threads would relieve contention on the server of loading, as well)
Weird Harold
Opinionated Old Fart
- Joined
- Mar 1, 2000
- Posts
- 23,768
modest mouse said:
Actually, a page is however many posts you have set to display.
The problem with the long threads that I suspect of slowing the baords is that to find a specific page, the last post read, or the last post in the thread, the showthread.php script has to read and count each post in a thread. The more posts there are in a thread, the longer it takes to count them off to find a specific post. If you have ten or so people checking to see what the new word in "Something New and Different" is, that's 320,000 posts or so that have tobe read from the server's disks, processed, and counted by the script. That takes a good bit of disk access time and processor time. If there's three or four threads with 1000+ posts active at one time, the servers are spending a lot of time searching and selecting posts to display instead of atually displaying them.
Tht's why a thread with ten posts or so starts loading in about 10 seconds, while "Something New and Different" pauses abut 2.5 minutes before it starts to load. (at least for my 56K connection, those are the times.)
modest mouse
Meating People is Easy
- Joined
- Oct 21, 2001
- Posts
- 8,363
Re: Yes, but MM...
WH,
I wonder if the thread view couldnt be locked onto default viewing as far as number of posts per page. THus locking page numbers rather than post numbers as a guideline. Obviously this would require some change in architecture that I assume isnt available.
I had no idea it worked quite like that, it seems terrible inefficient. Thanks for the explanation.
I also wonder if the all time threads couldnt have the second through 5th from last pages nuked. Thats a dangerous precedent I suppose. Or those wishing to participate in them can simply start a new thread and save their own load times.
Cherry said:
WH,
I wonder if the thread view couldnt be locked onto default viewing as far as number of posts per page. THus locking page numbers rather than post numbers as a guideline. Obviously this would require some change in architecture that I assume isnt available.
I had no idea it worked quite like that, it seems terrible inefficient. Thanks for the explanation.
I also wonder if the all time threads couldnt have the second through 5th from last pages nuked. Thats a dangerous precedent I suppose. Or those wishing to participate in them can simply start a new thread and save their own load times.
Problem Child
titleless
- Joined
- Feb 21, 2001
- Posts
- 27,935
Oh...that's what those hideously long and useless threads are for...using up excess bandwidth. I get it now!
LukkyKnight
Equal Opportunity Enjoyer
- Joined
- Oct 26, 2001
- Posts
- 58,516
considerations
Browsers tend to caches images - and pages - locally to speed up loading times. Avatars, and buttons from the pages, etc., should only be loading once in awhile for each person who visits, not everytime a page is accessed.
Database performance can be a separate issue from server performance, which in turn can be unrelated to actual network throughput. Such issues are notoriously difficult to diagnose. However, files that need to be resized and/or indices which need to be rebuilt can render all the server and network performance in the world a moot point... at least on commercial products such as Oracle and Informix (I presume the same holds true for php, but I don't work with it, I have my hands full with other things. Hi perky ~LOL~) The other obvious issue about plain old file access is good old defragging, just as is done for home computers.
Disk caching can be a serious boost to performance on the server side. So can web server software set to cache certain pages and/or files (i.e. images) which are accessed frequently - moving them to a memory resident status alleviate signigicant I/O bottlenecking. This can frequenly be hugely productive when the database is already spread over an apparently sufficient number of spindles (that number itself is open to debate, of course, but I doubt php treats such concepts as carefully in their install docs as Oracle, for instance.)
So, all in all, I don't think avs are the issue any more than the submit buttons are, they are not huge files by any means. Slow old disks, or database deterioration are more likely culprits, and my money is on both the files being improperly sized and the indexes needing to be rebuilt.
But what do I know? I'm a bartender.
Browsers tend to caches images - and pages - locally to speed up loading times. Avatars, and buttons from the pages, etc., should only be loading once in awhile for each person who visits, not everytime a page is accessed.
Database performance can be a separate issue from server performance, which in turn can be unrelated to actual network throughput. Such issues are notoriously difficult to diagnose. However, files that need to be resized and/or indices which need to be rebuilt can render all the server and network performance in the world a moot point... at least on commercial products such as Oracle and Informix (I presume the same holds true for php, but I don't work with it, I have my hands full with other things. Hi perky ~LOL~) The other obvious issue about plain old file access is good old defragging, just as is done for home computers.
Disk caching can be a serious boost to performance on the server side. So can web server software set to cache certain pages and/or files (i.e. images) which are accessed frequently - moving them to a memory resident status alleviate signigicant I/O bottlenecking. This can frequenly be hugely productive when the database is already spread over an apparently sufficient number of spindles (that number itself is open to debate, of course, but I doubt php treats such concepts as carefully in their install docs as Oracle, for instance.)
So, all in all, I don't think avs are the issue any more than the submit buttons are, they are not huge files by any means. Slow old disks, or database deterioration are more likely culprits, and my money is on both the files being improperly sized and the indexes needing to be rebuilt.
But what do I know? I'm a bartender.
Cherry
~spankable~
- Joined
- Sep 6, 2000
- Posts
- 5,972
You are thinking from a single viewer's response time (turning off avs to view)
This saves all avs loading when you load a page and thus a great deal of bandwidth per pageview
Nope ... it would save the response time for just the person viewing avs ... I do believe when a poster has an av assigned to their post, the Lit software will find it on disk and associate it with the poster's profile regardless if a single viewer wishes to see it or not. The disk contention of finding the av would be alleviated if when the poster's profile does not indicate they have an av to display. It isn't a file space concern as much as disk head assembly contention being overloaded.
This saves all avs loading when you load a page and thus a great deal of bandwidth per pageview
Nope ... it would save the response time for just the person viewing avs ... I do believe when a poster has an av assigned to their post, the Lit software will find it on disk and associate it with the poster's profile regardless if a single viewer wishes to see it or not. The disk contention of finding the av would be alleviated if when the poster's profile does not indicate they have an av to display. It isn't a file space concern as much as disk head assembly contention being overloaded.
modest mouse
Meating People is Easy
- Joined
- Oct 21, 2001
- Posts
- 8,363
Cherry,
Understood that the primary speed boost would be to the person who chooses not to view avs, but wouldnt the lack of transmission of the avs to that person free up bandwidth?
Or are you saying that the av is accessed regardless of viewing?
I do agree with LK that the problem really is the servers and once upgraded/replaced...Poof, back to normal. I think any action we take can only help so much.
Btw, thanks to Cherry, WH, and Lk for the explanations.
LK, I'll have a Scotch, thanks.
Understood that the primary speed boost would be to the person who chooses not to view avs, but wouldnt the lack of transmission of the avs to that person free up bandwidth?
Or are you saying that the av is accessed regardless of viewing?
I do agree with LK that the problem really is the servers and once upgraded/replaced...Poof, back to normal. I think any action we take can only help so much.
Btw, thanks to Cherry, WH, and Lk for the explanations.
LK, I'll have a Scotch, thanks.
Weird Harold
Opinionated Old Fart
- Joined
- Mar 1, 2000
- Posts
- 23,768
Re: Re: Yes, but MM...
I e-mailed Manu and suggested he contact the programmers of the BB software with a couple of suggestions about better algoritms for finding the last read and last posts in threads. That's about the extent of the architectural changes that can be made without a total conversion of the database.
The biggest problem with long threads is that the database is (probably) a plain text file. That means the showthread.php script has to search line by line through the file to find the beginning and end of each post. It's definitely NOT a relational database with fixed length records that can really be searched efficiently. (although a creative programmer can read text files as if they were relational files, it's not a very common practice, and it's not taught as a way to access text files. Each programmer has to reinvent the process.)
You are correct that turning off all avatars reduces server load more than shutting off your own av. However, since the images are cached locally on most people's systems, it's not going to make a big difference in the time to load the pages. Turning them off will help the server load by NOT checking the avatar database at all when a user with them turned off request a page.
It will make composing the page a bit faster, but it will usually make a minimal difference in how long it takes to transmit the page.
Incidently, from the way the BB performs, I suspect the avatar database IS relational database. If it were not, then newbies' avatars would take longer to load than Laurel or Manu's. That doesn't seem to be the case.
modest mouse said:
I e-mailed Manu and suggested he contact the programmers of the BB software with a couple of suggestions about better algoritms for finding the last read and last posts in threads. That's about the extent of the architectural changes that can be made without a total conversion of the database.
The biggest problem with long threads is that the database is (probably) a plain text file. That means the showthread.php script has to search line by line through the file to find the beginning and end of each post. It's definitely NOT a relational database with fixed length records that can really be searched efficiently. (although a creative programmer can read text files as if they were relational files, it's not a very common practice, and it's not taught as a way to access text files. Each programmer has to reinvent the process.)
You are correct that turning off all avatars reduces server load more than shutting off your own av. However, since the images are cached locally on most people's systems, it's not going to make a big difference in the time to load the pages. Turning them off will help the server load by NOT checking the avatar database at all when a user with them turned off request a page.
It will make composing the page a bit faster, but it will usually make a minimal difference in how long it takes to transmit the page.
Incidently, from the way the BB performs, I suspect the avatar database IS relational database. If it were not, then newbies' avatars would take longer to load than Laurel or Manu's. That doesn't seem to be the case.
Dillinger
Guerrilla Ontologist
- Joined
- Sep 19, 2000
- Posts
- 26,152
Actually I know enough about this stuff to think, depending on how this place is programmed, that both suggestions could help. And, it doesn't hurt to try...
And... also I'm with PC on this one... those threads are hideously long and boring and really should be allowed to die in peace...
And... also I'm with PC on this one... those threads are hideously long and boring and really should be allowed to die in peace...
LukkyKnight
Equal Opportunity Enjoyer
- Joined
- Oct 26, 2001
- Posts
- 58,516
I said nothing about upgrading the servers to solve the problem. I thought it was fairly clear I was alluding to configuration issues confined to software (in my admittedly inexpert opinion since I've never so much as loaded php let alone administered a php db.) I've no idea what sort of hardware is involved.modest mouse said:
But here's a scotch all the same.
Svedish_Chef
On Walkabout
- Joined
- Apr 26, 2000
- Posts
- 9,937
Course that sounds all wonderful but.
Nope.
Harold, there is a difference between loadint the threads for you? Maybe only because you dont normally look at them so they arent at least partially cached.
Makes no difference to me what thread I click in.
Nope.
Harold, there is a difference between loadint the threads for you? Maybe only because you dont normally look at them so they arent at least partially cached.
Makes no difference to me what thread I click in.
Weird Harold
Opinionated Old Fart
- Joined
- Mar 1, 2000
- Posts
- 23,768
Cherry said:
First, if I turn off avatar display, then when the software composes a page, it does not check the avatar database. (You can confirm this by turning them off, and timing the lag between a request for a page and when it starts loading, and then repeating the time check with them turned on.)
Second, you're ignoring the cumulative effect of a significant number of people turning avatar display off. If the server save only a few cycles formatting each page for one person, then it saves a hundred cycles if a hundred people turn avs off. If that same hundred people also deleted their avatars, then it saves a hundred cycles when their posts are displayed.
However, as I said above, the difference in the type of database (text file vs relational) makes far more difference to server loads than whether avatars are included or not.
modest mouse
Meating People is Easy
- Joined
- Oct 21, 2001
- Posts
- 8,363
Lk,
You're right, sorry to put words in your mouth. Give me a moment and I'll be slurring my words anyway.
WH,
OK, I think I understand now, at least certainly better than I did before.
***
Maybe Laurel can do a virtual striptease and we can toss dollars towards a new server. C'mon Laurel, shake that ass for a good cause.
You're right, sorry to put words in your mouth. Give me a moment and I'll be slurring my words anyway.
WH,
OK, I think I understand now, at least certainly better than I did before.
***
Maybe Laurel can do a virtual striptease and we can toss dollars towards a new server. C'mon Laurel, shake that ass for a good cause.
Weird Harold
Opinionated Old Fart
- Joined
- Mar 1, 2000
- Posts
- 23,768
Svedish_Chef said:
Nope, caching doesn't seem to make any difference. If a page is cached, the new posts don't show up even though it loads in seconds. The problem with time differences in when I refresh a thread or use the "last post" button to go directly to the last post.
Either the last read button or the last post button force a complete scan of the thread to find the post requested.
Even just refreshing the last page of a long thread causes it tore-read the whole thing for me.
Svedish_Chef
On Walkabout
- Joined
- Apr 26, 2000
- Posts
- 9,937
Harold, I'm only surmising because mine works as quick on any of the threads with the exception of Reply with lyrics only because of the length of the posts.
Maybe its an individual computer thing, because if anything I have more trouble with the new threads themselves.
(Anyway I am honor bound to not let Slutboys thread die)
Maybe its an individual computer thing, because if anything I have more trouble with the new threads themselves.
(Anyway I am honor bound to not let Slutboys thread die)
Cherry
~spankable~
- Joined
- Sep 6, 2000
- Posts
- 5,972
IMHO...
Yes, exactly. The single viewer's selection to turn off all avs would save on bandwidth (less being transmitted); but, if we all removed our avs temporarily, the program would not be directed to the file where avs reside to fetch as soon as it sees the "no" for av use by posters, thus freeing up disk access.
It feels like a disk contention problem with the way it's acting (hanging up downloading images)... less of a bandwidth problem.
Programattically, at the Lit site, each post has associated the poster's profile etc. Regardless, if the information is selected for viewing by a single user, regardless of images held in the viewer's cache ... before it is even transferred for transmission, there is housekeeping done by the program at Lit which would fetch information (avs etc from user profiles) that could be causing file contention on the Lit side. Some of these problems are caused when the files are all on the same disk (file allocation), and magnified by heavy activity (when the files are being accessed continually).
By minimizing the use of avs, it would save on this contention (eliminating the step of finding it to display to all users - before the viewer has the option not to display avs off).
The best temporary solution would be for us to do both: remove the av from our profiles AND turn avs off for display... that would alleviate BOTH the bandwidth and disk contention possibilities.
It might be worthwhile for manu to tweek up the optimization by spreading out frequently accessed files to different disk drives, if that's an option.
modest mouse said:
Yes, exactly. The single viewer's selection to turn off all avs would save on bandwidth (less being transmitted); but, if we all removed our avs temporarily, the program would not be directed to the file where avs reside to fetch as soon as it sees the "no" for av use by posters, thus freeing up disk access.
It feels like a disk contention problem with the way it's acting (hanging up downloading images)... less of a bandwidth problem.
Programattically, at the Lit site, each post has associated the poster's profile etc. Regardless, if the information is selected for viewing by a single user, regardless of images held in the viewer's cache ... before it is even transferred for transmission, there is housekeeping done by the program at Lit which would fetch information (avs etc from user profiles) that could be causing file contention on the Lit side. Some of these problems are caused when the files are all on the same disk (file allocation), and magnified by heavy activity (when the files are being accessed continually).
By minimizing the use of avs, it would save on this contention (eliminating the step of finding it to display to all users - before the viewer has the option not to display avs off).
The best temporary solution would be for us to do both: remove the av from our profiles AND turn avs off for display... that would alleviate BOTH the bandwidth and disk contention possibilities.
It might be worthwhile for manu to tweek up the optimization by spreading out frequently accessed files to different disk drives, if that's an option.
Last edited:
JilloryIsMoist
Experienced
- Joined
- Jan 28, 2002
- Posts
- 51
Oh don't stop, don't stop, I'm about to come.
