The-hairy-one.com speed updates!
It.s about time! i have been trying to make The-hairy-one.com faster, but never managed it. The way it works at the moment is every time someone visits part of my site based around the open directory, it downloads the corresponding page from <a href=http://www.dmoz.org>dmoz.org</a> and phrases it, then displays it with the top bar, side bar and bottom bar, and that.s that. But I have noticed that it.s quite slow. Even when im connecting to it over a 100mb/s LAN. The connection between me and DMoz is not that fast. So I though to my self, instead of downloading the whole DMoz.org site over a matter of days each month (about 20 or so while <a href=http://www.google.com>Google</a> is attacking the site) and then downloading each page over and over again while users visit it, why not download the site to a local machine, currently my workstation during testing, and tell the script to use the local site for the webpage. In testing this seems to work VERY WELL and VERY FAST!!! So, I.m currently downloading the DMoz site fully, I have about 92mb down so far, and 9110 files downloaded, and it.s been just under an hour for that. I don.t know exactly how big it.s going to be, but it says there at least a total of 78000 pages! Hmmmm. I.m going to be a while! Once it.s done, ill copy it to my server, setup a new virtual server, for internal use only, and tell the script to use it. I.m going to have to make a new log for it because if I don.t, there will be 2 users visiting my site all the time and my log analyzer will tell me my sites traffic has doubled! so, I reckon a few more hours and the site will be quite fast. BTW, while writing the total amount of files has gone to 80500 and the downloaded is now 9468!