Speedy Forum Experiment

Jonathan Neal wrote on 2/26/2008, 12:21 PM
I was experimenting with Google Gadgets again and I came up with this web site today. This is very, very unrefined, but it uses Google to download the forums and report the results back to you. Right now I have it configured to monitor all of the forums and display up to 50 results. It will also let you live search all of the forums.

http://madison.thewikies.com/scsf/

Let me know what you think?

Comments

Chienworks wrote on 2/26/2008, 12:42 PM
What kind of load is that going to put on Sony's server?
Jonathan Neal wrote on 2/26/2008, 12:49 PM
The kind of load put on Sony's server should be the load of accessing 50 posts without loading any javascript, css, or images. I see 50 posts when I access any given forum.

I have little idea as to how the forums work on the backend, so I don't know which load would hit or ease the server the most. I can tell you that google is making only one call to the server, and that when you would visit any given list or thread on these forums otherwise, you would probably be making several dozen calls to the server.
johnmeyer wrote on 2/26/2008, 8:11 PM
Interesting idea. I couldn't get the search to work. It just froze.
TheHappyFriar wrote on 2/26/2008, 9:26 PM
What kind of load is that going to put on Sony's server?

if it's like a download manager that leaches, it's 50 downloads at once. If it's "nice" it will do them one at a time.

But I'd bet it's 50x at once. So if 15 of us did that we'd have the request load of 750 users. At once. Not downloading all the stuff 750 users would see (graphics, etc), just 750 users connected.

Isn't that considered some type of website attack too? When a few users flood with requests until it crashes the server?
Jonathan Neal wrote on 2/27/2008, 9:23 AM
johnmeyer, the reason it freezes up during a search is because it is downloading the response from the server, and I wrote the entire process as a single function. I'll split the function up so that you get a nice ajax display that a request is being made.

TheHappyFriar, I monitor the entire request and I make exactly one call to Sony's site. I understand your concern; while download managers like Flashget do that sort of thing, it does not happen with my script. I have no reason to download an HTML page 50 times, and browser http requests don't even support partitioned downloading.

I feel confidant saying that if you were to do the same thing I'm doing on Sony's actual site, you would be consuming many more times their resources.