|
Forums
|
|
FutureMusicCorp
Joined: 10/05/02
Posts: 7
|
I was for me yesterday (saturday).
|
|
|
Subtronik
Joined: 01/14/03
Posts: 1193
|
|
RE: Was the site down?
Sunday, May 2, 2010 at 7:31 AM
|
Quote
Top Bottom
|
Things have been spotty for a few days. What's up Gideon?
|
|
|
Sonic Wallpaper
Joined: 05/22/01
Location: San Diego, CA
Posts: 1266
|
|
RE: Was the site down?
Sunday, May 2, 2010 at 11:06 AM
|
Quote
Top Bottom
|
Hi all,
Most of the performance trouble we have are caused 'scrapers' or 'bots' that are sent to our site and either attempt to download the majority of the music on the server or are attempting to connect to all the mp3s to get the IDS info, and sometimes, it is a bot attack. These processes are done by sending up to far too many requests per second to the server which ends up using all the system resources and locks the server up as it attempts to respond to all the requests.
Once this happens, normal users start hitting reload, and all these requests start getting backed up forming a queue.
Blocking them now has to be a bit of a process because we can't install a software package that we used to run called BlackIce - which would identify attack behavior and add the IP address to a ban list. This software is no longer supported by the developer, and it won't run on the Windows OS we are now on. There are other apps out there like it, but I haven't found one that is free - as we can't afford to buy a premium server solution that would do the same thing as BlackIce.
At any rate, I'm working on identifying the IPs that are causing this, and will have them blocked soon.
|
|
|
Sonic Wallpaper
Joined: 05/22/01
Location: San Diego, CA
Posts: 1266
|
|
RE: Was the site down?
Sunday, May 2, 2010 at 11:45 AM
|
Quote
Top Bottom
|
Here was the part of the cause - I blocked this same thing about a week ago, but they have since changed their user-agent (a string of text that is used to identify a browser, bot, application, or device on the Internet, and is passed with each request to a site, etc.), which is what I was matching on. Seems to be a bot net, since...
* the IPs are not within a block
* the user agent string is the same for every IP
* they're all hitting the same page
* they don't request any other pages
* there weren't http referrers passed with the requests
* and all the above was true
I also blocked http://www.youdao.com - a search engine in China, they too were hitting the server hard, and ignored the rules I setup for bots. I banned them a week ago, but they sent a new IP range today.
At the same time... we were also being indexed by Yahoo, and Google, and I even see the MSN Bot was hitting us at the same time, although these 3 bots were making requests at an acceptable rate, not flooding the server with hits.
Once I put a new block in place based on their new user-agent string... the server came back online and is running normal.
|
|
|
Subtronik
Joined: 01/14/03
Posts: 1193
|
|
RE: Was the site down?
Sunday, May 2, 2010 at 6:22 PM
|
Quote
Top Bottom
|
Crazy... Like you said on the phone, it could be some little jerk kid in China commanding a hundred bots that's behind this.
Thanks for the hard work Gideon.
|
|
|
Sonic Wallpaper
Joined: 05/22/01
Location: San Diego, CA
Posts: 1266
|
|
RE: Was the site down?
Sunday, May 2, 2010 at 7:22 PM
|
Quote
Top Bottom
|
Thank you again for calling and pointing out that the site needed attention, and thanks for the appreciation 
Next Sat/Sun, I'll be sure to check the site throughout the day, but I'm hoping that this is the end of the problem.
|
|
|
FutureMusicCorp
Joined: 10/05/02
Posts: 7
|
|
RE: Was the site down?
Tuesday, May 4, 2010 at 6:07 AM
|
Quote
Top Bottom
|
Cheers for that, personally I reckon its all Icelands fault (they are responsible for everything else going wrong lately) 
|
|
|
|
|
