Close Combat Series :: View topic - T-House bandwidth problems

through

[^/[Print]\]

Close Combat Series -> Close Combat 2: A Bridge Too Far

#1: T-House bandwidth problems Author: Homba,

Posted: Thu Jun 22, 2006 6:00 pm
—
June 22: 11:05 pm central time update:
Additional bandwidth has been obtained and Ratings part of the site is back up. You can report losses again! :) Forum has been temporarily transferred to CCS while we continue to work on the problems associated with the webcrawlers draining bandwidth from TH forum. A lot of progress made, big thanks to mooxe for lots of help.

----------------------------------------------
There has been a huge spike in bandwidth use over the last few days from unknown outside sources, we are trying to solve the problem, but it may be over our heads. The 2gigs of extra bandwidth I got from the host was used up in under 24 hours, and TH is down again. We will continue to try to solve the problem. But I do not expect the site will be back up until the new bandwidth allowance kicks in on July 1. At that point we will make some decisions about what do to, including possibly disconnecting the TH forum, which seems to be the source of the problem. We have alternative forums, including this one, that we can use. I considered upgrading to 35G of bandwidth, but at the current rate of drain the 10 additional GB would have been used up well before July 1, so that does not seem like a useful solution.

Any of you who see this, please spread the word if you are in powerlobbies and see other CC2ers. I suggest you cease playing rated games until such time as the site is up and you are able to report again.

Sorry, I have no idea why this is happening.

Homba

Last edited by Homba on Fri Jun 23, 2006 4:08 am; edited 1 time in total

#2: Author: Styer27,

Posted: Fri Jun 23, 2006 1:05 am
—
Wow thats bad. I cant even get Power lobbies downloaded caus its on the CC2 tournamenthouse page, if theres any other place to DL power lobbies you should post it here.

Thanks for this post, i didnt know what the hell was goin on till i seen it.

#3: Author: Styer27,

Posted: Fri Jun 23, 2006 1:16 am
—
I couldnt find a link to Power Lobbies but i found an old BattleHQ link and amazingly it still works the same way, i can connect to people in power lobbies through it Smile

Heres the link:

http://www.wargamer.com/battlehq/

#4: Author: Homba,

Posted: Fri Jun 23, 2006 2:40 am
—
We've got a fix plan implemented which includes a redirect to these forums for the time being, since our forums are the main target of these outside bandwidth-sapping webcrawlers. I have asked to buy or borrow two more gigs of bandwidth to get us through the month for the ratings part of the site.

Should have an answer from them in the morning and it is possible that the ratings part of the site will be available again tomorrow (Friday June 23). But don't count on it yet. Surely by the weekend though, knock on wood.

H

#5: Author: Homba,

Posted: Fri Jun 23, 2006 4:11 am
—
June 22: 11:05 pm central time update:
Additional bandwidth has been obtained and Ratings part of the site is back up. You can report losses again! Smile

Forum has been temporarily transferred to CCS while we continue to work on the problems associated with the webcrawlers draining bandwidth from TH forum. A lot of progress made, big thanks to mooxe for lots of help.

Things moved more quickly than I'd hoped, so keep fingers crossed we can put this behind us. On July 1, we'll open up the TH forums again, and keep an eye on the problem IPs that are actually bots draining bandwidth and ban them. Hopefully we can get things back to normal soon. In the mean time I am still blogging the World Cup in 'THE MESS' area of CCS forum!

H

#6: Author: Ocelotl, Location: Oceanside California

Posted: Sat Jun 24, 2006 4:22 am
—
Evil or Very Mad

Creepy crawlers, things that go bump in the night man what a pisser. Is this situation a random occurence or an intentional attack? GL with your struggles Homba. Lookin forward to when everything returns to normal. Smile

#7: Author: Homba,

Posted: Sat Jun 24, 2006 5:01 am
—
hey Oce, I don't know. One of my thoughts was that if every site on the net were experiencing this sort of crawler activity, the bandwidth drain would bring the whole www to a standstill. Are we being individually targetted? I don't know.

#8: Author: CB_Recon,

Posted: Sun Jun 25, 2006 12:37 pm
—
Homba, cant u prevent this by having the sign up process go directly through you or another admin? as it stands now, all you need to do is sign up and get a confirmation email right? I would also suspend guest from posting.

#9: Author: Homba,

Posted: Mon Jun 26, 2006 3:52 am
—
I have to confirm forum accounts manually before they work. But you have a point - i may make the forums VIEWABLE to members only, so you cant view them at all if you're not a member - the thing is, I dont believe the google crawlers have to have a forum account to view the forums anyway. But I could be wrong. When the new month starts, i'll try making the forums viewable to members only and see if it helps.

H

#10: Author: Mousegrey,

Posted: Tue Jun 27, 2006 4:17 pm
—
Homba,

My first thought was that TH was prob. under a DOS-attack from someone.
but in that case you should get a lot of pagerequests. If it's bots that crawl the site: isn't there a piece of html that takes care of that...? Will have a look too.

This may be interesting:
http://bitworking.org/news/No_Fishing

#11: Author: Homba,

Posted: Wed Jun 28, 2006 3:27 am
—
mooxe mentioned that he tried to add a line of code that would block the crawlers, but he didn't think it had worked. If you know how/what to do, regarding that, let me know.

#12: Author: mooxe,

Posted: Wed Jun 28, 2006 3:41 am
—
Crawlers, or robots should not be taking up the amount of bandwidth that they are. A robots.txt file was added to deny all robots to all folders, it didnt work, misconfiguration or whatever I am not sure. Its a unique problem I think. The bots right now are being redirected to this site, and they are showing up in my logs. The bandwidth limit here is 2000gigs, where as TH is 25. I think the best option is for TH go members only July 1st for a limited time untill we can figure out a better way to deny the robots. Deny by IP is not a long term fix, its more of a bandaid.

We will win sooner or later and many gigs may die in the struggle.

#13: Author: Homba,

Posted: Sat Jul 01, 2006 3:56 am
—
Ok, TH forums is back up for the new month. We've made it visible to registered members only, (not guests) so you have to log in to see the forums. We think this'll help the problem and will monitor to see if it does.

Thanks mooxe for all your help.

H

Close Combat Series -> Close Combat 2: A Bridge Too Far

output generated using printer-friendly topic mod. All times are GMT

Page 1 of 1