In Marimacc's post on the old forums, she wrote:

[the change] will also allow the forums to become more stable, as they will be hosted with the rest of the Gen Con site .

By resting within "www.gencon.com", which is also where registration happens, it means that -- at minimum -- the forums are sharing the same load-balancing hardware and -- worst case scenario -- are on the same physical servers are the event registration engine, etc.

Every fiber of my "I do this for a living" being is crying out that this is a nightmare waiting to happen. Someone explain to me how I'm wrong. (And please, use long technical terms, because that's really what I'm looking for in order to feel warm and fuzzy about this).

Becuase the old forums were hosted IN the office.  They went down all the time.  For power outage reasons at the office (which actually happens a lot), for issues with that local server, etc.

They are now being hosted at the same place the rest of the site is, an actual server farm with redundancy and back up power, etc.  They will therefore, in fact, be much more stable, and much less prone to being down for a day or even a weekend or more.

EXCEPT in those cases when the actual server farm and network gear is being hammered during event registration, a time when we have seen portions of the server farm infrastructure fall completely over.

And in those cases where that happens, the forums have been a vital method of communication, for (a) Gen Con to users, to communicate what's happening and resolution timeframes, (b) users to Gen Con to communicate what problems are being seen, in real time, and get them due attention, and (c) user to user to share thoughts of their registration experience.

During "peak demand" periods, if that infrastructure is hammered to the point of breaking, you lose not only the application in question, but also the communications medium which has been heretofor a primary-source for information about what's going on and how to diagnose it and get it resolved.

I don't dispute that having them hosted in an actual colocation facility - generally speaking - provides them with much greater stability. Where I get off the bus is that having them still be in the same domain-space as the registration site, using the same infrastructure as the registration site, is - frankly - a rookie mistake.

Thank you for your input, as always, Derek. :)

As long as the forums are not on the same server as any of the registration system it "should" be ok during registration. Just make sure it is not on the same network segment within the hosting facility.

Based on observerd behavior, and 20+ years of IT experience, the problem during registration is not an overload of network bandwidth to the servers, but overload of the servers themsleves. The traffic gets there, but the server cannot respond in a timely manner and it does not failover in a graceful manner.

The forum is on "www.gencon.com". The reg system is on "www.gencon.com" ... that means that -- at some layer -- the same hardware is handling both forum requests and registration requests, even if its just a load-balancing layer that is slicing /forum/ and sending to "Pool F" and slicing /reg/ (or whatever URL pattern that uses) and sending that off to "Pool R".

In which case, if you end up overloading the load-balancing layer, you will have negatively impacted forum performance at the same level of impact as you have the registration performance.

That's the BEST case scenario. The worst case is substantially worse, as you might imagine.

That's based on 20+ years of building out data-center environments with literally the same sorts of problems Gen Con faces.

I figure if you want to get a hold of GenCon you email or call Customer Service (even in an emergency) no?  Expecting a forum thread to be your primary immediate path to getting things resolved... meh... maybe it's very possible but probably not the fastest. 

Personally I think it's better to sit and wait for more data on how things work on these 'new forums' get some burn in time for the new forums and let it grow on me before I make judgement.  Besides, it's their Con and their forums I can make all the suggestions about how I want things done or how I would do things but until I start my own Con or things go sidesways and I'm in charge of GenCon (Something very bad probably happened if this is the case)... Good job team GenCon and thanks. 

You may be new, or new enough that you've not been around for a truly great melt-down (it's been a couple years), but when it happens, the forums are vital.

marimacc

-forum moderator

As the forums become very sluggish and almost-unresponsive during event pre-reg, I want to just say, "Yep, I told you so."

Yep. Totally agree.

seem to be responding fine to me.

It was quite heinous in the first 30-45 minutes.

I mean if you think a 15 second refersh is agonizingly slow I guess it was, but so far it's been up and steady the entire time.

How about 503 Service Unavailable after more than 15 seconds?  I was getting those off and on for the first half-hour or so.  (A bunch early on, then less as things calmed down.)

Yeah, there were a BUNCH of 503 "No Backend Server" available errors for quite a while. There were also times where it literally took two minutes to load the page.

And "no backend server" means this wasn't "oh, your local ISP is bad", that's "the load-balancing layer is falling over".

I initially got several 503 errors too.  I wasn't trying to use the forums or process a wish list, but I like to keep track of our events and see how many tickets are sold.
 

Other than i had to wait till i got home (since i was out of town without my laptop at noon), i had no issues.