I'm not sure how much help I can be with respect to gauging interest. I've seen vendors at smaller cons selling 3D printed stuff (a vendor at ICON in Cedar Rapids, Iowa last month was selling just these sorts of things), but I'm not sure how well they were selling.
Normal attendance at Gen Con is around 70,000 unique visitors. So as a rough estimate, you might look at the attendance at the cons you've been to, and how much you've sold, and figure it's proportional.
But booth rental, while a big expense, is just the start of it. You'll also need to consider the cost of lodging, meals, and travel. You'll probably need someone to help out at the booth (to cover breaks at the very least), and you'll need to deal with those expenses too. Plus, the expense of producing all of the wares you'll have for sale. You'll need a lot more inventory on hand for gencon than you'd need for a small local con. Can you afford the up-front outlay, and can you afford to sit on whatever stuff doesn't sell?
I think there are some setup fees, loading dock fees, etc. associated with the convention center as well. I'm not well versed on all of that, but I have heard dealers complain about them in the past.
I hope that's helpful.
Walt