Thank you for your hard work robin! Were having a skål here in the bunker for you.
I'm so glad we're back in business here! Although I have steeled myself for a rash of typos until everyone's shakes abate.
I didn't realize how much I read pinside daily until it was taken away, I was going through withdrawls!
Quoted from RCA1:I saw a comment elsewhere on line being very critical of Robin for running Pinside from a personal server at home, instead of from a professional hosting service that could never possibly crash.
Glad you had backup, since it looks like lots of folks on webnx are screwed!
Those people are clueless, they all kept saying it’s a personal server when WebNX is a professional outfit who goofed up hard lol.
The Facebook peanut gallery proved they are toxic af.
Quoted from alexanr1:May want to consider hosting on AWS in the future....
No thanks, Amazon doesn’t need to own the last free half % of the Internet.
Quoted from Lermods:Was kind of an eery silence in the pinball world for 6 days.
For all the haters, it really showed what a force Pinside is. I'm willing to bet BUSINESS was down across the board.
Quoted from Friengineer:Some blame fire suppression, some blame Robin, I blame all those price bubble threads.
I could see Robin being like...nope turns out those threads didn’t survive the backup. Darn corrupted sector
Good job Robin !
But look out for a better provider .... I read in the internet that they tried to fight a fire in a data center with ... water ....
That seems very stupid.
Great job on getting us up and running again so soon! I can't figure out how to attach a pic yet, but maybe its just me!
Quoted from robin:My last database backup was dated 10:02pm, while the server went down 11:39pm. So we lost 1h37m of data. That means any posts, PMs or Marketplace ads placed during that time are gone.
Oh man, that's exactly when I made some of my best posts!!!
Seriously though, thanks for getting it back up! I'm sure it's quite stressful.
Glad to see you’re back up and running finally. I hope you can stay that way for a while. Can’t wait to see the new servers in the future! Take care and best wishes.
Robin -
Thanks for your hard work. I really appreciate it.
Put your next server someplace where they use halon as a fire suppressant.
Thanks so much for bringing Pinside back!
I guess it’s true what they say about not knowing how good something is until it’s gone
I sure did miss it.
Thanks again
Welcome back Robin!
Thanks for everything you do, I can't image what you have had to go through to get things back up so far.
If you need any help please let me know.
I have my own servers out here in the Bay Area:
All virtual on multiple hosts / load balanced. Redundant: uplinks, firewalls, 1g Cisco WAN switches, 10g Cisco SAN/LAN switches. Redundant Flash SAN (2 active/active nodes)
If you need any server admin help from a fellow pinhead just let me know! (I'm sure others would be happy to pitch in as well!)
Cheers
~Steve
Quoted from PinMonk:My first question on learning what happened was "What kind of datacenter uses WATER as a fire suppression choice instead nitrogen or another non-damaging non-electrically conducting gas?"
Crazy. We all learned something here.
https://datacenterfrontier.com/generator-catches-fire-causes-lengthy-data-center-outage-at-webnx/
The real question is, what kind of datacenter puts the generators INSIDE with the servers?
WebNX = clowntown.
Quoted from Billc479:Robin -
Thanks for your hard work. I really appreciate it.
Put your next server someplace where they use halon as a fire suppressant.
fire codes in an lot of places make you have wet with an dry system.
Welcome back Robin! Look at it this way- the worst has happened and now you are back and better than ever before .
Thank you and the support staff for all your hard work!
I’m surprised the site isn’t hosted on a cloud provider. New “server” in <1 minute, multiple availability zones per region, DB replication across AZs, etc., etc.
Great job Robin!! I'm sure things will be even more secure and backed up than ever now!
Talking to a friend the other day I pointed out that the root cause of all of this was most likely those two useless pinside gif threads. All of the bandwidth and storage space they use probably caused the server rebuild to take twice as long even if they didn't cause the initial power failure and fire.
With the recent announcement that Pinball Rescue was closing for good and then Pinside going down... I was going full pinball panic mode. Still kind of am, but relieved to be back.
Pinside is an awesome and vital resource for the community. In fact, I wouldn't be where I am today without it. <3
Thanks Robin for working hard to bring the site back. There was a gaping hole in my life the last week and Im glad were back!
I work for a company that does nothing else but assess data centers for their ability to withstand stress. That's what we do and after looking at literally tens of thousands of data centers, all I can say is you have no idea how silly some site owners are. The crazy choices they make. It's a combination of cost, business model and expertise.
That said, the hosting/co-location world is very Darwinian... with so many to choose from, it really is survival of the fittest!
Quoted from Joe_Blasi:fire codes in an lot of places make you have wet with an dry system.
I'm dating myself - should have said FM200. Also, the purpose was to thank Robin for his work.
Fire suppressant discussions should follow NFPA 12 and NFPA 2001 as amended. Hard to imagine why the section with servers only would fall under class 2 water suppression unless they were mixed with other combustibles.
Quoted from jorro:Ahhhhhh i missed you pinside!!!
Thanks for all your hard work Robin.
We need some kind of TShirts!
Fuck Corona......, i was survived the great pinside blackout 2021!
Robin you should sell t-shirts that say this. I'll buy one!
"I survived the Great Pinside Blackout of 2021"
Thanks for getting everything back and running, robin. Its been a weird time in pinball lately. This site was down for a week, the virtual pinball community has been going through some major drama and site closings. Nothing too serious on the grand scale, but it's nice to have some normalcy back. I miss the pinside machine database as much as or more than the forum. I've always enjoyed reading through the reviews and watching the linked youtubes to kill time here and there.
I "donated" about an hour before the site went down with my pinball for sale post. I immediately had people interested but couldn't msg them for 5-6 days due to the outage. Maybe pinside could donate to ME & my buyers for our inconvenience?
I am addicted to pinside !!!
Got very stressed out every time I checked and seen that it was still down....
Had nothing to do at work...... when it was down....LOL
Quoted from kevmad:"I survived the Great Pinside Blackout of 2021"
And in years to come. "My parents made me in the Great Pinside Blackout of 2021". Baby boom in 9 months.
LTG : )
Wanna join the discussion? Please sign in to reply to this topic.
Great to see you're enjoying Pinside! Did you know Pinside is able to run without any 3rd-party banners or ads, thanks to the support from our visitors? Please consider a donation to Pinside and get anext to your username to show for it! Or better yet, subscribe to Pinside+!
This page was printed from https://pinside.com/pinball/forum/topic/back-after-a-major-outage/page/2?hl=catvilledoorman and we tried optimising it for printing. Some page elements may have been deliberately hidden.
Scan the QR code on the left to jump to the URL this document was printed from.