April outage period

So as you’re probably aware, the forum was down for a significant period of time over the last few days. It seems to be stable and reliable again right now so I thought it’d be a good time to provide an update.

On 2 April, Dreamhost (the forum’s cloud hosting provider) started a significant upgrade to the Cloud services that was supposed to be at most a couple of hours and had no particular prediction of downtime.

Once the upgrade was complete there were “connectivity issues” which basically seemed to mean every single site using DreamCompute was down and not responding to usual fix attempts, in fact I couldn’t even SSH in to try any commands or check on status manually.

After sixteen hours of down time they announced the engineers had “deployed a fix” at which time I attempted doing some hard reboots, etc and was now able to SSH in but none of the commands really accomplished anything. Some advanced users were reporting they could do some complicated fixes but all of them would risk an inexperienced user like me destroying the site.

Eventually the support twitter started advising people to lodge a ticket - and after a very sternly worded ticket they did whatever maintenance was necessary to this particular instance to get it up and running again.

Then there was a major update that needed to be applied manually via SSH that ended up with another twenty minutes down time.

Since then there has been no further update, so Dreamhost has not yet confirmed that the “fix” is stable and everything is good, however it seems for now the site is stable enough and I am continuing to monitor what information I can get.

Special thanks to @Urben for pinging me when the support twitter started messaging people to lodge a support ticket if a few basic “turn it off and back on again” solutions didn’t work.

I will update if there are any further meaningful updates to this matter - but for now I’m just going to knock on wood and hope it does stay online and stable.

47 Likes

Hopefully nobody read this as “outrage” this time again…

eyes shift nervously

13 Likes

“turn it off and on again” :laughing: :laughing: :laughing:

5 Likes

No matter if it sounds funny, sometimes it’s really helpful

Exactly! It works 9/10 times!

Thank you for keeping us apprised of the situation. For a while, I thought it was just me.

2 Likes

Delete your cookies!!

1 Like

Wise words, from a chef.

7 Likes

Thanks for your good care and update Jarbinger!

There were some concerns over funding during the downtime. Are there any updates on that? HMF has got an income stream that can keep itself running?

There once was a patreon but it seems to not exist anymore. I guess jar is doing extra work for us. Setting up smuggle points maybe?

I have to admit, when I first read the title of this thread, that’s exactly what I thought it said, but to my relief, I was wrong :laughing:

image

9 Likes

Don’t know if this is just me or a general thing, and if it’s related to the recent outage, but I’m getting a lot of errors today.

One that I managed to copy-paste is this one, when trying to read a topic:

Sorry, we couldn’t load that topic, possibly due to a connection problem. Please try again. If the problem persists, let us know.

I also get errors when liking a post (something about not getting access to that content).

And I get a 502 error when trying to post this.

13 Likes

Yes,me too

20 characters

Yeah, today forum is capricious

Topics seem to load most of the time… But when I want to go back to the main index - I usually hit the ‘Home’ key on my keyboard, then click the link at the top. But sometimes it goes up to some odd post and not to the Top/1st post.

Screenshot_Samsung_Internet_Beta_20190409-142954
I seeing this after liking messages

1 Like

3 Likes

It’s cause you’re using the light mode, fuckin’ noob

1 Like

I assumed it was the forum disapproving of what I’m ‘liking’.

2 Likes