Forum outage 01/10/2018

Forum News, Announcements and any issues you're having with the forum

Moderator: myglaren

User avatar
xantia_v6
Forum Admin Team
Posts: 7908
Joined: 09 Nov 2005, 23:03
x 512

Forum outage 01/10/2018

Post by xantia_v6 »

This forum was down for about 5 hours this morning (1/10/2018).

The exact cause of the failure is not clear yet, but while attempting to resolve a server performance issue, it was found that the forum database was corrupted and has been subsequently restored from backup. This process took rather longer than normal.

Fortunately the failure occurred shortly after the daily backup was taken so there were no more than about 2 posts lost.

My apologies to anyone inconvenienced.
User avatar
Paul-R
Donor 2020
Posts: 5441
Joined: 07 May 2009, 16:24
x 803

Re: Forum outage 01/10/2018

Post by Paul-R »

Welcome back - it wasn't here five minutes ago.
MikeT
Posts: 4809
Joined: 11 Jun 2007, 16:17
x 188

Re: Forum outage 01/10/2018

Post by MikeT »

Not saying it's the forum server Mike but over the weekend the forum randomly failed to load a few times for me.
User avatar
xantia_v6
Forum Admin Team
Posts: 7908
Joined: 09 Nov 2005, 23:03
x 512

Re: Forum outage 01/10/2018

Post by xantia_v6 »

It was probably the onset of the same issue.

I think that there were some hung threads in the http server which were gradually sapping the server performance, and I was just unlucky that when I restarted the http service this morning the database was left in a state where it became corrupted after the restart. It took a couple of hours of testing to get to the point where I was sure that it was a database corruption and not something else (the symptoms were very strange). Then the database restore (which normally takes about an hour) somehow did not work the first time, and I aborted it after 2 hours, then started it again.
User avatar
Skull
Donor 2020
Posts: 764
Joined: 02 Aug 2013, 13:35
x 97

Re: Forum outage 01/10/2018

Post by Skull »

Contact us message sent

May be a problem my end but just seems to affect FCF :?:
Scrrenshot
Scrrenshot
User avatar
xantia_v6
Forum Admin Team
Posts: 7908
Joined: 09 Nov 2005, 23:03
x 512

Re: Forum outage 01/10/2018

Post by xantia_v6 »

Thanks again for the report. The server is getting intermittently overloaded, I have seen loads 5000% higher than expected. This is causing a number of symptoms, but the root cause is still being investigated.
MikeT
Posts: 4809
Joined: 11 Jun 2007, 16:17
x 188

Re: Forum outage 01/10/2018

Post by MikeT »

Bloody Russians

:twisted:
User avatar
GiveMeABreak
Forum Admin Team
Posts: 26052
Joined: 15 Sep 2015, 19:38
x 2988

Re: Forum outage 01/10/2018

Post by GiveMeABreak »

Good point - I'm sure our servers aren't hosted in Holland!
User avatar
xantia_v6
Forum Admin Team
Posts: 7908
Joined: 09 Nov 2005, 23:03
x 512

Re: Forum outage 01/10/2018

Post by xantia_v6 »

It could also be the Americans...
User avatar
GiveMeABreak
Forum Admin Team
Posts: 26052
Joined: 15 Sep 2015, 19:38
x 2988

Re: Forum outage 01/10/2018

Post by GiveMeABreak »

I think it's more likely to be Brussels though...
Hell Razor5543
Donor 2021
Posts: 12014
Joined: 01 Apr 2012, 09:47
x 1513

Re: Forum outage 01/10/2018

Post by Hell Razor5543 »

What about the Chinese or North Koreans?

I doubt this is helping Mike one little bit, but we can have fun trying to guess the cause! :D
User avatar
bobins
Donor 2021
Posts: 4170
Joined: 05 Jul 2012, 18:07
x 1338

Re: Forum outage 01/10/2018

Post by bobins »

Might be the French taking exception to all this free info on here that's prolonging the life of their cars and stopping them from selling new ones :wink: