Categories
General Technical

Update on the System Recovery

Hello,

I realize that for many of you this has been a long and troublesome process, and I can thankfully say that ShoutWiki will soon be back up and fully functional.  Currently, I’m working on recovering all the databases; it’s a slow process, but so far all the databases that have been recovered (there are around 700 in all) are completely intact which means that you did not lose any pages, etc. from this crash.

After all the databases are recovered, we will need to basically reinstall the server, and once that is completed ShoutWiki will back in action.

Thank you all for your patience during this time, and I hope that you continue to choose ShoutWiki!

Regards,

Skizzerz
ShoutWiki Technical Team

UPDATE 1 (31 March 2011, 3:14am UTC): Current status is 125 done, 521 to go. Will continue to update as this progresses.

UPDATE 2 (31 March 2011, 3:52am UTC): Current status is 274 done, 372 to go.

UPDATE 3 (2 April 2011, 6:55pm UTC): There were about 10 wikis that didn’t recover properly, so I’m working on a way to recover them. Once this is done, everything else should go relatively quickly.

UPDATE 4 (4 April 2011, 11:56pm UTC): Clarification on the above: When I said they did not recover properly, this does NOT mean that they can’t be recovered. The issue lies with some stuff wrong with the backend behind the dbs, so certain large wikis with lots of revisions and log entries took too long to dump in one go before MySQL crashed. What I’ll be doing for these 6 wikis (got 4 of them to work normally since the last update) is dumping them table-by-table, so your stuff will not be lost. As for an update on how other stuff is going, we are now waiting for the Head Tech to reinstall the Operating System on the server so that we may begin restoring our backups. Due to the nature of the backup restoring process, wikis will become available as the backups get restored, so check back in a day or three to see if your wiki is up.

UPDATE 5 (9 April 2011): OS is going to be reinstalled sometime later today, and provided all goes well, so will the wiki restorations. Wikis will be restored in alphabetical order and will become available as they get restored (meaning some will be up before others). I’ll post another update when the restorations actually start.

UPDATE 6 (16 April 2011): The tech team has taken backups of databases and important configuration files. The CTO estimates that we might get started on the recovery as soon as this weekend, but it might take a major part of the next week. We’ll keep you posted on this. –Jack Phoenix

UPDATE 7 (19 April 2011): The operating system has now been reinstalled and the backup files are being copied to the server currently. Recovery will begin later today. –Jack Phoenix

UPDATE 8 (19 April 2011): Data recovery proper has begun. During this process the site may display various error messages. –Misza

UPDATE 9 (19 April 2011): The wiki restoration process has begun. Wikis should become available as they get restored. They are going in random order, so we cannot give any ETAs on specific wikis. Please note that just because your wiki is back, it does NOT mean that all the restores are done. As such, if issues arise, please alert a staff member on IRC so that we can look into it. We are also considering rolling out the 1.16 upgrade after the wikis get restored, so you may see some downtime there as well. –Skizzerz

UPDATE 10 (21 April 2011): We have restored the vast majority of the wikis, as people were quick to find out, we ran into a LOT of issues trying to restore Autopedia and Simpsons Fanon. We’re going to continue working on ways to restore these two wikis, but in the meantime, we will be pushing out the 1.16 upgrade to the rest of the wikis. After the upgrade finishes, editing will be re-enabled. –Skizzerz

UPDATE 11 (27 April 2011): We’re working on both issues, the MediaWiki upgrade and restoration of Autopedia and Simpsons Fanon databases, but it turns out that MediaWiki’s default upgrade script is not suitable for our purposes as-is, which means that we have to write our own upgrade script. This is something that we want to do right, without rushing, in order not to mess up anything, which is why it might take a bit longer then expected. Once the techs have finished writing the upgrade script, the actual upgrade process should be relatively quick, after which the database will be unlocked and you can resume editing. We’ll keep you posted on this. –Jack Phoenix

UPDATE 12 (1 May 2011): The update process has begun. In the meantime, a “ShoutWiki is currently undergoing maintenance” page will show until updates are complete. Once complete, editing will be re-enabled. ETA is about two hours from now, provided no issues are found. –Skizzerz

UPDATE 13 (1 May 2011): Updates are complete and editing has been re-enabled. If you run into issues, please let someone know on IRC and open a bug in our bugzilla: http://bugzilla.shoutwiki.com — At some point in time we’ll be re-updating to the newest 1.16 version, but this will not cause any downtime and should be for the most part invisible to you guys.

By Skizzerz

ShoutWiki staff member

387 replies on “Update on the System Recovery”

Everyday I rise and click on the link at the top only to get an “Oops! Google Chrome could not connect to stargate.shoutwiki.com”, I then come to this blog to see people complaining. I then spend the day checking at least 5 more times each time dying a little inside.

I’ve built up a few pages to upload once the site comes back online, but without a wiki it just feels pointless. As a free service I do realise that you have no reason to bring the site back on as quickly as possible, but 25 days is a quite enough for anyone to stand.

Unless the site is back on in 5 days (1 month out) and some sort of assurances can be offered to counter the slow times, site outages and server failures I am moving all of my wikis elsewhere and convincing everyone else to leave as well. I mean this is absolutely ridiculous!

I think tomorrow will be the breaking point for me. I’m free for the weekend, and could use it to re-build content. If it is not back, I will have lost every ounce of patience.

I can`t help but think that Wikia may have something to do with that. You may be deliberately stopping the backup because you may have got something from Wikia.

Think of it – 25 days of site non-availability with not enough information. A perfect chance for Wikia to say “That will never happen to us”.

I hope I am mistaking. I hope you have a SERIOUS problem preventing you form quick “re-start”.

I have read complaints about the contents of all
But how else can we do?
After all, the official server is in shoutwiki
Although my patience had almost reached the limit
But I will continue to wait
Before repaired in shoutwiki
I think we temporarily use the wikia to it!
(I’m from Taiwan)

@AWAMember: that was my guess from the start. Now that the case is getting old my opinion is whether it was wikia’s hand or just a sheer lack of professionalism they are not willing to recover from this because they can see no point to it. “It was not that popular to continue”. I can only pray for ability to download the backup in the end. 🙁

@AWAMember: If it’s that f**ken Oasis skin, and it will be free, no deal! The day ShoutWiki gets Oasis is the day I leave with my wiki!

How can you guys suggest that they deliberately broke ShoutWiki for Wikia’s benefit?! They would not do anything like that!

YES! An update!
@Silver Hopefully, I can restore the Jeep pages,
@Mthigator Maybe those 6 wikis took a long time, IDK.

Finally…thanks for the update! Will the actual website (shoutwiki.com) be up so I can re-create my wiki? Thanks again!

I hope this all goes smoothly. The downtime has been an actual nightmare as the wikia staff found that we moved and have been harassing us the past month and its been hell.

Re:OS is going to be reinstalled sometime later today, and provided all goes well, so will the wiki restorations. Wikis will be restored in alphabetical order and will become available as they get restored (meaning some will be up before others). I’ll post another update when the restorations actually start.

I very Moving!

27 days… I’m already starting to contact someone who can help make my wiki independet.

If this reaches 30 days without a single wiki back online, it will be Mass Wiki Moving all over again! I’m starting to become inpatient, y’know!

The Spyro Wiki came from Wikia thanks to Oasis, and now it’s moving again. I’ve had enough!

It seems that the OS is now or near to be downloaded, the error screen is not the same. There is still hope. 😉

I think the OS is back up.I am now getting a cannot connect to MySQL message, instead of could not find server message.

I hardly consider a ‘different’ error message is good news. Only two more days to get it back up or I am leaving, and convincing everyone else to leave as well.

Even if the message is different now, it could be days (weeks, even) before at least one of the wikis is back up.

And I’m stuck here until I’ve got myself some helpers. I’m still too young to afford an independent site (since I’m almost 17).

Me and my boyfriend have been able to write three entire stories for Legacy of the Forest wiki during this downtime. It’s getting ridiculous.

Come on guys, you can use Notepad to write your articles ’cause you suppose to know wiki syntaxe, and when the site is back, you just have to upload the content. It will just be a massive amount of new pages at once, right?

I agree, but it helps when you write new articles or when you have made a notepad copy of your article, especially the longer ones.

I swear to god, I’ll forgive ShoutWiki after this is over, but if this happens again, I will walk out the door. You’ve cost now 30 days; 30 F*CKING DAYS!! THINK ABOUT IT!! 30 days worth of missed car blogs and updates. IT will really be impossible to recover from this crock of shit you’ve caused ShoutWiki.

@silver Yes, one time, I can forgive, but if it happens again (an outage of more than one week EXCEPT for something like a mediawiki upgrade) and if any data is lost (ex post march 7) I will likely leave.

All of you seriously, shut up… Before you keep on ranting about how this is the staff’s fault, this error was first off caused by their hosting company, not the ShoutWiki staff. Why don’t you blame those who are actually responsible for this instead of the people who are frantically trying to get everything running again?

Second, you make it sound like the world is ending tomorrow. If you can’t edit the wiki, ever heard of Notepad or any text editor? If you quit yelling about the issue and instead focus the time on writing content on text editors to upload later, you might ACTUALLY save TIME! (awesome right?)

Oh and last: Shoutwiki staff probably could speed the process up but is it really worth doing so and sacrificing the chances of recovery?

Given how slow shoutwiki’s service was before this disaster, the service will be even slower now due to all the editors checking their work and the massive backup all the editors will be doing.
On top of that put all the edits that the editors will try to do.
The practical result will be another month of no service.

I agree with Silver Knight. I’ll be making periodic backups and moving if this happens again.

On another note, seeing comments on here every day from the autopedia people acting like their wiki is the only one suffering is getting almost as annoying as everything still being down.

@Kirbman Myself and Silver Knight are the only people from Autopedia commenting. Heck, Autopedia is not even my main wiki. It just affected us alot because we lost 400 pages and 500 images from the crash, and has five active users. The wiki in my sig is my top wiki.

You have until the end of the day to get them back up or I am leaving with all my wikis and referring everyone else to do the same. 30 days to restore a site, pathetic.

This was my first time attempting to make a wiki site and I only had it for a week or two, so my loss is nothing compared to many veterans here, but the whole site is likely erased because of when I made it (don’t I have great timing?). I had barely begun to understand the coding to boot!

I guess my point is, it’s a slight disappointment to have this be my first experience with a free wiki, but I’m taking what I did learn and writing in notepad for now so I hope it comes back soon so I can see how my new pages will look. Do your best Shoutwiki.

Too bad I cannot move to an independent site; I’ve got no helpers! If I could, I’d move ASAP!

This is the most pathetic thing ever!! Wikia would never do something that like this. Almost 30 days!! Jesus christ!! Thats a month!! YOU should feel disgraced.

…and thats 30 days. Well I am going to be taking all my wikis and leaving. 1 month is RIDICULOUS!

I URGE EVERYONE ELSE TO LEAVE THE SITE AS WELL. You screwed up Shoutwiki. Goodbye.

Leave a Reply

Your email address will not be published. Required fields are marked *