Categories
General Technical

Update on the System Recovery

Hello,

I realize that for many of you this has been a long and troublesome process, and I can thankfully say that ShoutWiki will soon be back up and fully functional.  Currently, I’m working on recovering all the databases; it’s a slow process, but so far all the databases that have been recovered (there are around 700 in all) are completely intact which means that you did not lose any pages, etc. from this crash.

After all the databases are recovered, we will need to basically reinstall the server, and once that is completed ShoutWiki will back in action.

Thank you all for your patience during this time, and I hope that you continue to choose ShoutWiki!

Regards,

Skizzerz
ShoutWiki Technical Team

UPDATE 1 (31 March 2011, 3:14am UTC): Current status is 125 done, 521 to go. Will continue to update as this progresses.

UPDATE 2 (31 March 2011, 3:52am UTC): Current status is 274 done, 372 to go.

UPDATE 3 (2 April 2011, 6:55pm UTC): There were about 10 wikis that didn’t recover properly, so I’m working on a way to recover them. Once this is done, everything else should go relatively quickly.

UPDATE 4 (4 April 2011, 11:56pm UTC): Clarification on the above: When I said they did not recover properly, this does NOT mean that they can’t be recovered. The issue lies with some stuff wrong with the backend behind the dbs, so certain large wikis with lots of revisions and log entries took too long to dump in one go before MySQL crashed. What I’ll be doing for these 6 wikis (got 4 of them to work normally since the last update) is dumping them table-by-table, so your stuff will not be lost. As for an update on how other stuff is going, we are now waiting for the Head Tech to reinstall the Operating System on the server so that we may begin restoring our backups. Due to the nature of the backup restoring process, wikis will become available as the backups get restored, so check back in a day or three to see if your wiki is up.

UPDATE 5 (9 April 2011): OS is going to be reinstalled sometime later today, and provided all goes well, so will the wiki restorations. Wikis will be restored in alphabetical order and will become available as they get restored (meaning some will be up before others). I’ll post another update when the restorations actually start.

UPDATE 6 (16 April 2011): The tech team has taken backups of databases and important configuration files. The CTO estimates that we might get started on the recovery as soon as this weekend, but it might take a major part of the next week. We’ll keep you posted on this. –Jack Phoenix

UPDATE 7 (19 April 2011): The operating system has now been reinstalled and the backup files are being copied to the server currently. Recovery will begin later today. –Jack Phoenix

UPDATE 8 (19 April 2011): Data recovery proper has begun. During this process the site may display various error messages. –Misza

UPDATE 9 (19 April 2011): The wiki restoration process has begun. Wikis should become available as they get restored. They are going in random order, so we cannot give any ETAs on specific wikis. Please note that just because your wiki is back, it does NOT mean that all the restores are done. As such, if issues arise, please alert a staff member on IRC so that we can look into it. We are also considering rolling out the 1.16 upgrade after the wikis get restored, so you may see some downtime there as well. –Skizzerz

UPDATE 10 (21 April 2011): We have restored the vast majority of the wikis, as people were quick to find out, we ran into a LOT of issues trying to restore Autopedia and Simpsons Fanon. We’re going to continue working on ways to restore these two wikis, but in the meantime, we will be pushing out the 1.16 upgrade to the rest of the wikis. After the upgrade finishes, editing will be re-enabled. –Skizzerz

UPDATE 11 (27 April 2011): We’re working on both issues, the MediaWiki upgrade and restoration of Autopedia and Simpsons Fanon databases, but it turns out that MediaWiki’s default upgrade script is not suitable for our purposes as-is, which means that we have to write our own upgrade script. This is something that we want to do right, without rushing, in order not to mess up anything, which is why it might take a bit longer then expected. Once the techs have finished writing the upgrade script, the actual upgrade process should be relatively quick, after which the database will be unlocked and you can resume editing. We’ll keep you posted on this. –Jack Phoenix

UPDATE 12 (1 May 2011): The update process has begun. In the meantime, a “ShoutWiki is currently undergoing maintenance” page will show until updates are complete. Once complete, editing will be re-enabled. ETA is about two hours from now, provided no issues are found. –Skizzerz

UPDATE 13 (1 May 2011): Updates are complete and editing has been re-enabled. If you run into issues, please let someone know on IRC and open a bug in our bugzilla: http://bugzilla.shoutwiki.com — At some point in time we’ll be re-updating to the newest 1.16 version, but this will not cause any downtime and should be for the most part invisible to you guys.

By Skizzerz

ShoutWiki staff member

387 replies on “Update on the System Recovery”

Can we have an update please? Werent wiki restorations supposed to happen on Saturday? Fourth time you’ve missed your dates. This is getting RIDICULOUS.

I agree with that last post by Hockeyben. If you say that something is going to happen today and 3 days later we haven’t even been told why it hasn’t happened yet we kind of feel abandoned.

I hope you guys at Autopedia didn’t lose as much as you think. It might help to search Google for your pages and see if any that might have been lost were cached.

@Escyos We’re at 29 days now, what time zone you’re in doesn’t change how long it’s been down.

@Kirbman I managed to save around 100 of them. The other couple hundred were copied from another car wiki in which Autopedia is merging with.

@kirbman
1. 15th (14th)
2. 16th
3. 17th
4. 18th
5. 19th
6. 20th
7. 21st
8. 22nd
9. 23rd
10. 24th
11. 25th
12. 26th
13. 27th
14. 28th
15. 29th
16. 30th
17. 31st
18. 1st
19. 2nd
20. 3rd
21. 4th
22. 5th
23. 6th
24. 7th
25. 8th
26. 9th
27. 10th
28. 11th
29. 12th
30. 13th

30 days. Pathetic.

@Escyos today is the 12th, in times zones where its the 13th, the servers went down on the 15th rather than the 14th

lol at all these people leaving really. I’d hate to see them wait at a restaurant.

It’s not like they wanted the system to fail on them (That’s already Livejournal’s job), and they’ve informed their members of what they were doing. I’d like to see these whiners do better than the team fixing a huge MYSQL database, because I assume it can’t be easy, IT prick or not. Don’t let the door hit you on the way out!

Jack, Skizzerz, whoever else from ShoutWiki might be reading the blog comments: You guys need to SERIOUSLY get with the program. And I don’t just mean in the technical sense of getting the wikis working; I also mean in the public relations sense.

Each time you say that something will happen by a particular date and it doesn’t, ShoutWiki takes a credibility hit. And each day that goes by with no further updates after a non-happening, you take another credibility hit. You need to get your rears in gear, get onto the blog and say SOMETHING — at this point, even a lame excuse would be better than no communication at all.

If your situation is that you’ve decided to close up shop entirely, that would be unfortunate. But if that’s the case, how about at least making data dumps available so we can take our wikis and go find a host that will provide reliable service and regular communication?

@Mythigator

What’s the point, Mythigator. They’re like Wikia; they’ll just ignore you and do things their way. I’m starting to feel that by time they fix this mess, I’ll be an old lady.

@Everyone: I hope that you realize that our systems did not fail, our host’s did, and as a result, we suffered. We have to fix things that they essentially broke and try not to break anything in the process of doing so — you want your wikis to be fully functional, of course, and that’s what we’re aiming for!

@Mythigator: I personally prefer fixing things as quickly as possible and then announcing “we’ve fixed it!” instead of giving lame excuses to the general public.

And no, we have no plans to “close up shop entirely”, don’t worry about that. Closing the site without giving people a chance to grab a backup of their data would be quite dickish in my opinion.

It is no secret at this point that mistakes were made. We’re hoping to improve the service and naturally prevent disasters of this magnitude. Nevertheless, I think that most of you would appreciate the work we do a bit more if you knew the differences between ShoutWiki and other wiki hosting services (I don’t mean just the differences in features and whatnot, for example).

@Jack Phoenix: Thanks for the update, just seeing a new message to confirm that we haven’t been forgotten is nice. Any idea when the restorations will begin?

@Escyos: Nevermind your inability to understand basic math and time zones is hopeless.

Well, thanks for the update, Jack. I really do hope that ShoutWiki is back up soon. But what’s the progress on re-installing the OS? Any estimations on when the wikis will restore? Thanks again.

@Anno

If AWA moves, can the shoutwiki site stay up? I’m an admin on Awa (however, my rights were lost as they were given on march 12), and I would still like to contribute to the shoutwiki one, as well as the new one.

I would also like a status update on the repairs.

@kirbman – the 14th is the 15th, so the 15th is the 16th, 16th is the 17th…….12th is the 13th, if its a day ahead it stays ahead, even if only by one minute into the next day. And I thought all the idiots stayed behind on wikia….

@everyone else – the site is down and has been for a long time, the site was slow before hand and frequently unaccessable. Over 1 month of down time and no progress is not good. Move to other hosts, Shoutwiki screwed up.

@ hockeyben

Then have two awa’s sigh. That is the same problem again we are all having with wikia.

I probably won’t move to my site, as I don’t know how to install new mediawiki myself. But maybe wikii?

We will see how slow the server is when this returns, then we can all vote on it.

@Escyos As Jack said, Shoutwiki DID NOT screw up, the host did, so stop ranting on this blog and go ranting to them.

And don’t bother prompting everyone to leave Shoutwiki, because they won’t unless they are as unpatient as you.

@Escyos

Hey, give them another chance, Escyos. At least Jack was able to tell us something, y’know.

I’m upset that a month has passed, too, but we got an update from Jack, which is better than nothing, y’know. I understand you hate ShoutWiki now, but some are willing to forgive and forget, y’know.

@UxieLover1994

I understand that it was not their fault, but when something you are paying for (shoutwiki paying for their servers) then you get on it quickly.

They lost some data because they didn’t back it up and thats whats taken so long to get back. I don’t hate them, I am annoyed that they took so long. And one update is not good enough, now if they posted updates everyday then maybe that would be better, but they didn’t.

A one month failure is not good nor is it excusable. They knew that it was taking longer than expected yet only gave up an update once every week or so.

I’m not giving them another chance, I subscribe to a one-shot policy with everything in my life.

@Escyos

Yes, it’s fustrating, but I’m not giving up on them (yet), since they are still under constuction with their community, y’know. (Yeah, I talk like Raijan from Final Fantasy VIII)

I’m planing on moving to Wikkii should this last more than two months. You can move if you want (I’m being friendly), but some will stay, ether because they like it here (WikiSimpsons) of that they cannot afford their own host (Spyro Wiki, which is my homebaise).

@Everyone else

31 days… don’t bring it to 40!

@Escyos Would we listen to Queen, Beatles and other excellent groups, for whom noone gave a crap in their beginnings, if everyone had one-shot policy?

What happened to the wikis coming back LAST Saturday. They’re probably not even going to be back by THIS Saturday. Seriously, if you’re gonna say it’s gonna be up by a certain time, it needs to be, or you lose credibility thats kind of what’s happened here. Now, I’m not moving, but if anything like this were to happen again, i will.

@Shoutwiki Staff: If you guys can change your datacenter provider without adding downtime I think that would be best.

All of you above (especially Escyos) need to shut up and get your facts straight before talking crap. Firstly, this crash wasn’t an error on Shoutwiki’s part. FYI, their systems were running better than fine until their hosting company screwed up. Yes folks, that’s right; their HOST COMPANY, not Shoutwiki, screwed up! So if you want to rant at anyone, go rant them. The crash was not in Shoutwiki’s control.

Furthermore after the crash occurred, the Shoutwiki staff indeed (as you would expect) asked their host for details in problems behind the crash so they could know what to do specifically to recover. News for you folks, their host refused to give out any of that info and simply gave them a “your problem, fix it yourself” scenario (why, idk… because they suck, right?). Since the staff had no idea what has gone wrong, they’ve had to restore EVERYTHING from scratch. That should satisfy your query of why the site has been down for so long (yeah, wiki farms aren’t a simple wiki, there’s many wikis that can take ages to recover completely, do some research). Only one staff member has the appropriate access for the re-installation of the server OS, and when you only have one staffer working on such an issue, naturally the process will take a while. Its not (and can’t be) within anybody else’s control. And don’t just think that a re-installation for a server hosting a wiki farm is as simple as your average windows install; there are complicated issues (as well as crash problems) behind it that need to be overcome. Hacks can be done, but is it really worth risking another failure just for the sake of time? If you’re that impatient, write your articles on a text editor or on another wiki for previewing (without saving). Just because your wiki is down doesn’t mean you can’t keep working in the background.

And no, Shoutwiki isn’t closing shop, as per what Jack said above. Suggestion to you is to just wait at least a bit more, even if its been a long time (and its understandable why many of you are upset). I personally find it unfair that you’re ranting Shoutwiki staff for something that wasn’t their fault. Perhaps this might have given you a better insight as to why its been a long wait and open your eyes a bit wider to the situation.

@Nick I agree with just about all of this. But all three techs are working on this. Skizzerz has recovered the databases, and I believe Misza has access to the OS (Not sure, but Just assuming as he is the head tech). If the host basically said that’s not our problem, that leaves the staff in a difficult situation.

@Hockeyben My point exactly. And yes, Misza is the one with access to the OS. Its pretty much up to him to get SW restarted now, except that the underlying issue is that re-installing the OS isn’t as simple as one might think.

@Nick

I am aware that it is not entirely Shoutwikis problem, but they are still at fault, they lost data and have spent too long recovering the system.

Seeing as they are paying for the datacenter and they were the ones to screw up, im not holding them responsible, however 1 month+ is not forgivable. Simple as that.

@Escyos Dude, SHOUTWIKI DIDN’T SCREW UP! In any shape or form!

Blaming Shoutwiki is like shooting a person and when it dies blaming the paramedics, because they couldn’t resuscitate him.

@Nick

I agree with you, Nick. I know that it’s not their fault, but it was not until Jack gave us his update that I regained patience. It seems only one of us (Escyos) gave up on them.

@Escyos

I’m willing to forgive the Staff once the site is back on, but you ”need” to relize that it’s not their fault this crash has happened. Nick’s right, y’know! GROW UP!!! These things happen, and I’m willing to forgive EVEN if it lasts A WHOLE YEAR!!!

FWIW, in this industry relying on your host to keep the server backed up and not having automated daily backups of the entire server IS the company’s fault. It’s inadequate protection of your customer’s data. If ShoutWiki were actually being paid for professional services their customers would have rightfully demanded a refund and found a new host.

FOR THE LAST TIME I AM NOT BLAMING THEM FOR THE CRASH!

I am blaming them for the lack of progress, notification of progress and inadequate back ups.

LEARN TO F**KING READ!

@Escyos

Do you want this to turn into a fight!? Ether get patience or go find a new host and leave your wiki behind! I AM SICK OF YOUR WHINING!!!

@UxieLover1994

I was merely correcting people who are misquoting me. You would do the same.

Im already looking for a new host, and have found one im justing waiting for Shoutwiki to get it together so I can leave.

Intermittent outages and slow times followed by a crash which facilitates the loss of data. There not getting any better and if history has taught us anything its that they won’t. They screwed up by not backing up data. Simple as that.

@Escyos

Just remember: Not everyone is planning on leaving, plus you cannot make everyone leave. I’ll be getting a databaise dump every Sunday, just in case this place crashes again, and it cannot be fixed.

@UxieLover1994

Take your time!? It may not be a problem for you to wait 60 days, but Autopedia can’t. We’ve lost 32 days of car blogs (New cars, car updates, etc). Autopedia admin Red marquis isn’t even sure if we can recover them all.

@Siver Knight

Who knows; we might get back our wikis on Monday night, but if you start to become as inpatent as Escyos, then just create a new wiki and do the missing pages until ShoutWiki returns; then get your remaining pages. This is a tip for the inpatient.

@Escyos
If you have already decided to leave Shoutwiki, then stop commenting and just leave. Stop commenting and find something better to do than wingeing, and once your wikis are back, just go. However, if everyone had the same “policy” as you and left at the first sign of trouble, where would the world be today? Mistakes are always going to be made, but once they are fixed, they are usually never made again. People learn from mistakes, and put measures in place to pervent them, or at least stop them from causing as much damage as they did in the first place. If we all stopped doing something, or gave up on something when it first had a problem, we’d all still be living in caves. It’s people with rationales like yours who stop society from advancing.

Leave a Reply

Your email address will not be published. Required fields are marked *

food penis enlargement pills cbd gummies for inflammation male enhancement pills safety rejuvenate gummies for ed amazon probiotic gummies male enhancement pills 4 inches does cvs have male enhancement pills el toro gummies ed side effects of goli apple cider vinegar gummies consumer reports on male enhancement pills will cbd gummies help with ed male enhancement pills headache cbd gummies sex reviews tropical punch canna gummies e 3 male enhancement pills thc gummies side effects male enhancement pill victi ron maclean male enhancement pills best male enhancement pill side effects 6 months on penis enlargement pills joint relief gummies male enhancement pills and you meth male enhancement pills over the counter canada do extenze male enhancement pills work ultrabrand male enhancement pills how long do cbd gummies effect you male enhancement pills canada shoppers drug mart does truth cbd gummies work male enhancement pills high blood pressure gummies for low libido truth cbd gummies male enhancement pills with tiger prostate health gummies what best male enhancement pill what penis enlargement pills acrually work endocrinologists male enhancement pills

natural male enhancer pills whats the best penis enlargement pills male enhancement pills 2024 starts with a v black panther male enhancement pill rhino 7 male enhancement pills side effects trisha yearwood gummies reviews stim rx male enhancement pills normal dose cbd gummy arouza ultimate male enhancement pills reviews does cbd gummies improve ed biolife gummies reviews the top ten male enhancement pills no thc cbd gummies when to take cbd gummies for anxiety penis enlargment pills for girth 3d black panther best male enhancement pills livalis male enhancement pills side effects rigid rx male enhancement pills where to buy power cbd gummies near me choice cbd gummies all natural hemp extract 3ko male enhancement pills gummies cbd for sex king cobra male enhancement pills natural sexual male enhancement pill male enhancement pills cialix biolife cbd gummies erectile dysfunction keoni cbd gummies price cost of regan cbd gummies vitamin gummies iron prime cbd gummies price full body cbd gummies for enlargement boner bears work

cbd calm gummy stimulus cbd gummies are cbd gummies legal for teenagers vibez cbd gummies customer service number cbd gummy make me feel swimmy head do cbd gummies cause constipation full spectrumhemp gummies with cbd karizon inc cbd gummies cbd wellness cannabis oil gummies non psychoactive cbd gummies where to buy green lobster cbd gummies sour gummy bear cbd edibles near me are cbd gummies safe grown md cbd broad spectrum gummies rejuv boost cbd gummies the five cbd gummies cbd gummy white label cbd gummies for joint health cbd gummies dosage guide keoni cbd gummies stock cbd gummy with melatonin green earth cbd gummies golden harvest cbd gummies total cbd gummies rx cannaleaf cbd gummies spectrum cbd gummies diabetes best cbd gummies on amazon for anxiety best cbd gummies for weight loss uk gummi bears with cbd oil price of royal blend cbd gummies cbd gummies have weed cbd adhd and bipolar gummies

best cbd gummies for stress and pain redeem therapeutics cbd sleep gummies do smilz cbd gummies contain thc truth cbd gummies price where can i purchase natures boost cbd gummies proper cbd gummies dose cbd gummies for pennis growth near me what happens if cbd gummies melt sarahs blessing cbd gummies cbd gummies vegan uk cbd gummies uk wholesale shoprite cbd oil gummies cbd gummies for sleep 2024 rx select cbd gummies square care cbd gummies ultra cbd gummies diabetes edible gummy bears cbd cbd gummies naples cbd gummies amsterdam lunch box cbd gummies how many milligrams of cbd gummies does cbd gummies contain thc cbd gummies make u high kenia farms cbd gummies cbd gummies queens ny cbd gummies for tennitus cbd edible gummies effects cbd and thc gummies reddit prime cbd gummies amazon amazon cbd gummies hemp bombs does cbd gummies make ur dick hard cbd gummies to help quit drinking cbd gummies fox news cbd gummies high quality global green labs cbd gummies can anybody buy cbd oil and gummy vitamins best cbd gummy for weight loss quit smoking cbd gummies reviews

reviews for slim dna keto gummies golden gummies weight loss oprah slimming gummies reviews how to use acv keto gummies kelly clarksons keto gummies premium blast keto acv gummies reviews is keto blast gummies for real turbo keto gummies shark tank reviews slimfast acv gummies keto gummies max science do apple cider vinegar gummies work weight loss gummies hydroxycut keto vitax gummies stiftung warentest bewertung dangers of weight loss gummies sureslim keto gummies reviews ikaria gummies for weight loss which are the best apple cider vinegar gummies gemini keto gummies review pro burn ss keto gummies best time to take apple cider vinegar gummy oprahs keto gummy safe gummies for weight loss weight watchers gummy shark tank keto blast gummies reviews are goli gummies the same as apple cider vinegar dischem keto gummies south africa

how to make gummy edible slime how to take keto luxe gummies lets keto gummies ingredients weight loss gummies do they really work xtremefit keto acv gummies reviews cross fire keto gummies how to gummy bear slime acv keto gummies 1050 mg slimming gummies for weight loss do gummies for weight loss really work nutra haven keto plus acv gummies does 6 pack keto gummies work tru bio keto gummies side effects fitlife keto acv gummies reviews best time to take goli acv gummies how to make gummy bear slime with flour keto pink gummies envy fit advanced keto gummies what is keto fusion gummies acv keto gummies reviews reddit keto bhb gummies shark tank ketosium xs acv gummies weight loss gummies uk goli apple cider vinegar gummies age limit oprah slim candy keto gummies algarve keto gummies reviews kelly clarksons weight loss gummies review of royal keto gummies is oprah selling weight loss gummies diet gummy bear korea