Categories
General Technical

Downtime since June 17th

Hi everyone,

this is a blog post I wished I never had to write, but here we are again: the server’s down. It’s frustrating for everyone involved, and we’re doing our best to bring it back online, but it seems that it’s going to take a while.

I want to apologize on my own behalf as well as on the behalf of the whole staff team about our poor communication during this downtime. We realize that many of you would like to hear an official word about what’s going on when the site’s down and we were unable to provide that to you in a timely manner, which is not acceptable. Rest assured, we will definitely aim to improve the communication between the staff and the users once the site’s back up again.

What can I say? We believe in our mission and we have no intention of “closing down the shop”, despite what other sources may claim. But it doesn’t change the fact that we don’t have people 24/7 to work on this issue. Downtime is always a critical thing, but this seems to be something only a few select people from the tech team are able to fix. We want to bring the site back up as soon as possible without losing anyone’s important data. Needless to say that it’s not the fastest possible process in the world.

The techs are busy working on our new, more stable server, which will be located in the United States of America. This will replace the Polish server as our main webserver and bring the much-needed stability. This server was planned to be our backup and was close to being ready to be deployed for production when these new issues occurred, we are now changing that plan.

I want you to know that we haven’t given up — and we have no plans to do that!

Jack Phoenix
Head of the Customer Support Team

UPDATE 1: To clarify a few points. This is NOT a repeat of last time. Last time the server itself crashed and needed to be restored from a backup, and said backup was corrupted. This time there was no need of a backup to be restored, and as such there shouldn’t be any data loss, although we cannot confirm this for sure. The main issue here is not recovering databases, but rather moving everything to the new server, which should prove to be much more reliable than our old one. While we’re working on moving everything over, I’ve managed to set up a hack to make MySQL behave properly and actually display the wikis, albeit in read-only mode. Please note that since this is a hack, you might see the same “unable to connect to socket” error as before — this is temporary and if you reload the page in around a minute you should see it. –Skizzerz

By Jack Phoenix

I'm the head of ShoutWiki's Customer Support Team.

151 replies on “Downtime since June 17th”

@Bryce53 – I would rather edit my wiki. Its now for so long uneditible, that I start to thinking about closing my wiki and merging with Wikipedia.

Update please, you guys? How long until I can edit my wiki again? Just an estimate would be better than nothing!

@Alexander – If you had read through the comments/updates, you would understand why the move is taking so long. ShoutWiki techs are volunteers, thus not paid, and thus do this in the time they can get. They didn’t plan this move, so they couldn’t plan with their techs to have enough free time to make the move.

And honestly, come on everyone, I think we’ll get an update when they have a chance to give us an update. I come here to read information about what’s going on, see the progress, and whatnot. I don’t come to the blog to read through the comments where people are, and excuse me for being rather blunt, but whining and crying about wanting an update. These people have other things to do as well, so come on guys, relax and be patient. You’re not the only ones who want to edit their wikis again. I would like to as well, but I have enough patience to WAIT.

Seriously, give them a break. They’re trying. =/

I can currently not give an estimate that would be accurate. As much as I would like to give an estimate, making an estimate right now would require me to make a guess based on unknown variables, and I just don’t want to risk being wrong and causing all of you to get your hopes up for nothing.

From what I understand the move is almost complete and the techs are closing in on the last few issues. If their test run after that goes well we should be back to editing again soon. But unfortunately I cannot predict if the move causes new bugs, or if it causes other fixes to require tweaking.

Let’s wait…

Great job, ShoutWiki! I’m impressed on how well you handle this situation. Although Wikia is more reliable server-wise, everything there (staff, community, etc.) is downright horrible, and I’d rather have an unstable server over all of that.

If ShoutWiki can be reopened within two or three weeks from now, that would be impressive. I have the amount of patience to wait.

Oh, and BTW, ShoutWiki is a relatively new company that is proving to provide excellent service so far. They were trying there hardest to solve issues caused by the Polish datacenter they previously trusted. If you don’t have the patience to stay calm and wait, ShoutWiki isn’t really a good choice for you guys. If you’d like something that would always be reliable, but be provided with horrible support and community, go to Wikia. There.

[[Special:Export]] is working, at least. You can export the contents of the wiki as a backup, like Ciencia Al Poder has already pointed out up there.

Best regards.

p.s. is Monaco a paid feature now?

Bryce53, Monaco was going to be free until August, although I’m now guessing that this’ll probably be extended again due to the time lost.

@Hockeyben, it is not set in stone. I’m just guessing that as that is what happened last time there was a crash.

Sorry Hockeyben, but unfortunately, you’re *not* going to get an update as frequently as you’d like to; you already know that the move is almost complete and if the test runs smoothly, we will be back to editing soon.

August 1 is a good time for editing to come back, but I doubt that it would be that *short* because they need to test it. I can wait two, three weeks, even I have the patience to wait for two months. Patience allows the feeling that editing will be back soon.

@Alexander
As I have stated above, this is not just any move. A lot of fixing is also involved and we want to make sure and double check that everything is done right.

On top of that it is vacation time in most of the world, and inbetween planned vacations and summer jobs, free time to do all this work is limited. Had we planned this move we would have cleared our schedules and done nothing but this move for a week or two and be done with it.

But we really felt, and still feel that the longer we stay with the Polish datacenter the bigger the chance of yet another screw-up on their end with again disastrous results for us and all of you. That is something we are not willing to accept, so the decision was made to go ahead with move even though we do not have the time we’d like to have for all the work.
I am hoping the techs have a bit more time this week, and we will keep you updated.

I just hope that you guys have chosen your server more carefully this time. I remember when you were moving to the Polish server in November, how it was praised as “better, faster, and more reliable” than your old one. That wasn’t the case. Just hope that you’ve chosen more carefully, and that this will be the last of these agonizingly long outages. Also, will we get Monaco for free for the days lost?

Hopefully the site will be back by August 10th.

@Hockeyben
Every day you write same comments and ask same questions. ShoutWiki won’t be back any faster even if you shout here every day. Techs are doing their best, but they need own time too. Don’t give them deadlines, it’s disturbing them and us all. We all want to edit again, but you have to have patience. I don’t care if it takes another two months, because we need a stable ShoutWiki. And they write updates if they have time or reason, not because you demand them. Patience, man! Please!

I have had enough with the users who are impatient because of this cause and demanding updates. If you really aren’t patient enough to wait, ShoutWiki isn’t the wiki farm for you. Go to Wikia. There will always be a stable server, but the quality of the wikis, especially the reading / editing space and skin, will be mediocre.

It may indicate that the Polish server is being deactivated and that the American server will be activated after that…

Gosh, this is so frustrating. I know bitching on blog won’t help but it’s the only thing I can do atm.

If you’re fed up with shoutwiki’s downtime and you choose to bitch on blogs, ShoutWiki is certainly *not* the choice for you. Instead of complaining, go to another wiki farm, so you don’t have to complain anymore.

‘Nuff said here.

Any update why the site is down? I want to move to an other now, but I don’t know how I import all my files

Alexander: Use [[Special:Export]] on the shoutwiki wiki with all pages. Remember to activate the option for retrieving the ful lhistory and not just the last revision.

Of course, if [[Special:Export]] actually works on that wiki since it’s getting weird database errors. It was already said in these comments that all of you export the wiki so at least you don’t lose all the information in case ShoutWiki never recovers again.

Weeeeell, I’m back from a month of vacation. And it seems I’m not back to praise Shoutwiki, but to bitch as well. Don’t want users bitching at the blog comments? Close them or delete them, just as Wikia would do. At least here we have a (small) window to vent out the frustration.

In just two days it will be two months since the downtime started. Right you are I’m going to use Special:Export. And you can bet I’ll import it on another farm, as soon as I can find one. I was thinking about Referata…

I’ve read a few voices stating two things:
1) Have patience
2) If you don’t like it, go to Wikia.
I haven’t seen any staff voice against these advices. I know that’s not enough to assume they endorse it, but it certainly doesn’t increase my confidence. Could some Shoutwiki staff please give an official stance about it?

About patience: I think two months is more than enough patience. The mere fact that users are posting here means they’re still coming, trusting you to have their wikis up and running. And that despite the recent history of troubles with the service. Don’t disparage them so easily. They’re already having patience. The ones who haven’t have moved or forgotten (or worse, they have considered lost) their time investment here.

And about Wikia, I can assure you I’m a dissatisfied Wikia’s customer. And I try to do what I can about that. I passionately hate the new skin, and I was one of the user’s campaigning against it. But they haven’t heard us, so I’m trying to move out. I tested Shoutwiki, and as far as I’m concerned, they’ve failed my test. I just cannot commit my main wiki (which is *the* official wiki of a live MMPORPG) to a provider who cannot guarantee uptime. And I’m not asking for 99% uptime. Just 90% would be fine. But Shoutwiki has been up for editing less than 50% of the time in the last months. What can I do, pray tell me? I need a service, and Wikia offers it. I just need a reasonably reliable competitor to Wikia to switch to. Can Shoutwiki be it? Sadly, I think not.

I think that it’s safe to say this: we have not abandoned the site and we will not be “closing down the shop”, no matter what.

Now, let me clarify a few things and explain what’s going on: we’re working on transferring the wikis to a new, American server.
As far as I’ve understood, there are some checks that need to be made and some configuration needs to be updated at least before we can start using the new server.
Skizzerz would know more, as he’s the person working on the transfer.

So, yes, what I’m trying to say is essentially “have patience“. It isn’t much, but I can’t make promises on anyone else’s behalf.
I’m more than well aware that the communication between us, the staff, and you, the users, has been far from ideal during, well, pretty much every downtime.
It’s something that we definitely want to improve on.

Of course I cannot and will not recommend going over to a competitor’s site, but in the end, you have the power to make that decision: you decide if ShoutWiki is the right hosting service for you or not.

There’s a few things that we haven’t extensively discussed in the public and while I don’t feel very comfortable sharing these details, at this point I think it’s only fair that you know what’s going on.
Firstly, ShoutWiki is and has always been a for-profit. We, the staff, believe in our skills and we think that we’re able to deliver a service better than anything that has previously existed to you and we’d like to get paid for it so that we can buy our food, pay our rents and pay for our Internet connections…and of course, cover ShoutWiki’s expenses.
Secondly, we are not incorporated yet, but we’d like to be one day. It’s hard and expensive work, sadly.
Thirdly, while our staffers are among the most capable you’ll ever find anywhere, they are not paid, unlike Wikimedia Foundation’s staff or Wikia’s staff.
Yes, it’s a lot more motivating to work when you’re getting paid and yes, having millions and millions of dollars to spend on things is a going to help you to get to the list of top 1000 websites in the world, but right now, it’s not something that we can afford.

We have big plans for the future, which will hopefully include incorporation, among many other things. We at ShoutWiki hope that you agree with us that there should be a wiki hosting service which honestly cares about its users and we sincerely hope that you will be with us building that service today, tomorrow and the days after tomorrow.

Any suggestions for alternative fikifarms? I’d really hate to move tho. It would be third time for my wiki… But maybe just for making more pages and then importing them here when shoutwiki’s up and running again. That way I wouldn’t have to just sit and wait. Any experience on this “method”, anyone?

\(^o^)/ I have gained 60% of my patience again, ”’thank you”’! And we understand how you feel, not being an official website. 🙂

Well, I’m following this blog for a while ’cause I want to move my wiki from Wikia. I already checked alternate wikifarms and was not overly excited about their services, so I wait for ShoutWiki to be back, because I at least want to test it (till now I can’t even create an account).

Until today I missed some actual information from the staff, but Jack’s post above gives me new hope, so I’m willing to wait furthermore – hoping that my patience will pay off.

So guys, take the time you need to get a better server running which can prevent more problems; a stable ShoutWiki is worth the waiting, I think. And in the meantime I can write and save my articles and templates offline, if there’s nothing better to do 😉

@Ichinisanchigo, ShoutWiki is an official website, it just doesn’t have paid staff.

If I had millions of pounds/dollars to throw away, I would definitely throw them at ShoutWiki. Unfortunately though, I don’t.

The only way to get better than ShoutWiki (if you ignore the downtime that is) would be to go independent with a competent person as a server manager to look after the wiki for you. Unfortunately, many of you can’t afford that so sticking with ShoutWiki is your best bet in my opinion.

That being said, tomorrow is 2 months since the downtime. I’m sure no one needs reminding but still, be patient. There can’t be that long until all the bugs are squashed and ShoutWiki is back to it’s former glory.

Leave a Reply

Your email address will not be published. Required fields are marked *