jump to navigation

False Alarm May 31, 2009

Posted by ismywebsite in general.
Tags: , , , , , , , ,
add a comment

I have excellent news. Node 3, managed by OSHS, which has now only had one instance of downtime, will continue as normal for the time being. After prolonged discussion with the company for the last half of this month, we received the following reply:

Thanks for your long messages.

We confused your account with another account so our apologies for that.

For the time being you can remain [here]. Should any problems arise we will keep you informed.

Thank you for your business. It is much appreciated.

So, in short, the final result is an even faster and more reliable service, now that the server has been cleaned of a full third of all accounts there, and we have planned a further clean out of another third of the accounts, all of which are either from the system we used prior to November last year, or appear to be under construction.

Node 4 is still being dropped, and interesting enough, the company has now decided to triple the costs as well. So it’s a good thing I’m leaving when I am. We have backups taken already, and the new server is just about ready with your new cPanel accounts. We encourage you to still take regular backups if the contents of your site are dynamic, and/or to warn users that content over the next few days may be lost. The old cPanel will not be destroyed, but will eventually go offline on the old host after our term expires.

Bringing Our Hosting Back To Earth May 23, 2009

Posted by ismywebsite in general.
Tags: , , , , , , , , , , , , , , , , , , , , , , , , ,
4 comments

Hey everyone,

Well, I hate to not have much notice for something this major, and this has been in the works for quite some time, but just now it has been finalized. We will be dropping Galaxy Hosts as our service provider for Node 4.

I understand that many of you will probably want to know the reasons, and there are many. Here’s just a quick tour of the service we experienced from this company from various angles:

Reliability
One thing could always be counted on. There would never be a consecutive string of 3 days without downtime. We did achieve a couple, and at least we haven’t dipped below 90% in average overall uptime. This was largely due to server crashes from a large server load, yet we offered many practical solutions that would help solve the problem. None were accepted.

Security
How’s a major security hole, enabling anyone to gain the equivalent of root access? Well, in due fairness, they did fix it, eventually, after we pointed it out to them and gave them some guidance as well. For a company that’s been at this 2 years, they overall did not appear very competent in the area of server security and handling, and also refused to accept any help or suggestions, which would have really been mutually beneficial.

Features
Unlimited bandwidth, quota, and features were mandatory, due to a cPanel glitch. Despite our being well within our limit of space used, they would not enable overselling to fix the glitch, despite our repeated requests to upgrade to an account with it enabled. It wouldn’t have been so bad if the Addon and Parked Domains did not default to zero, but alas they did so we had to move many users off for that very reason. Not impressed at all.

Support
For the first week or so we were using them, support was excellent. They welcomed us to their hosting, and warmly greeted us. Finally, they realized we actually wanted to use the full extent of what they offered, and problems started to creep up. Before long, there wasn’t even a working support panel to communicate with them, and you couldn’t always be sure they even got your message.

Many of you will have personally experienced many of these problems, and many of them build on one another. We find this service is simply not of sufficient quality, and you as our clients deserve better. Lucky for you, it’s really not all that difficult to find a higher quality service. I could look just about anywhere.

What this means for every user on this node currently, is the following:

1) You should already be taking regular backups of your account, especially on an unreliable node such as this one. If you don’t know how to make backups, there is a tutorial in the ‘Tutorials’ section of your account. Be taking backups daily throughout this period. We can restore your account, and have plenty of hosting space, but we may not be able to restore any lost data.

2) For all users who have valid ads placed, we will be (hopefully) relocating your accounts to another node. I will be assembling a team of volunteers to assist with this. If you are interested, and you’ve submitted a volunteer application, then just reply to this blog post.

3) If all goes well, we will start transfering the accounts to our newest Node 7, which should be set up shortly. The transfer will happen through May 25th and 26th, and will only happen to accounts with valid ads placed. Nameservers will need to be updated if you have a domain. The new nameservers are as follows:

ns1.ismyws.com
ns2.ismyws.com

4) All remaining accounts on this server will be permanently gone on May 27th, or within a short period of time thereafter. If your account does not have valid ads placed, we offer no guarantees of any kind. All clients should have ads placed, and if you have difficulties with this, just ask and we will be glad to assist.

All Systems Go! May 2, 2009

Posted by ismywebsite in general.
add a comment

Node 2 Back Online
Alright, so I’ve talked to Felweb now and we are working to sort out the issues.

We would like you to know you have a lot of users that are correctly using the service, but have a lot who do not update their scripts and are [full] of [security] issues. We are right now unsuspending your accounts, [and] we would like you to please do all the clean [up] and let us know when you are done. We will then inspect the reamining accounts and delete any suspicious file.

I have also received details of the exact issues we need to resolve. The most important thing right now is for all users of Node 2 to:

  • Ensure all of their scripts are up to date, with the latest versions. This attack resulted from vulnerabilities within your scripts.
  • If you have content on your website, ensure it is accessible from the main domain. Set up an entrance page, or a redirect. Either will work. We will be deleting empty accounts in the near future.

(These are both good ideas for users of other nodes as well.)

Node 6 Back Online
Will still be a short while before the DNS is fully updated on all websites, but it is currently in progress.

Help Wanted May 1, 2009

Posted by ismywebsite in general.
2 comments

I’ve decided that the best path to getting on top of all our problems is with qualified, dedicated volunteers. I’m currently accepting applicants here:

http://www.ismywebsite.com/volunteer.php

Of course support is bad when I can’t answer it, new websites aren’t accepted, and nothing gets done. If you’re not willing to help solve problems, you have nothing to complain about.

(And yes the application is very long so only qualified, dedicated people like you can get in. I’m ready to get serious, and I don’t need false promises.)

I haven’t received a reply. Was I accepted?
It’s not a matter of if, but of when. Applications that have more details will be chosen first to assist, while others will be chosen once those volunteers are trained and in position.

Did you receive my application?
There have been no reports of applications going missing that have been confirmed to be true. I have also extensively tested the system, and I have dozens of applications coming in at the moment.

When are you selecting the final team?
There is no final team. The exact structure will be slowly built up, volunteer by volunteer, as they are trained and assigned to their positions. At present, we have staff in place to process all email messages, and they are doing a great job of it. Once this group has had a chance to work together for a while and is working stably, selection will begin on the next phase.

Why do you need so much information?
The largest reason is to determine your dedication and competency. How you reply to those questions is a good indication of how you will reply to real situations once you are on the job. Each question has a specific goal in mind, or character trait that is being determined. Different traits are helpful in different positions. Each question is optional, but a refusal to answer any question may be used against you.

Node 6 Moved ‘Silently’ May 1, 2009

Posted by ismywebsite in general.
add a comment

On another such issue, not forwarded to me:

SERVER MIGRATION IS COMPLETE. PLEASE READ BELOW.

During the time of migration, your sites may have lost some information because once the migration is started and anything new you may get once it starts is lost once the migration is completed. You would of lost data from 4/19/2009 to 4/21/2009, which is only 2 days.

In order to not lose your most recent information, we would like to encourage you to log into the old server to get your information back. You can log in with your same login information, but make sure you log in through the right server i.p.

Old Server I.P: 216.45.59.90

Node 6 appears to be located at a new IP address, although Nixism hasn’t been completely clear on what this new IP address is. I have, however, managed to track it down on my own.

So, in just 48 hours all websites should be working at the new location.

Now, just to wait.

Felweb Changes Owners May 1, 2009

Posted by ismywebsite in general.
2 comments

So, as the story goes, sometime over the last 2 weeks while I was busy with exams, something came in that might have been important. It looked like this:

Hello,

We have some issue regarding security because of some WordPress out of date and some Joomlas out of date. We will be performing a Security Audit tomorrow, so we are asking for your collaboration in letting us know which accounts your users are not using so we can terminate them and avoid useless work.

We appreciate your colaboration. Regards,

Felipe Diaz
FelWeb Network

Oddly enough, no one from support thought to tell me, or even star it so I might chance upon it and actually do something. This was April 23rd. Three hours later,

Do not worry, this has been done by us already. All infected accounts had been terminated due to our TOS. This accounts were not up to date on their scripts making a security hole for our server, and indeed today we had a vulnerability. This vulnerability was trough an account so we needed to do this for the datacenter in order to keep things running as normal.

And then,

I am sorry if th[ese] messages are bothering you. Please have all your users update their scripts, as your users are the only users who have been exploited. The exploited files are normally seen as numbers.php (example 3846293.php) as user nobody and are always under a 777 chmod folder. So please make sure your users chmod maximun on 775.

Nothing important here either, I guess. All resolved. Followed by an invoice, also unstarred and unmentioned. And not important either:

We have implemented the new PHP handler SUPHP to achieve a higher security performance on the server reducing 90% of the vulnerabilities in scripts. Also, this handler will eliminate the problem of files being owned by nobody instead of your group user making the file ineditable, so this time you will be able to eliminate yourself any suspicious file.

It is important that you do check on all your client´s websites, to confirm that all of them are working as normal. If any website do not work as expected you need to check the next things before contacting us:

1. Is the script downloaded from the internet? If it is, is the version installed the last one available?
2. Do you have folders with 777 permissions? If so, please restore them to 755 and your scripts will continue working with it, but without the vulnerability of getting exploited injected files.

Should you have any question please do not hesitate in contacting us.

Or this,

Greetings,

Despite our measures this morning the server was attacked and vulnerated. More than 50% of our clients are affected with this downtime because we had to take the server down for reinstallment. If you are not affected, please do not read this message.

We are at this moment installing the OS and cPanel. All information is stored and will be transfered back.

Estimated Resolution Time: 11 PM

We apologize for any inconvenience this may have caused.

Felipe Diaz Castellar

Finally, I did received word of this and it was starred:

Hello Matt,

We are sorry to announce but the new owner of FelWeb Network had decided not to host free hosting providers anymore due to the insecure matters it involves. You can request any backup you would like within the next 48 hours as maximu[m].

We thank you for your always preference on us and I was personally happy to work with you when I was the owner. That is why I had to tell you myself about this notice.

Best regards,
Felipe Diaz

So there you have it, ladies and gentlemen. This is why your accounts are not working anymore. I am very sorry for not taking the time to notice something like this, but in all honestly I definitely think we need either an improvement in the people handling support, or an improvement in the people handling support, and I can’t do it all, all the time.

I’ve dealt with this now, doing the best I can do, and fighting for 10 last days that we can use to transition properly into in a new server. Such ends our 1 year relationship with FelWeb, through good and bad. For you, there are 5 other servers, soon to be 6, and you need just ask to get an account on any one of them.

For now, exams are over and I’m back to sort out the mess that always becomes of this hosting when I can’t be there to clean it up. First up, establishing a team.