IsMyWebsite Free Hosting Development Blog

First Official Contest on Forums! November 30, 2010

Looking for the next way to help IsMyWebsite, and earn credits? This just may be your chance! Check out the “CONTESTS” forum for more information.

Making The 13th Impossible November 18, 2010

Something awful happens the 13th of every month, part of Murphy’s Law. I am aiming to keep out Murphy next month. Here’s how the current disaster can never happen.

1) Clearer Communication
I am working on some strategies to improve our communication, including making it more convenient for our team to email eachother, easier to check that email, and groups for departments. Thus, it is possible to email the administration, in entirety, without having to remember everyone’s name. This will make it easier for clients who need help, which isn’t being provided by support, to contact us. It will also make it easier for us to keep in touch with eachother. I am also establishing more solid policies, which are mandatory for all volunteers to follow.

2) Full Website Change Logs
Every single action, from a registration, website creation, change of account setting, cPanel access, modification of cPanel, and of course deletions or errors, will enter a log, along with complete PHP code to undo or redo that action. Thus, if it ever happens that thousands of accounts are removed, we know who did it instantly, and can begin a discussion with that individual without having to guess who they are, and ask around. For this to work completely, raw database access will have to be completely disabled. Nobody will ever have access to the database, and all actions must happen through objects, sets of code which securely handle commands, and ensure they get appropriately logged.

Thus, not only is it possible to know who made changes, but they could be restored in 5 minutes without affecting any current clients or data in any way. Not even losing 5 minutes of changes by restoring a 5 minute old backup.

3) Automated Daily Backups
The largest barriers with this are about the storage of the backups, as well as getting a full backup in DirectAdmin. As an alternative to purchasing commercial storage, I am seeking three individuals with a shared hosting account, or server, on another host, with the ability to spare 1 FTP account, and approximately 4 GB of storage (leaving room to grow to at least twice our current size). To prevent any misuse, backups will be compressed, and key data will be encrypted. Passwords, specifically, cannot be retrieved, using a complex set of md5s, with a dynamic salt. If you can decrypt this, show me how and you get 200 credits:

15ed39ee9a37409832687df743c8d8a6

As for cPanel passwords, which need to be used for FTP (for example), they will be stored with a reversible encryption, however, at the client’s option they can be automatically changed every night just after the backup is taken. I will see if it’s possible to omit them from the backup entirely as well. (In this case, you would get a new password if the backup was restored.)

If you’re able to help, then send me an email and I will consider you. Please include the host you are using, recent uptime statistics (if you have them), how long you’ve used them (or been in existence), and your plans for the future (how long you plan to continue keeping the website running). If you can host more than one version of the site, let me know as well.

4) Tightened Hiring Policy
New rules surrounding volunteers will be implemented, including that every volunteer must have used our service a minimum of 6 months, they are required to actually host websites on the service, and they should theoretically place them on the worst servers, because this, I feel, is the only way to truly ensure any problems get fixed. For the record, my websites are currently on every single server, with one of the most important on Node 1. In this particular case, there was no issue here, however another recent event has brought this to light.

5) New cPanel Access Controls
On the subject of volunteers accessing client cPanels, it’s fully necessary. There is no other way to detect many types of abuse, or fix issues with the accounts. It is not a good idea to store passwords unencrypted in plain text format, no matter where you store them. So please make sure you are protecting important critical data in your account.

No volunteer will ever receive the actual cPanel password. Instead, when they wish to access your account, the password will be changed to a temporary password. This will be logged, including the reason for access, a log which you, as the client, will be able to see. When finished, the password will be restored.

6) Review Process For New Clients Only
I think it’s quite ridiculous that trusted clients still need to wait for their websites to be reviewed. The invite system is designed to keep evil-doers out. It’s a good idea to actually know the people who you invite. Despite adding costs, people are still inviting largely random people in many cases. Do remember you are also responsible for those people who you invite. If they abuse the service, we will come back to ask why you invited them. That said, I want to encourage everyone who has friends who can use our site, to invite them when we get to the new version. I’m preparing our marketing department to heavily promote it then.

What I’ll Need November 17, 2010

Okay at this point I’ve kind of given up on VectorLevel. They keep claiming to be able to help us, but when I ask for them to deliver I’m just not seeing the results.

Promise #1 – Ticket Marked Urgent – After 12 Hours

“Hi there, we are looking at what we can do for you and will get back to you as soon as possible. Admittedly our backup strategy is designed for catastrophic restores than individual restores but we think we can pull something for you anyways. Thanks!”

Promise #2 – After 2 Days 22 Hours

“Can you tell us t[he] database name please and we will restore it? What sort of date are we looking for?”

Promise #3 – After 3 Days 8 Hours

“You should be ok now. Let us know if everything is ok. Thanks!”

Promise #4 – After 3 Days 22 Hours

“Thanks Matt I will get that restored now and keep you updated.”

So basically, I’m in a very tricky situation. I can go through and restore one by one the accounts affected, and the websites, a very lengthly process, only to have some of it removed when the backup goes through. Or the backup could never come, in which case we have to do that anyways.

At the moment, I can’t be certain of much. Any PMs I answer, accounts I restore, data I modify, could all be gone tomorrow and though I’d have a backup, it would be tricky to restore only select entries of it.

I’m working out an offsite backup strategy for us on HostGator. Right now I just need some place that will allow me to automate a FTP upload of the backups we take. I’m aiming to have three places, for redundancy. I also have to work out how to encrypt the backups so nobody could possibly retrieve for example client emails, without first decrypting. Account passwords are already encrypted on the new site.

Developments November 16, 2010

1) The backup of the site currently is from October 10th. So where did I get the beginning of the month from? Why directly from a Skype conversation with Brendan:

[1:45:35 AM] Brendan McNiff (Cobrastrike – ADM): Unfort[u]nately, I did not think to take a backup before restoring.
[1:45:47 AM] Brendan McNiff (Cobrastrike – ADM): That was an error on my part. I admit that.
…
[1:46:10 AM] Matt McGirr (Azoundria – Adm): If there’s a possibility your backup will have useful data we need to explore it
[1:46:35 AM] Brendan McNiff (Cobrastrike – ADM): Let me do a current export and see what is different in that header.
[1:48:53 AM] Brendan McNiff (Cobrastrike – ADM): Appearantly 10.25 is the date there.
[1:50:07 AM] Matt McGirr (Azoundria – Adm): Hmm
[1:50:13 AM] Matt McGirr (Azoundria – Adm): When did you delete the users?
[1:50:41 AM] Brendan McNiff (Cobrastrike – ADM): Approx 11/1
[1:50:49 AM] Brendan McNiff (Cobrastrike – ADM): I believe
[1:51:10 AM] Brendan McNiff (Cobrastrike – ADM): 11/5 at the latest

2) The backup at a later date which I took, was of the new site’s database. The last backup I took I remember like it was two weeks ago, but it was in September.

3) I’m not mad at Brendan for stepping up and taking initiative about removing inactive users. For sure, there’s a difference of opinion on whether that was a good idea or not. What I’m upset about is not knowing who or when it was done, and that it would appear, a backup was not taken directly before it happened either.

4) VectorLevel gave me this nice reassurance on the 13th of November:

Hi there, we are looking at what we can do for you and will get back to you as soon as possible. Admittedly our backup strategy is designed for catastrophic restores than individual restores but we think we can pull something for you anyways. Thanks!

As of this moment, they have not responded further on any details or procedures available. Is anyone else having trouble accessing their main site at http://www.vectorlevel.com/? (No problems with their client area.) HostGator has weekly offsite backups offered as part of their service, and I’m quite certain they can offer a restore within 2 days.

5) Since this time, Stefan thought it would be a good idea to update the DNS server, thus ensuring any websites which got destroyed in the database are now offline.

It’s my fault, I did it as usually. I wasn’t thinking it would affect the current websites. Since it was my mistake I’ll add all websites manually if it’s needed but I need to know how.

The procedure is as follows:

(a) Have the client recreate the website.

(b) Accept it onto the same server it was created on.

So Stefan can be reached via PM at the username ”Gatz”. I’m sure he’ll be more than happy to help you. I’m quite loaded with messages at the moment.

Why Is _____ Gone? November 13, 2010

____ could be any account, website, or message created or modified within the past 2 weeks. It does seem we have a bit of an issue.

At the beginning of this month, Brendan decided that all our older clients shouldn’t appear in our database. I disagreed because I felt that we could reclaim a good 10% when we are, what I would call, ready for them.

I’m not mad he removed them. I’m mad that nobody told me this happened, or who did it.

Two weeks later, when I figured out it was him, I discussed my issues. He happily restored all the accounts. Case closed in his mind.

Turns out, his idea of restoring the old users was overwriting our current database with the one from 2 weeks ago, that contained the old users. Unfortunately I neglected to mention this tiny issue to him, and he was having a “bad day”.

Now, this is just a bit complicated to sort out. All we have to do is grab the latest database, and sort out the changes from the old one. A bit tedious, but computers love tedious work.

Except, there’s one catch. While Brendan was diligent about making a backup the first time around, not so the second time. The last backup he took was from the first time around. And I’m no better. Our fallback appears to be VectorLevel if they have a backup.

So we need a strategy in this area. I have some ideas, but I’m open to suggestions. This will be item #28 at the conference tomorrow.

Thus, if anyone messaged me I didn’t get your issues. So we will have a period at the beginning to get all other issues. And then after that, you need to message me via Skype. Or you can email me to my IsMyWebsite email.

Technical Difficulties (Resolved) November 13, 2010

As of 8:28am GMT all issues with the main site are resolved.

Daylight Savings Time November 7, 2010

Hi Everyone,

I just want to quickly apologize for not factoring Daylight Savings Time into the time I gave you guys for the conference. I debated and decided the best solution was to start it an hour later, since the worst case is someone arriving early for it then, and having to wait an hour. So the time is as follows:

GMT – 12 | November 13th 2010 5:30 AM
GMT – 11 | November 13th 2010 6:30 AM
GMT – 10 | November 13th 2010 7:30 AM
GMT – 9 | November 13th 2010 8:30 AM
GMT – 8 | November 13th 2010 9:30 AM
GMT – 7 | November 13th 2010 10:30 AM
GMT – 6 | November 13th 2010 11:30 AM
GMT – 5 | November 13th 2010 12:30 PM
GMT – 4 | November 13th 2010 1:30 PM
GMT – 3 | November 13th 2010 2:30 PM
GMT – 2 | November 13th 2010 3:30 PM
GMT – 1 | November 13th 2010 4:30 PM
GMT = 0 | November 13th 2010 5:30 PM
GMT + 1 | November 13th 2010 6:30 PM
GMT + 2 | November 13th 2010 7:30 PM
GMT + 3 | November 13th 2010 8:30 PM
GMT + 4 | November 13th 2010 9:30 PM
GMT + 5 | November 13th 2010 10:30 PM
GMT + 6 | November 13th 2010 11:30 PM
GMT + 7 | November 14th 2010 12:30 AM
GMT + 8 | November 14th 2010 1:30 AM
GMT + 9 | November 14th 2010 2:30 AM
GMT + 10 | November 14th 2010 3:30 AM
GMT + 11 | November 14th 2010 4:30 AM
GMT + 12 | November 14th 2010 5:30 AM

If anyone has any doubts about the timing then please email me.

WebLyte Commitment To Uptime November 5, 2010

So I just wanted to let everyone know, that WebLyte is in complete agreement the downtimes on the 24th and 30th are unacceptable. They actually appeared to have been cause by CPU overusage from a couple of our clients, topsongs and redperuh.

We’ve let them know they are free to suspend any future clients who overuse CPU. It all comes down to intelligently designing your website for efficiency. And don’t use WordPress for anything beyond a simple blog. That’s 90% of the CPU overusage right there.

With exception to these two cases, WebLyte has maintained nearly 100% uptime. They have been fast and efficient in upgrading the server to have two new features, PDO, and MySQLi. We are quite impressed with the service thus far.

I do want to apologize to everyone who was on Node 4 before, during our fun history with GalaxyHost, Aquarius Storage, and IsMyReseller. This time, I am picking the server provider.

Conference November 13th November 4, 2010

I was cleaning up my Desktop and came across these goals we came up with one meeting back in August. I just wanted to reinforce the tremendous value of shared goals and a common picture. I know many organizations that have thought they were all headed in one direction, only to find out one day, the place they thought their teammates were headed, and the place they were actually headed, were two very different places. Really strong organizations can usually make it through that kind of turmoil, but at the very least it is a situation we want to avoid at all costs. So I fish this out again to remind everyone of the common goals we all share here:

1) Improve the world. Make a difference.

2) Be different. Change the face of web hosting.

3) Uncomplicate hosting. Provide fast, immediate help.

4) Keep all clients happy.

5) Keep a consistent service.

6) Answer all questions.

7) Provide the highest quality standard.

8) Be flexible enough to help any client.

9) All revenue will be reinvested.

10) Expand our outreach.

This is why conferences and the like are so very valuable to any group of people. They allow for issues to be solved, and everyone to be heard. One thing I got a lot at the last conference, was the question – “Why are there clients at this staff meeting?” Granted, I didn’t explain beforehand and I should have, though it struck me as quite exclusive and not an attitude of the team builders I know I worked with. So let me be perfectly clear. It’s probably more important we get clients at the conference than staff. Our team (for the most part) knows what they’re doing and know each other. But they don’t know all of you clients, and they don’t know the issues you face as clients. And those are what we need to fix more than anything.

Here is the list of items we discussed. The first 10 are from the last conference, along with some decisions we made. The rest, are issues we are discussing on the 13th:

1) Stefan Chiriac – Ways to keep our current clients.

1.  no wasting meeting
2.  make clients feel welcome at meetings
3.  organized meetings
4.  better support (usefullness and response time)
5.  Keep them interested by doing new things for Ismywebsite
6.  Feedback form, make it easy to find.
7.  Surveys and similar
8.  more and better tutorials.
9.  Success need more members
10. contests (best site, etc…)
11. IRC, not custom build
12. ticket system

2) Stefan Chiriac/Tamas Fenyszarusi – Ways to attract new clients. Marketing.

– 1.  demo account (video?)
– * 2.  alternative to invite system (ticket system?)
– * 3.  each staff gets to know new clients
– 4.  build connections
– * 5.  IsMyChat
6.  interactive video or normal video or intro video
7.  Advertising

3) Imtiaz Rahman – Improving speed on support issues. Possibly getting a ticket system.

1.  get a ticket system
a. must login
b. like, but not WHMCS
c. MAKE OWN
2.  upgrade using ticket system
3.  new ideas via ticket system
4.  tool for viewing space left on node
– 5.  fix public server status page

4) John Stimac/Nick Liu – Login session cookies.

“I guess there is nothing to talk about… Dev must to fix it”

5) Stefan Chiriac – “C***py” servers. How we can improve.

1.  clean up servers (monthly?)
2.  remove bad nodes.
3.  more abuse staff

6) Tamas Fenyszaruzi – Informing clients where to go for help.

1.  welcoming@ismywebsite.com
2.  information center
a. links to common areas
b. link to ticket system

7) Imtiaz Rahman/John Stimac/Austin Mushu – A better tool request process. Departments not having tools they require, or requesting too many tools too quickly.

1.  prioritize
2.  todo list

8) Tamas Fenyszaruzi – Process for changing key passwords.

1.  email everyone relavent
2.  department heads change pass

9) Imtiaz Rahman – All staff not having forum accounts.

1.  All staff should have a STAFF FORUM ACCCOUNT
2.  Gatz is writing an email to Matt.

10) John Stimac – IMW chat hackability.

1.  Bass is fixing

11) Stefan Chiriac – Ways to hire/fire [staff] members without asking an admin.

12) Nick Liu – What’s his role?

13) Stefan Chiriac/Imtiaz Rahman – Open forum registration. “I think either CobraStrike needs to release that thing that makes all IMW Clients a forum account automatically or he just needs to open forum registration”.

14) Stefan Chiriac/Imtiaz Rahman – Inactive clients. “You’re gonna send that e-mail I wanted or not?”

15) Matheus Pratta – Empty sites. There is tons of on IMW. Like, If you are not active for 1 year, your websites and account will be terminated?

16) Imtiaz Rahman – Sometimes we accept sites, their cpanel works but not the TEMP url. Other acceptance bugs.

17) Imtiaz Rahman – Server status page says MySQL is offline always. “Maybe there can be a work around for that?”

18) Nick Liu – “Abuse Tools Suite: Viewer Bob”

19) Imtiaz Rahman – The Advertising System needs to be “modified” to 1 view per per day per ip.

20) Imtiaz Rahman/Brice Corley – Maybe a Staff only TODO list.? So we can keep track of whats going on? Originally the /todo was supposed to be staff only.

21) Imtiaz Rahman/Brice Corley – Proposal to get rid of all inactive Advisors.

22) Brendan McNiff – Setting up logs of staff action.

23) Stefan Chiriac – So in the end Davesh is a support member or is fired?

24) Brayden Peacock – Some advertisement codes don’t work…

25) Austhin Mushu – More clarification on department duties.

26) Chiriac Stefan – Role as administrative assistant and finding a replacement Head of Welcoming.

27) Jay Chesla – Wants to revisit “switching from resellers to a colocated server”

28) A system for backups of the main site.

If you have any to add, then just let us know. The conference is November 13th, and starts 10:30 AM GMT – 6. If you want to attend, then add me (azoundria) to your Skype and I’ll make sure you get invited when it runs. If you don’t have a Skype client, you can download it from http://www.skype.com/. Skype is very awesome and free and cross-platform. (Works on Windows and Mac and Linux.)

If you do decide to come out which I hope you do, here’s the few rules we have (since this is a professional conference):

1) Represent yourself well. Proper spelling and grammar is a must.

2) Stay on topic. If you have to leave, do so quietly. If you arrive late or come back, also do so quietly. If you tell people you are leaving, because we’re a really nice crowd, we feel compelled to say goodbye, and this interrupts the proceedings. We have a lot of issues to get through.

3) Keep positive. If you disagree, state it in a question, not a statement. For example, instead of ‘your idea sucks’, ‘Can you think of any problems with your idea?’. And likewise if others disagree with your idea, do your best to listen to them and accept what they have to say. Should be common sense.

So that’s about it. Any questions – comment. See you all there.