jump to navigation

Biggest Project Yet (For Domain Management) May 30, 2011

Posted by ismywebsite in general.

So it’s getting to be quite frustrating for domain owners (who bought domains through the system) to manage them. So much so that I’ve got a request lately to transfer a domain to another account. This defeats the entire feasibility of the domain system if they are somehow subpar domains, not able to be controlled, or their reliability depends on my limited time.

Thus, it’s really important to get a huge project finished, that would be used for a lot of systems. Essentially, it’s a proxy. It is able to crawl and explore the web in the same way my browser does. With a catch – it all has to be PHP-based. If I ask it to visit a site, it needs to keep track of it’s cookies that it’s picked up. If I ask it to click links, submit forms, etc… they need to be submitted such that it’s undetectable that this is a PHP client and not a standard browser.

My previous attempts have fallen short of working, due to my limited understanding of the whole cookie protocol, as well as very tight security on the websites I’m trying to access. If one thing is wrong, the whole request fails, making it very difficult to figure out what went wrong. Yet any browser I use has no trouble doing the exact same task (no problem at all).

The end goal is that you would be able to set up new nameservers for a domain that you own, and it would log into the applicable registrar (1and1, GoDaddy, WildWestDomains, MyDomain, etc…) and use their control panel to change the nameservers. The relevant output would then be returned. So effectively, an API could be built. In the future, this would expand. It would be possible to change the WHOIS details. It may even be possible to renew domains automatically. (Registration would still require approval to prevent system exploits.)

What else? This would allow for automatic daily calculation of AdSense earnings. AdSense does not have an API. Currently, I have to manually download a CSV, open it in a text editor, then copy and paste it into a tool I built to parse it. With this system, the tool could run each night (or a few times a day) and automatically grab that data. It also allows us to hook up with many other advertising networks, majorly expanding the selection. Finally, I was even considering Minecraft servers. However, all these do not have APIs and so they need a proxy / browser simulator to really be usable.

The main reason I’m posting this is to see if anyone knows of software that can do it already, or may know a developer who can build something like it. This is something I really would like to get going on.



1. Jack - May 30, 2011

Great! I bought habbosource.com, this is gonna help me a lot.

2. Deltaforce229 - May 30, 2011

I don’t know if this will help you but CloudFlare has an api and supports web hosts. CloudFlare is a DNS service that by default forwards all traffic through their proxy servers in order to filter out bad things like spammers, email harvesters, etc. while also speeding up page delivery and reducing load and bandwidth usage on web server. As I said previously they also have an API to control your account via any means you choose. I’m not sure however if the API supports changing, altering, or adding every feature/option they provide.

3. ismywebsite - May 30, 2011

I could build that myself. It would solve the nameservers problem, but none of the other problems. I really would like it so clients can set nameservers, registrant details, possibly even perform renewals, and of course all those other applications I mentioned.

4. Hostify Networks - May 31, 2011

I’m struggling to understand why you would want to write some elaborate custom browser/proxy thing that will never work right. (don’t you have better things to do?)

Most/all of those registrars have APIs readily available to their resellers that expose the functionality you seek. I’m guessing that you’re trying to do all of this from a basic account with no API access. Do yourself a favor and sign up for a reseller account with a decent registrar.

ismywebsite - May 31, 2011

Largely because so many services don’t and wont have an API at any point in the near future. I’m not one to keep watch and wait potentially many years to add a new feature. That to me would take far more time and energy over the long run compared to building a set of classes that simulate a browser. Not to mention that I think of new applications for this everyday.

In terms of your registrar option, here’s a breakdown of costs for the first option I looked at (OpenSRS, a division of Tucows):
– Set up fee. (Unknown as it’s unpublished until I enter credit card details.)
– $3.00/domain in pure profit to Tucows. (Possible discounts for volume.)

Other options I looked at have monthly costs associated as well. Also, in terms of pricing I cannot beat some of the other pricing I receive. As most of the domains ordered last only one year, they promotional first-year pricing means a big deal.

Plus, you have to consider that as a supporter of free market it’s my duty not to shop in one place all the time, to spread my business around so that we don’t end up with just a few major players or even a monopoly. This also prevents getting stuck dependent on any one company for everything, and allows a secondary benefit of researching and knowing the domain market when finding providers.

Finally, there’s the major but temporary hassle of moving all the domains onto a single provider. This may also have additional costs.

Well, all the system has to do is simulate the web requests. I already know how to fake a page request, and make it look like it’s from my same browser. I just have no idea how to grab the cookies, store them, send them with the next page request, or if there is anything else I would need to be aware of.

Hostify Networks - May 31, 2011

cURL handles cookies and should be able to do what you want. Still, it makes more sense to use something you know will work rather than rely on something that will break when the registrar changes the code of their pages on a whim.

Yes, it’s not free, but it’s totally worth it if you’re serious about offering this service. The costs can easily be offset by the paid service you’ve talked so much about (assuming you’re doing it right). You can also find domain reseller accounts for sale or even for free when combined with other services. I’m willing to bet that at least one of the resellers you are currently hosting with offers a free eNom account.

5. ismywebsite - May 31, 2011

Okay, setup fee for Enom with a package which still costs twice as much as 1and1 for the first year on each domain is $995! In the second year it pays off with a savings of 50 cents/yr.

The cheapest is $195 which still costs nearly three times as much per domain for the first year. In all subsequent years, it still costs ~20% more.

6. ismywebsite - May 31, 2011

Okay, it turns out that such a partial solution to a subset of the problems exists for only $65 to set up (which becomes the initial account balance) and no additional fees. Domains are as cheap as the Enom premium which is stupidly expensive ($1595).

7. Kristoffer - May 31, 2011

ResellersClub have an initial deposit that you can reuse, and the price/domain/year go down as you order more.

8. nqzdepofltr - June 7, 2011

Any news?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: