Proxy Builder Blog

 I stumbled across a post on my favourite proxy forum today asking how do filtering companies know which sites to block? As a proxy webmaster it is greatly beneficial to us in traffic and revenue to keep our web proxies unblocked for as long as possible.

When I think about it there are so many different angles that filtering companies can use in order to determine whether your website is a proxy. There are several large companies that offer huge scale filtering and monitoring services like Cyber Patrol, Websense and Bluecoat. A large part of their business model will be devoted to seeking out and blacklisting sites that are will be banned to certain viewers.

I am pretty sure they will be analysing some of the  features below to automatically to work out if it is a proxy. If you want to stay unblocked then remember these things will be checked when the filtering companies’ web spiders come crawling through your site.

Domain Name

As you would expect this will be one of the first things that filtering companies and their categorizing systems review. You can expect that these companies will be pretty advanced in the way they analyse the words in the domain. Any words like proxy, bypass, unblock and anonymous will raise a red flag and call them to do a manual human check. The easiest way to avoid this is to use a domain with no relevance to proxies or other potentially banned words.

Filenames and Images

Many of the companies that provide software for blocking the internet rely on a vast series of spiders and automated bots that crawl the web looking for bad sites. Your site will probably have to under go a human visual verification check  if the files in your template are named like proxy-head1.jpg or proxy.css.

Content and Header information

If you write information about proxies and how they work your site is likely to get you blocked for good. Every bit of the HTML in your pages will be searched by bots in order to categorize and define your website. It is best to avoid all mention of proxy related words from your content if you want to avoid the block.

Links & Affiliates

In the process of sifting through your proxy website HTML, the software will be analysing which sites you are linking to. If a large amount of the outbound links are already blacklisted or the domain’s contain certain words then your site may also get blocked.

IP Address

The I.P Address your proxy website is hosted on can be very important, for example if your site is on a server dedicated to proxy websites, in the long-term your IP address will be banned. This obviously isnt an ideal measure for filtering algorithms to decide on because many servers host multiple sites on the same IP. It will however be analyzed along with the above factors.

To summarize, I run a very successful network of proxy related websites and I expect that most are blocked by filtering software now. I never use these methods so I can’t offer a statistical comparison - this is purely common sense thinking. My success was based on targeting high traffic keywords in the search engines. There is certainly a large audience out there looking for reliable web proxies that may last a week or two longer . The best way to do that is to stay out of filtering software databases.