.

Build Your Own Bot Trap

Monday, November 8, 2010

Build Your Own Bot Trap

With Amazon now imposing usage restrictions on their API data feed, it becomes more important than ever to make sure you’re not wasting resources on unwanted visitors to your sites. By ‘unwanted’ of course, we are referring to search engine spiders who comb your site, indexing hundreds (if not thousands) of pages, and give you [...]

8 Comments Read full story »

Robots to the Rescue!

Friday, January 23, 2009

It seems lately that everyone is talking about duplicate content penalties, and how they only have adverse effects on your site, be it AOM or anything else. I’ve talked before about the DCP, and made some suggestions to help. Another thing that you should have in your arsenal is a robots.txt file, so we’ll discuss [...]

3 Comments Read full story »