As anyone who has shopped around for web hosting will know, almost every site will offer ‘unlimited’ features. You can get ‘unlimited’ space; ‘unlimited’ subdomains; ‘unlimited’ email accounts, and so on. In truth these are not actually ‘unlimited’, but they will usually offer you more than you could reasonably use. The dirty little secret they don’t tell you about is that in one important aspect, you are very limited.
This is in how much of the server’s CPU you are allowed to use. Unless you pay for a private server, you will have to share resources with other people who also have accounts on the same server as you. The more of you there are sharing the server, the smaller each individual ‘slice’ will be. Drive space can be adequate for everybody, and it’s easy to offer an excess of intangibles like subdomains and email accounts. But a CPU is limited by how many operations it can carry out per second (known as a CPU cycle). If you use more than your allotted amount, it can be trouble. In excessive cases, eating up too many cycles can get your account suspended or shut down by the host. Every application you can put on a web server will consume CPU cycle time. And AOM is no exception. As hosts become ever more stringy about this issue, let’s look at some ways that you can reduce the CPU load generated by your stores.
1. Bots. It should come as no surprise that the busier a site is, the more CPU time it will use. Many people with AOM sites who are warned by their host may be surprised, especially if they do not see corresponding traffic. How is their store using resources if they’re not seeing the hits?
The answer of course, is bot traffic. Search engine spiders that slam a site, indexing hundreds (or thousands) of pages endlessly, wasting bandwidth, Amazon resources and CPU time. In return for which, you receive absolutely nothing. Far and away this can be the single biggest drain on a site. Your website control panel (such as cPanel, the most popular) should allow you to check your bandwidth. Many will have a stats program (like AW Stats) that can break down bot traffic. And some hosts will let you access your error logs, showing requests for non-existent pages. All of these should provide clues that your quiet little site is actually a bubbling hotbed of activity. None of it results in sales, and can actually harm your chance of selling if the site is down for overusing resources.
Installing a bot trap can help cut down on the waste, freeing up your site for humans to shop at. It’s not a magic bullet, but more and more it becomes a vital tool to regain control of your CPU usage. Information on how to set up a bot trap can be found here.
2. Slow The Crawl Rate. Often once you begin to investigate the reasons your CPU cycles are draining away, you’ll discover that Google is a major guzzler. Of course, you want Google to visit your store and enjoy the experience, raising your profile in search results. But often it’s like having a rich uncle come to visit and having him eat you out of house and home. Eventually you have to establish some limits.
The solution is to sign up for Google’s Webmaster Tools feature, which is free. You can set a crawl delay so that the googlebot does not burn through your site, indexing like crazy. A slower crawl will help ease the burden on the server, and might keep the CPU meter out of the red. The crawl delay is good for 90 days, so it might be something you would need to stay on top of. But it’s better than having the site go down. Bing and Yahoo have similar control settings, but unless your stats indicate that they’re crawling your site excessively, it’s best not to change from the default settings. Most of the time, it’s ‘G’.
Note that there is a ‘crawl-delay’ directive you can implement in your robots.txt file, but it’s not certain that Google obeys this. The Webmaster Tools option is preferred, not least of which because it’s a certain way to let Google know about your site, and you can use it to see how they view your site. If you’re struggling with indexing, it can only benefit you to use their own tools to try and improve things.
3. Cache Settings. AOM will cache pages fetched from Amazon so that the page can be retrieved from the server for a period of up to twenty-four hours. This takes up more space, and will always mean a balance between the two (speed vs. space). However, a cached page does not need to have the data reparsed by PHP like a fresh page direct from Amazon. So all things considered, caching does save CPU time, even if only by a small amount.
When a page is more than 24 hours old, it expires and is no longer used (a fresh page is pulled from Amazon instead). Cached pages are eventually removed, and you can set the odds of a particular page being cleared off the server. The default setting is 20, which means a 1 in 20 chance the page will be deleted the next time the cache cleaner operates. You may think that increasing the odds (such as to 5, or a 1 in 5 chance) will be more efficient. But like a slower crawl rate, a slower cache removal can help balance out a server’s load. If you happen to visit a site that’s undergoing a serious cache cleaning, you might find that the store pages load somewhat slower than normal. Bumping down the cache cleanup may help lighten the load even more, but at the cost of having a larger cache build up over time. In terms of CPU load, the best option is certainly to have the cache turned on, and probably leave the settings in their default positions.
4. Image Cloak. The Image Cloak feature located under the SEO tab of the AOM control panel was designed to hide references to Amazon in the product images pulled from their server. Amazon will not allow the images to be cached (unlike the rest of the page), so cloaking the image URLs should help remove a potential source of duplicate content from the watchful eyes of the search engines. However, the extra processing required not only slows down the image display, it can also create a significant CPU load. You can check this by looking through your stats to see how many times the img.php file is accessed. This is the file that does the actual conversion; cloaked URLs will always reference ‘img.php’ instead of Amazon. Remember it has to load about ten times for every category page, plus whatever is required for images on the item pages. It might mean having to weigh up the pros and cons of the SEO benefit from using cloaked image URLs versus the CPU drain.
Note that the URL Cloak does not cause a similar load on the CPU, since it doesn’t require processing of any images from Amazon to hide their source. URL cloaking does not contribute to any meaningful server drain. But you may want to consider turning off the Image Cloak if you’re concerned about your CPU cycle usage.
5. Culling The Herd. If you’re running a large number of AOM stores on a single web account, you might want to consider your strategy. Take a hard look at your stores in terms of profit and loss –
- Profit – How much does each store make on a yearly basis?
- Loss – How much does it cost you to operate each store? Domain name renewal, server cost, etc?
Amazon makes it easy to examine your income for more than a year (4 quarters) in the past. Do you have stores that are losing money; i.e. cost more to renew than they make back over a year? If so, why do you keep them? Not only do you have stores that are failing to generate income (which is why you created them in the first place), but underneath, they may be wasting CPU cycles, being slammed by bots and sinking your bandwidth as well as your Amazon usage restrictions. And that could be a threat to your successful stores. If your account is suspended, or dropped by your webhost, your productive stores will suffer.
Note that we’re not talking about a single site, where server resources are not the issue. Find out how you can make that single site better. We’re talking more about someone who might have 25-30 stores or more, all running on a single account. With a year’s history, you should be able to tell which sites are making money, and which are suffering. Don’t be afraid to cut your losses and drop the poor performers. You’re not going to miss a store that made all of two dollars last year.
As mentioned at the top of this article, CPU usage is not something confined to AOM alone. Your stores may share space on your account with blogs or portals or photo galleries, and they all consume their share of your allotted CPU time. But if you (or your host) are concerned about staying within your limits, the steps outlined in this article should be the first things to try. If the concerns persist, you may want to consider other options, such as cutting back on the other sites. Or getting another hosting account to share your sites across. Even a dedicated server. Hopefully you can eventually put the brake on the CPU cycle.