Sorry if you have found our site inaccessible over the last 24 hours, we have had a lot of downtime ‘thanks’ to bots misbehaving on our site. Because of its nature, people find our site useful for scraping product data, and we don’t even mind that up to a point, and we have until now had fairly relaxed rules about it.
But we can’t have people trying to crawl our site multiple times a second, we don’t have the server capacity for that and although we do have some throttling in place already, super-heavy crawling has basically caused a DOS on a number of occasions, like yesterday.
So this is why we can’t have nice things, and why we’ll have to introduce a more restrictive policy in future. Users trying to crawl our site too frequently, certainly any more than once every couple of seconds – legitimate users just don’t need to click around that fast! – will initially be temporarily blocked and then if they continue the practice, permanently blacklisted.
To all the legitimate users of our site, thanks for using our service and respecting our bandwidth and server capacity.
If you believe you have been blacklisted in error or you want to grovelling apologise and promise you will never do it again (some form of peace offering may be required), contact us using the contact form.