Here is a useful function to detect, in your controller, when a bot or web spider is clicking your links and pushing your buttons.Place this in helpers/my_helper.rb (use your own controller name obviously).

def MyHelper.is_bot?(request)
request.user_agent.downcase =~ /bot|slurp|spider|crawl/
end

Simple enough, but useful.

In my case, on a site i’ve been building, www.movie-cat.tv, the googlebot was coming through and marking all movies as “unavailable” through a simple crawl. If the bot is detected, the controller should not mark the movie unavailable, instead return before performing some action. This solved it.

GoogleBot, MSNBot, Yahoo Slurp, BaiduSpider. Modify the regex by adding any other bots you want to exclude, and separating them with a pipe “|”.