If this is your problem test out the routinely produced robots.txt file which will enable you to some diploma to preserve a whole lot of Bad Bots and...