One even tried to BS our mgr and the discussion went on something like this..
dude: "We're just like google, just indexing..."
mgr: "No, you are not."
dude: "Yes, we are... Can't you let our spiders in?"
mgr: "No, you are indexing and gathering information from our site and selling it for profit... the material is copyrighted and we don't have a deal with you."
mgr: "We will not lift any blocks until we have a deal and we get percentage of your profit."
dude: "Ehm, ok. We will come back with an offer..."
... and then weeks pass, often without any callbacks.
I routinely check the server logs and block spiders that look commercial... google and others are ok but msn is blocked.