I haven't really been scared, per se.
I never fail to laugh my ass off when I encounter the Raider that sounds like Moe from The Simpsons though!
Sometimes I feel a pang of guilt (or something like that) when I shoot a fleeing raider in the back. But I usually assuage myself with the knowledge that they would have pursued me and dealt the same punishment to me as well.
Also, if you have a chance, check out the McClellan Family Townhome in Georgetown. The first time I had the robot read the "children" the poem, I was very moved by the whole experience. I even Googled the poem and discovered the author Sara Teasdale had written some other very good poetry, if you're into that sort of thing.
The game has a ton of atmosphere and I'll go back to it and play it for a week or two before putting it up again for a bit. But I've never been scared by it.