One of my favorite Lovecraft stories is Herbert West: Reanimator. Rats in the Walls is a good one, too.
I like his writing because of the way his stories are set up. I appreciate his approach to horror. What makes his stories scary isn't what is described, but what is not described. What is left for the reader to infer is made more frightening by virtue of being personal. The reader's imagination fills in the blanks, drawing off the his/her own worst fears, and weaving them into the story.
I try to embrace that technique, if not the exact writing style. At least that's my goal -- that is what would make me a better writer.
I have a story here. I'm not going to shamelessly self-promote, but if you look under my name in the members section (or under Gay Male), you'll find it.
Admittedly, I like horror. I've been asked why dozens of times, and I really don't know the answer to that. I just do. I love the thrill of suspense, and the cold chill of terror, and that gut-wrenching epiphany when I realize that something in the story has gone horribly wrong. Then I'm hooked. I want to know what happens next.
As a writer, I'm cruel to my characters. I swear, if they were real people, they'd hunt me down and kick my ass. In fiction, I like to read about impossible things. I like to see characters pushed to their absolute limits, then nudged just a bit further. Maybe I'm a sadist. I don't know. Horror fascinates me. Yet I'm not all that enthralled by graphic violence. It's what isn't seen that gets my mind racing, and for this reason, watching a hack n'slash film does nothing for me. Oh, another fake blood spatter. *yawn*
My favourite H.P. Lovecraft story would have to be Pickmans Model and it's for the exactly the same reasons as naudiz describes. I love his writing, even though it's horror, it has an elegant style you don't really see with modern authors.