Probably because of the belief that the right hand is the hand of God, and the left hand (path) is supposed to be indicative of Satanism. That's why for many years Catholic schools tried to stop people from using the left hand when they were left handed. I believe it was said that Jesus served at the right hand of God, hence being a "right hand man". That's where I think it comes from.
Lefty's have rights to :)