Apart from making love or kissing, holding hands is the most intimate thing you can do with a partner -and it's publicly acceptable, too!
Because of the way we evolved to walk on two legs, our hands developed as super-sensitive tools for exploring and manipulating our environment. The hands and fingers contain a huge concentration of nerve endings, each with its own special direct connection to our brains.
Holding someone's hand is a sign of trust and confidence, and a recognition of another person's presence as a desirable thing.
I like the interlaced fingers type of hand-holding, which seems to maximise those feelings of security and allows the clear communication of feelings through little squeezes.