Brrrrrrrrrrr tis a chilly subject !!!

I don't go out if it's too cold nowadays, but for many years, and not so long ago I had to be out like it or not, whatever the weather 7 days a week.
The most important aspect of keeping warm is keeping dry. Sweaty palms can at least be uncomfortable, at worse be dangerous. Wool is great for wicking away moisture from the skin, as are newer man made materials.  Any of the newer technical gloves intended for cold weather wear will both be waterproof and allow the skin to breathe. They are usually available in wide range of styles from outdoor clothing shops that specialise in mountain-wear etc.  Some are designed so you don't lose the dexterity of your fingers.  They do mittens that can be folded back to use your fingers.

Cold will also effect other aspects of your photography too.
Batteries will drain much quicker, and condensation can be a real problem. Cold cameras and equipment brought indoors or into a warm car will get condensation inside and out - don't turn it on !  Lots of silica gel in your bag will absorb some moisture and wait until you're sure temperatures have evened throughout your gear and no moisture is present before you power it up.   Filters will collect moisture too, so worth checking your filter has not just frozen over while you were setting up the perfect landscape shot ;)  Pipe lagging around tripod legs will make them much more comfortable to hold, save a bit to go on top of the car window for when you can't be bothered to get out the car and rest the camera lens on top of the open window. 

Wear loads of layers of technical clothing and don't wear cotton you and it will freeze !!! 

And every one said, 'If we only live,
We too will go to sea in a Sieve,---
To the hills of the Chankly Bore!'
Far and few, far and few, Are the lands where the Jumblies live;
Their heads are green, and their hands are blue, And they went to sea in a Sieve.

Edward Lear
http://www.nonsenselit.org/Lear/ns/jumblies.html