Riding in cold weather

First, when the cold is extreme and your ride will be a long one, it might be safer to use some other kind of transportation. Otherwise...

Bring a cell phone if there will be service coverage along the route. If your bike breaks down, stopping to repair it or walking may expose you to too much cold. It might be better to get a ride.

You could carry two insulated water bottles:

Clothing

You can still get too much sun exposure when it is cold and overcast, because harmful ultraviolet rays can penetrate cloud cover. For suggestions about dealing with that, see the section of this site concerning sun protection.

I live in a mild climate. Temperatures at worst are around the freezing point. For that, I usually just wear:

Wearing several thin layers on each body part should give excellent protection. After you have been biking for several minutes, you might notice that you are warming enough that you are on the verge of sweating. In that case, removing one or more clothing layers will help.

Torso

Here is an example for the torso:

Hands

For the hands:

Or just wear snow gloves

Feet

For the feet:

Head

To keep your head a little warmer, you could use a helmet rain cover.

I usually prefer to use the hood of my windbreaker, with my helmet over it. The helmet straps keep the hood close to my head, so my vision isn't obscured when I turn my head. My face isn't covered, but it's good enough.

When it's really cold, you can cover most of the exposed skin from the neck up by wearing a balaclava (like a ski mask). Or, instead you can get more ventilation and less coverage by wearing a bandana combined with a legionnaire hat. Either of these two options will help deal with the wind chill created by your forward motion.

Covering your nose and mouth traps your moist exhaled air, which escapes upward. The air then goes behind any glasses you are wearing, causing them to fog up. My experience has been that even an anti-fogging agent (wiped onto the inside of your lenses) cannot always prevent this. In this case, you have to either remove the glasses or move your covering's fabric so it leaves the nostrils and mouth exposed.

Arms and legs

For the legs, you could wear leg warmers. Over that, put on pants (or shorts if you think you'll get really warm later).

Similarly, for the arms, you could wear arm warmers. Over that, put on a long-sleeve shirt (or t-shirt if you think you'll get really warm later).

If you become too warm, you could simply slide the ends of each warmer toward each other, as shown below.


Cold weather arm warmer Cold weather leg warmer

If that's not enough, you can slip them off entirely. However, sliding leg warmers over shoes could be a hassle (and messy, if the shoes are dirty or wet). I imagine that a tailor could somehow modify a pair of warmers to make them tear off, by sewing on hook-and-loop (a.k.a Velcro).