It is essential for every cyclist to wear the proper cycling clothing correspondent to current weather, especially on cold rainy/windy conditions. Wearing insufficient or incorrect pieces of equipment can lead towards discomfort like freezing, humidity and will further result in injury or illness. Starting from top to bottom:
Image Source: Roadcyclinguk.com
If it is above freezing point wear a regular sport cap under the helmet. It will provide some insulation for your head and, on rainy conditions, will keep water from your eyes. On extremely cold dry conditions wear skullcap or ski mask. Cycling in such harsh environment will make your breathing difficult and try to inhale with nose and exhale trough mouth, so you can prevent cold air from reaching throat and lungs.
For sweat absorption and insulation put an under vest. If it is not to cold you can wear regular cycling jersey with long sleeves, but if it gets colder, it is important to have jacket over it for preventing cold air and wind to get under. For rainy days it is better to have jacket with hydrophobic layer, but if you do not have one you can bring rain cape. To be more noticeable in traffic make sure you have fluorescent vest or stripes on you.
It is enough to have regular cycling long pants, but if wind gets stronger it is a good idea to have bib-short under it to provide better insulation and sweat absorption for your crotch.
Image From: Roadcyclinguk.com
If your core is warm enough, you will not have problems with your hands even if you have thin gloves for warm conditions, but still it is a bit uncomfortable while descending downhill to feel chill hitting under sleeves and your fingers, so wear winter gloves and stuff your jacket sleeves under them to prevent cold air entering.
Toes are the most vulnerable body parts that can be exposed to cold and even with warm core it is still hard for body to generate enough energy to keep them warm. The efficient way to prevent this is putting two pairs of socks (tick over thin layer), putting overshoes or both. In very cold dry conditions, it is good idea to put two pairs of overshoes (thin layer over thick) and only one pair of socks. Keep on mind that it is better to layer the outside than inside, because putting too much socks can result in bad circulation and discomfort.