Oh, I know all about it.
I don't have anything "heated", but I can layer as well.
The really bad thing is when I've worn just enough to keep me warm going to work and then it's gotten colder overnight.
When I get into more populated, high traffic areas, the ambient temperature is a little warmer. But there's a nice long section of nothing but open fields between where I work and home. That's always the coldest part.
Still, I've done ok most times I've ridden.
Lately it's either been threats of rain or a lot of wind... or just laziness