Hypothermia isn't fun. I got a case of it when in an air-conditioned bathroom when I got COVID (the all fluids leave your body through both ends and fever sweats type of covid). So surprisingly easy to do. I've gotten mild frostbite on my ears just going for a 40-minute walk, so now have to keep my ears covered in even mild cold. Once you do it once, it gets easier to do it again.
In my cold weather safety training, I was told that exposure does help reduce the chances of getting frostbite... But once you get it, it greatly increases the chances of it happening again.
Like many things in the first responder world, I'm unsure if it is a tradition passed down or an actual medical phenomenon.
That said, when working out in the cold, I find just covering fingers, toes, and ears works pretty great. So my outfit is basically pants, t shirt, hat, gloves, and shoes when shoveling snow or doing outdoor swings. Be mindful when you are getting cold and don't be a tough guy.