Well, winter came and went here and seemingly never showed up. Only a couple weeks was it actually very cold.
Leaving the key in the start position for a while before fully starting the car, I never had any issues, even on the cold days.
I suppose the fuel pump may be going bad after all, but...