I’m the same. Listening to weak-signal CW on the (poorly-located) KX2 loudspeaker in the open air can be difficult at the best of times. But it’s impossible when sheltering from the wind and/or rain under a tarp (or in a tent). The noise of the flapping tarp/tent inches away from my head forces me to use either over-the-ears Bose headphones or clip-over-ears earphones with a close-fitting woolly hat.
That lesson I learnt from failing to work Transatlantic S2S stations one very stormy day in a tent …
One saving grace is the joyously low background noise on the HF bands away from civilization. [At the risk of boring you again with this old anecdote] the first time I used my KX2 on summit, the S0 noise on 30m on switch on convinced me something was broken (e.g. antenna), Then I tuned around and a strong CW signal nearly blew the headphones off my ears.