Even with a full drysuit, a swim in cold water can make your hands painful within a few tens of seconds, and doing anything like gripping a paddle almost impossible, as I found the first time I cocked up in a snowmelt river - it was only a thirty second wade to the shore from when the river became shallow only a short way downstream of my swim, but I really struggled to manage any kit to the bank. Tight drysuit wrist seals don't help with this !

The best defence against cold water is a bombproof roll ! Equipped with that you can overdress for the air temperature and cool off by voluntary temporary rotary immersion when needed ;-) Survivial is a lot more likely in the case of involuntary less temporary immersion if you are dressed for the swim.