Definitely feel way safer and warmer in my jet pilot vest. I surf deserted south coast beaches sometimes (see my pic) and it makes a difference.
I don't find it a problem at all getting under waves. It also reduces your drag time under water on bigger days. Helmet also helps specially when you get smacked in the head with a 9' lump of epoxy.