If you can swim you can swim, the question is if you may swim.
This is a matter for your local council as per local government act
633 Bathing (including nude bathing) and other water-based recreational activities
(1) A person who, in a place being-
(a) a public bathing place under the control of a council, or
(b) a river, watercourse or tidal or non-tidal water, or
(c) the sea adjacent to (although outside) an area, or
(d) a public place adjacent to any of those places,
fails to comply with the terms of a notice erected by the council is guilty of an offence.
So so if the council says you may not use the beach then you may not swim, whether you actually can or otherwise.
Of course, if you don't tell anyone you can do whatever you want, but it will be an offence, nude or otherwise.

Edit: you could take a boat far enough off shore to not be considered adjacent to an area or leave territorial waters altogether, in the latter case you would then be subject to quarantined restrictions on return.