Bucket I use a 26.5 X1 select in my 105 wave board and I am 105kgs. I also have an 95 litre OES twinny with 16.5 cm fins. Sounds a bit odd that you are loosing speed when turning downwind on a wave. You should be getting more powered

At your weight, board setup and sail size your fin should be fine. My biggest sail is a 5.8 and I go out in 15/16 knots no probs and wave sail (just).
Think Leech has the right idea. Let someone have a go on your kit and see what they reckon.
Me thinks it might be a technique thing that can be sorted by more practise. Get ya mates or partner to film ya as well and see what you are doing. It helps heaps.
Also crap onshore waves are challenging at the best of times