The fact it sound like a bad idea probably means it is but what would happen if I put a larger fin(something like an RSX or small formula fin) in my 133 litre slalom board?
For context it's a 133 an hypersonic and I weight around 75kg.

we used to do this on slalom boards just prior to the wide craze changing boards shapes. all in the name of getting going in no wind. then fomula took off and the game changed.

in light winds, the worst case scenario is porpoising which is hard to control. end game is wide is good in light winds.

Depending on board width and fin type , I would think somewhere from 44-48 cm would be about right. A larger formula type fin would be a compromise solution.

I tried similar on my Powerbox modded Speed RB. 50cm JP Supersport fin= early planing. But what a mongrel to turn.

Years ago, I used to have the older Hypersonic 125, then changed it for the Hypersonic 105. I think yours is the newer version, slightly shorter length.

On both boards I used an old Select Super Fast 58cm with a 9m and 11m. But I only did this super light wind days. Worked fine in light winds, but as soon as the wind picked up, it was horrible.

Iv'e never known a board to work with fins ranging from 34cm to 58cm. The Hypersonic was brilliant back then. I did my first 35k run on the HS105, loved it.

Same as above, I tried a 60 cm rsx on RRD superlight (90 wide) and it was nice in very light winds, but as soon as you hit a gust it would try to lift the tail, was not accelerating and hard to control.

Here is my experience:

I used a 60cm RSX fin on a 145L Exocet Warp Slalom 80 (80cm wide) as I was getting spin out when using a 9.8m sail with 49cm Select Slalom fin.
Worked a treat.

Currently using a 50cm fin on a F2 XS Large (125L, 80cm wide) which works great with a 7.6m sail.
The fin was taken from a F2 Thommen Course Race Large (158L and only 68cm wide) and it worked well with this board too.

I would say give the larger fin try if you can get a cheap one. It helps with planing in marginal wind.

I think you'll find it unbalanced and hard to control. Generally bigger fin goes with wider board. There might be some conditions it works ok. Let us know how it goes

Agree width of board and sail size choice determine fin size range.
My personal preference is not to go to big sail and fin as sometimes the biggest sail needs a bigger fin. But when a smaller sail and fin can be pumped and more efficient to go faster than bigger sail weight and ability to control easily (and not fight it).
I like speed, manourability as preference for small gain in aspects if consistency of bigger and heavier sail size.
That said your 12knots may not have as much power/planning ability as someone "miles" apart.
Difference between Maui and Auckland as gusty breeze needs higher average velocity. And always easier in cross shore vs on shore
Looks onshore with swell this weekend

It's all about having enough torque from the fin to drive the power of the sail. You want to find the balance between low end drive and top end speed with maneuverability. I like to have some fin support when heading upwind or to plane early when it's light.

Then, the board itself (overall width + tail width) will help to leverage the added fin length. If the board is nice and wide it'll benefit from having a longer fin. What are your board's dimensions?

Examples: used a 60cm fin for a 9.0 on a Formula for freeriding, for my weight 75kg (at the time) the 70cm fin was way too big. Too big of a fin forces us to use crazy wind angles. Also used a 50cm for 7.0, both were perfect. On my 145L board, I had a 48cm or so, worked well for 7.5 and was fine for 9.0 too. Going bigger on the fin wouldn't have increased performance as the board was quite narrow for its volume.

The more hiked out we are, the more leverage we have against the fin. The larger the sail the more leverage we have at our disposition therefore fully utilize the benefits of a larger fin.

Another example. I have a 105L freeride board and even upping the fin length, 7.0 never worked well with it. The board itself just likes smaller sizes. So it'll depend on the shape as well.