Recommended sail sizes are the range of sizes that the manufacturer has deduced work best with that size board. Other sizes may work OK, but will not be optimal. In fact, I find that each of my 4 slalom boards works best, (is optimised), with only one size sail for me. If I change sail, I almost always change to the matching board. (All cammed Race/Slalom sails) ie: For my 175cm and 73 KG its IS80/5.7m, IS87/6.3m, IS97/7.0m, IS117/7.7m. If I use the smaller sails on the larger boards it feels less balanced, but mainly, if I go to the smaller board with the smaller sail, I go faster and I am more balanced and comfortable. Using the larger sail on the smaller board does not get the best from the larger sail as the board does not have the volume and planing area to be as efficient and carry the larger fin nessasary to get the best from the larger sail.
But, if it was the only gear I had, and it was the only way I could go for a sail, the biggest and smallest mismatches in the above gear would be sailable, and I could probably enjoy a bit of fun.

When it comes to Freeride gear and Wave gear, the situation is similar, but the emphasis is more about having the volume and fin to carry a larger sail on a smaller board. Smaller sails can work better on larger boards, as areoagnr says, but then the boards can be too large for the amount of wind (which is why you want to use the smaller sail) and chop/bouncing and control (windage/lifting) become the issue. This can be less of an issue on flat water.
So there is Optimal, and most efficient. And there is just getting out and having some fun with what you have at hand.