Just returned home from Port Douglas, with wife and 2yo daughter. A magic, hassle free holiday with some of, if not the best sailing I've had in my two years windsurfing.
Flying Qantas really made It easy. I was able to use my, and, as 2 year olds require their own ticket, my daughter's checked baggage allowance for my gear. 1 bag with 2 race sails and 1 mast, and the other with my 122 board, boom, harness, extension, fins, and some fill e.g - clothes. My gear didn't make it on the plane on the way up, but was delivered to my door in Port Douglas the next day!...undamaged

This left myself and my daughter with 7kg of carry-on each, heaps, if you can wash whilst away.
Port Douglas was great for relaxing and eating and not bad for shopping either. Not a lot for the kids, but the Daintree was a great day trip, as was Kurranda and Mossman Gorge. We spent no time in Cairns but looks good also and only an hour down the road. Ports local sailors are very friendly, and are very helpful if you need help with local knowlege or gear, like spare lines, thanks Ed

.
Pretty much bump and jump at Port, but yes, GI is an easy day trip giving 5 hours on the island for $86 per adult. You can stay on the island also!
We will definitely return for more.
Good luck.