I am thinking what you are experiencing is a board way to big for you so it is 'corky' and wobbles easily. Especially if you have a gonad man stance with feet out on the rails.
Try as many smaller boards as you can before you downsize. There are heaps out there but width comes at the cost of surfability even though many will deny it.
I would love to have seen the 11 6 in 8 foot. In 2007 I rode an 11 6 Naish in big waves and nearly died I got dragged underwater so long after a wipeout.