Some more food for thought:
At 97kg I am able to surf a 84 liters board ( a 7'10" x 27.5" Imagine Impact I tried once in June), I could only stand up and take off in the following 10 seconds, otherwise I would fall as the needed concentration was too intense. And I was doing nothing special on the wave, as I was to mentally exhausted after managing the take off. And it was glassy.
Although I SUP often on a 105 liters "Tomo" shape (wide everywhere, 5th from left below), I got a traditional performance shape in 115 liters 7'4"x28" (4th from left below) was hell for me. So after 3 sessions with it, I was not really enjoying missing waves by falling on takeoff (although performance on hollow waves was stellar), and was feeling that I was not progressing enough in the result, so I surfed my other SUPs for 2 months, all 125 liters except for the Tomo. And I am sure the difficulty was more in the 28" width than the volume.
I was then totally surprised when re-trying the 115 liters to find it "easy"! the sessions (3 per week) on my other ~125 SUPs have made me progress enough to train me efficiently for the 115 liters! So going down progressively in size is VERY productive, by being confortable on a board size you actually train a lot for the smaller size... you are definitely not "wasting your time"! On the opposite, would I have had only the 115l in my quiver, I would have been less eager to get out with it in imperfect conditions, surfed less, and progress slower.
Quiver for reference: