Like most questions the answer is: it depends

I think that modern slalom boards are less technical than some may imagine. I was really surprised about four years ago when I was given a go on an RRD XFire V2 and went wow, that's not hard at all !
In flat(ish) water they will be roughly as easy as a freeride but a bit faster although getting into foot straps and initiating a gybe may be harder.
In rougher water, I think for intermediate sailors the speed difference will be less, and in some cases a freeride may be faster.
Slalom boards are also more physically demanding, especially the larger ones with 40+cm fins. And as Sean says, after a long session you're more knackered from the jarring and constant foot input required.
The solution is simple - get both