I voted for 'Depends on the car'.
I've not owned a car with an automatic gearbox that has been built more recently than 20 years ago, and even in that one, the experience of sporty driving wasn't really negatively affected - so with modern boxes I'm sure a great time can still be had, even if you're not swapping cogs manually.
I'd even say that it can add some character to the car and driving experience. My 635CSi was a 3 speed, non-switchable auto. Kick-down was engaged by a button behind the pedal. It was riotous fun stomping on the accelerator to get kick down on the exit of a bend, when the 220hp would overcome the traction offered by the terrible Michelin TRX's.... I've never spent so much time sideways as I did with that car. A 3 speed auto would not have been my first choice for that car, but nowadays it's one of the things I remember most fondly.
One of the other things I do like about spirited driving in automatics, is how easy it makes left foot braking. I'm no rally driver, and I can't heel'n'toe in a manual at all, but I always left-foot brake in auto's just out of habit, which I do find quite enjoyable.
I think it takes some skill to adapt to an auto box, and get the best out of it. You still have to think about the engine revs, and when the car is likely to shift, and how to optimise that. In my 328i Sport, which had an auto 'box, I'd get in to the habit of trail-braking slightly too lightly, followed by a slight stab of the brake (all LFB'ing of course), often synchronised with a flick of the 'Sport' switch next to the shifter. That slight drop in revs when in sport mode would always force the 'box to drop down a cog. It sounds like a faff, but after a while it becomes just as much second nature as using a clutch and a stick.
I also think the perception of manual shifting versus automatic varies depending on where you live. Here in the UK, pretty much everyone starts learning to drive in a manual, it is entirely the norm. In places like the US, it appears to have been the less common option, so I think manual shifting is seen as a bigger deal. I could be wrong on that.
The way I see it, 98% of my gear shifts are made out of necessity, not by choice, so it seems crazy to put up with all that, just for the sake of mechanical interaction with the car the other 2% of the time. Having said that, my last 2 cars have been manuals - so what do I know

Driving pleasure is more than just manual shifting.