Wife has a 2014 D4 HSE and I used to have a 2011 TDV8 VSE, the biggest difference is the quality of the interior, RR is far superior and the ride comfort is also in a different league. The D4 is really practical and a great load hauler but I don't get the same feeling driving it as I did the RR. If you dont need the 7 seats I'd be inclined to go for the real thing, just get a good one with an extended warranty. Try and get a 61 rather than an 11 plate if you go for a 2011 as the very early ones (mine was a Jan build) had lots of electrical issues. 2021 P400 Silicon Silver AB
Gone 2011 Bournville TDV8 VSE
Gone 2008 Black Canadian HSE LUX

Just changed my 2013 Disco 4 HSE Luxury for a 2015 4.4 Autobiography. Lots of sensible comments already made... D4 is more practical with 7 seats, more boot space, more stowage space around the cabin, simpler materials mean less worried about muddy boots, and on the driving side it is hesitant when starting from a stop (you get used to this but not ideal).

I had no issue with the handling or acceleration of the D4, but the L405 4.4 is quicker and softer - today I thought I was floating over potholes and undulations in the road. As for the L405 I'm loving the pace and immediate response (no hesitation), the way the power builds if you're gentle with the right foot and the cabin is much quieter at any given speed. Smaller details like the digital dash in the L405 are nicely done and the panoramic roof is loved by all the family.

I cleaned the L405 today and it's noticeably more fiddly than the D4...

As you say try both and work out which best meets your needs