The price tag is that way because there are only 500 of them and we live in a free market economy (or close enough to it anyway). It's not because Lexus is being greedy. We are talking about a car that has only
1/20th the volume of the Gallardo, has far more bespoke components.
I would not say the engine is the only place the LFA shines...
The pic above shows the attention to detail and quality of materials. Just about everything silver in the cabin is magnesium or titanium. The ventilation housings are made of machined aluminum, whereas those in the Gallardo are plastic bits that look like they came from the supplier of the Mazda3. Even the fasteners aren't off-the-shelf.
The Lexus is heavy because it's been overbuilt (think multiple 24 hour enduros on the world's gnarliest circuit), has the counter gear set, rear-mounted radiators, structural solidity that makes it feel mid-engined, and has all the luxury amenities and warranty backing of any other Lexus.