Having sat through quite a lot of end-user sessions, I would make three points:
1. users do often notice these differences subconsciously, even if they say they don’t. Say “which do you prefer” and they’ll tell you the one with the nicer shadow (for example) even if they can’t articulate why.
2. When a site is more aesthetically pleasing, users report greater satisfaction with the product and find it more usable. Of course, there’s diminishing returns here, but premium brands may get more returns on these subtle extras.
3. Developers tend to prefer sparse, lo-fi designs more than non-developers.