That's true for very large spaces/distances (except at night) where atmospheric scattering comes into play, but consider looking at a flat surface with stacks of objects on it. Lower elements are typically more occluded from lighting due to shadows from higher and neighbouring objects (ambient occlusion), with objects closer to you getting more illumination.

The brain is known to make use of such cues to gauge depth, and it's used in several optical illusions. (You can even make use of this principle with flash photography techniques to create approximate depth maps for surfaces. It's quite effective.)

I would argue that a model based on close distances of stacked items (e.g. stack bits of card or paper) is generally a better model than one based on large outdoor spaces, as it is a closer analogy in most situations. (Reading on-screen being reading off a relatively-close surface.)

But both can work. Look at screenshots of a game such as Spelunky: dark backgrounds, light foreground objects. The eye is drawn to the lighter surfaces and can ignore the darker background. But if the background was a distant landscape (e.g. Rastan -- don't know why that was my first thought... showing my age!), rather than a relatively close cave wall, your approach of desaturated lighter backgrounds might work just as well, as it would be more realistic. And at night, the opposite would be true.

Shadows are a very different thing though. The shadow of a playing card lying on top of another playing card does not make the entire lower card darker.

Try looking at the examples thinking they have depth, oriented so their depth is directly heading away from your eyes.

The one with dark on top looks like a tunnel to me; the one with light on top looks like items layered on each other.