It's not a simplification. It's information theory. It's not about one-to-one encodings (eg, bit 14 is whether you live in Arizona or not -- such a scheme would be ridiculous), but rather the amount of information that a discriminator reveals.
For instance, knowing that a person lives in the US reveals a little under 5 bits of information (there are a little over 2^28 people living in the US, according to Google). The entropy is just a measure of the amount of information that gives us in narrowing down a population; not an exact encoding.
For instance, knowing that a person lives in the US reveals a little under 5 bits of information (there are a little over 2^28 people living in the US, according to Google). The entropy is just a measure of the amount of information that gives us in narrowing down a population; not an exact encoding.
*Edit - s/science/theory/i