Rereading the post in question, I'm not so sure that I'd be afraid of "theft of my personal data" so much as "a plan for theft of my personal possessions." A map of my house with an inventory of my possessions would mean a lot to thieves who wanted to hit targets quickly and optimize their spree for a value-to-weight ratio. That kind of data could make organized burglary very profitable.

The overlap of "the entities that carry out data breaches on digital entities" and "the people who break into your house to steal stuff" is not really well correlated. This is for a variety of reasons, but the most boring is that hacking websites is way smoother when you're a faceless entity far from any kind of jurisdiction, and breaking into somebody's house is something you need boots-on-ground for.

Scammers and thieves are using data brokers more and more all the time. They already buy up lists of rich elderly people, lists of people with dementia, and lists of people with low IQs or poor educations who are often easier to trick out of their cash.

Most criminals breaking into houses aren't buying up targeted lists of likely victims from data brokers yet, but it's effective so you should expect that the number of criminals turning to those resources will only increase.