I'd like to know of any countries where a foreigner can be there without a valid visa/authorization and not be summarily deported if they are discovered.
In Germany we have a thing called tolleration [0]. It actually covers quite a few people, because quite a few people seek asylum because legal immigrantion is often dysfunctional. The difference maybe is that it is difficult in Germany to be employed or take part in life without any formal registration. Deportation is often difficult because it needs to be safe and the country of origin has to cooperate. Because Germany only has EU borders, pushing people beyond its border is pretty pointless once they have settled (they typically return within a few days). But we also have over night deportations of families (children having grown up in Germany, people having jobs) with police raids in some cases, that leave people traumatized.