Also the attack can be performed multiple times and if a person travels it could narrow down the possibilities quite a lot.

They had an example of the attack getting two locations back, Las Vegas and San Francisco.

So the target is somewhere in the many thousand square miles in the circle that encompasses almost half the US!

Let's say they travel between NY and LA, how many sources of data will you need to know who was in NY on a specific date and LA on a second date? Feels like only the government can reasonably locate that.

The government is a plausible adversary for Signal

FWIW if it's the government, wouldn't they be able to just get direct access to Cloudflare logs - in real-time even - and thus observe and track the specific incoming connection to fetch the cached image?