If the people paying the tolls work everyday in New York, then I think it would be reasonable to consider them to effectively be New Yorkers as much as they are New Jersiers (New Jersites?).

Like you say though, it's complicated. Especially when you have New York and Jersey City that in a lot of ways are almost one city that happens to be in two states.