Look up "pose estimation" as a research topic

https://arxiv.org/search/?query=%22pose+estimation%22&search...

See also

https://github.com/pliablepixels/zmeventnotification

for a rather mature system that adds person and object detection to a security camera system.

Okay I went and checked out pose estimation, which delivers a superior result (additionally I might be able to extract which way people are facing which would be amazing for art purposes).

Thanks also for the library you posted, I think internally it also uses YOLOv8.

Now the only remaining questions are:

1. How to do the math to translate the perspective camera image into positions on a floor plan, but I already got an idea where to look as this is known math.

2. How to stabilize results that pop in and out of existence — I guess I will just write my own overlay that has lingering states, at least for entities at the center of the image.

3. How to get all of that running with sufficient performance on my old Laptop with it's RTX2080m, because I won't put my tower into an exhibition space for a month. Either way it doesn't need to be fluid 60fps, maybe even 2 fps would be okay.

4. Into which OSC messages to translate the results, as I am collaborating with a VJ.

All of those questions are sort of managable I think.

Good tip, I read over this topic not thinking it would be of any help for my problem, but now that you mention it... maybe pose estimation works better than tracking. I havw to test this.

Can you assume that every person is carrying a cellphone, or other similar device? You could use wifi or Bluetooth somehow to track those devices...

Alternatively, could you place sensors on the floor under carpets to measure the weight of human bodies?

Or use opencv or something like it, but use infrared cameras from above. I'm not sure how hats would affect that...

I'd like to avoid depending on the devices on people's bodies for philosophical reasons, also testing it could be even harder.

But I read some researchers had success to do pose estimation from a wifi router, I have just no idea how to replicate that myself.

A sensing floor could work for this, but given the scale of the area this could be too expensive.

I actually wanted to use cameras from above, infrared could be a good idea, I will check into it — thanks for your suggestions!

I just remembered I still have velostat lying around, so maybe I try to make a scale model of a sensing floor first to get a feel for how feasible that would be.

A team I worked work along side seemed to have very good luck tracking people and even gestures using several kinects. With luck you might be able to do the same

I will check kinects, on a first glance both size and positioning might be an issue.

Ok checked more, the space is huge, so unless we get upwards of 10 of those this is probably not an option, but I will keep it in mind for future projects.