Calculating the Real-World Distance Between middleTip and wrist

Hi -

I apologize if this question has been answered in the past, I can't seem to find a clear answer. I'm wondering if there is a reliable way to leverage individual landmark points from VNDetectHumanHandPoseRequest to calculate a real-world distance. Like the wrist point to the tip of the middle finger and return a calculated result like 7.5" for example.

My assumption is that the same methods used with a manual hitTest to find the distance betwen two points (like in the official Measure app) could work here. However, with hitTest being deprecated, that leaves me a bit of a loss.

I'm happy to continue to dig through the documentation, but before I do, I was hoping someone could let me know if this is even possible or if we're still not quite there yet to be able to leverage the Vision points to calculate an accurate distance (on modern devices that support it)?

I appreciate any feedback or points in the right direction!

Up vote post of kane_c Down vote post of kane_c
570 views
Posted by
kane_c
Reply
Add a Comment

Replies

Vision points are normalized. Vision has no idea how far away your image subject is. You might be able to use information from the TrueDepth camera (if present, and in use) to derive subject distance, or you can infer this information if you know the size of the subject and the angle of the camera's viewport (it might be digitally zoomed and scaled). Bear in mind that the hand may not be parallel to the plane of the sensor - if you hold your hand at 45 degrees to the sensor it is going to look 29% shorter.

Posted by
ssmith_c
Up vote reply of ssmith_c Down vote reply of ssmith_c
Add a Comment

This is very helpful, ssmith_c! I think you've sent me down a solid path with the TrueDepth camera. I really appreciate it!

Posted by
kane_c
Up vote reply of kane_c Down vote reply of kane_c
Add a Comment