RoomPlan Interactions

Question

l4nos OP

Created 2w

Replies 1

Boosts 0

Participants 1

HI there,

I would like for the user to be able to tap on a wall that has been highlighted as scanned (the white outline) and see basic information about the wall (in a pop up view modal) without being taken out of the scan session.

As a first step though i'd simply like to be able to tap on the scanned wall whilst still in the session and see in the NSLog, the data about that CapturedRoom.Surface.

I'm storing the CapturedRoom on update of the sesssion using the RoomCaptureSessionDelegate and I have added a UITapGestureRecognizer to the room capture view.

However i've tried a number of ways (hit testing, raycasting) and i'm unable to target the wall behind the users tap gesture.

If anyone can give any advice even if just the principal of how to achieve this.

Boost

Answer 1

l4nos OP

1w

So i've managed to solve this. And i notice not many people reply once they have found a solution on here, which is a shame as it would help others facing similar issues. So for that reason i've decided to post my journey of discovery below.

The ARSession only contains detected planes, not the actual walls, which appear to be inferred from a combination of vision, LiDAR and AI magic.
The actual walls themselves are stored in 2 places.
- 1. The CapturedRoom -> Surfaces collection
- 1. The ARView responsible for rendering the scene (Inside RoomCaptureView.subviews)
The ARView is controlled by Roomplan directly and ray casting to this is near impossible. Also entity by point isn't possible either due to absence of collision components.
When you try to add collission components to the WallEntity's inside the ARView -> FirstAnchor-> ScanContainer, they almost immediately get removed, as the ARView is being drawn or updated, it seems to completely replace the entities as opposed to updating them, thus removing any collission components at play.
Due to the above, direct tap ability against the walls being drawn by RoomPlan isn't possible. Instead you need to use that data to build your own Model entities that sits above (on a separate anchor), ensuring the Model mesh, dimensions, position, orientation and collision components all remain in sync with the drawn WallEntity.
Once you have done this, you can target those newly drawn ModelEntities using arView.entity(at: YourPoint). Once you identify the tapped model entity, you can reverse associate it with the corresponding CapturedRoom -> Wall.
This in theory should work with any surface or object detected by Roomplan.
This method also allows you to show / overlay visual feedback to the user so they can see which walls have been tapped on. Handy for user experiences such as room surveys.

Hope this helps, if anyone needs further assistance, reach out.

1