Detect tap on a mesh

I have been trying to detect a tap on a Mesh and failing. Was anyone able to get something like GitHub - metal-by-example/metal-picking: A demonstration of picking (hit-testing) in a 3D scene rendered with Metal work with the book’s code.

@caroline Can you please help with this when you get a chance? Thank you - much appreciated! :]

To pick or “click” on a geometry items, you need to cast a ray (line-segment) out from the camera and then test against every triangle in the scene.

This can be very expensive depending on the amount of geo / tries in a scene. To get around this there is a tree structure called an OctTree which is used to query regions in space.

I would suggest looking into projection from screen space into world space, this will help with the creation of a line segment from camera into the world. Then loop over every bounding box if you have a bounding box for your geo. Check if line segment intersect AABB. If it does intersect then loop over every triangle in the bounding box and use the line intersect with triangle formula, and see if any triangle returns true. As soon as you have a true break out of the loop and SUCCESS you have hit the geo.

Math centric write up : geometry - Determine if a line segment passes "through" a triangle ... - Mathematics Stack Exchange

Palatable answer: c++ - How to know IF a line segment intersects a triangle in 3d space? - Stack Overflow

I hope that helps

1 Like

Thank you for your reply, @rumor

One way I did it was by a render pass to a half(quarter?)-sized texture that rendered touchable objects in a particular color. Getting the screen coordinates, I could access the color in the texture and work out which object was hit.

I haven’t checked performance - it worked in my simple case - but do you think this would be a viable alternative?

2 Likes

Its a pleasure. I haven’t tried the colourMap way as I had read up that it can return false positives when selection occurs close to the edge of two objects overlapping, as there can be artifacting due to antialiasing/blending of the pixel colour. But the method you mention is much faster out of the gate, as its a color pick and you dont have to have all the OctTree overhead for a scene, and an OctTree for the objects Tri’s.

Once the scene you are working with gets populated with lots of objects, I would suggest the OctTree raycast method.

2 Likes