VR Drawing & Gesture recognition
Unity VR demo,
$P Point-Cloud Recognizer.
drawing in 3D space with recognition of preset patterns.
VR Drawing & Gesture recognition
tools & resources used
Hand Tracking and Gesture Recognition
Integrating Meta's XR All-in-One SDK and PDollar Point-Cloud Gesture Recognizer Asset, the user's drawing is compared with stored gestures. The gesture it most closely resembles must pass a certain threshold of accuracy to dispatch an event
Drawing in VR
For quick prototying, pinching index and thumb together starts drawing, and releasing stopping the draw. Pinching middle and thumb together erases drawing
Code snippit of draw data being updated and stored as 3D Vectors, along with updating the Line Renderer as the visual
WorldSpace to Screenspace
To compare the user's drawing to existing gestures, upon releasing the index and thumb, the drawing is projected from world space to screen space. It then compares with existing drawing gestures and returns the most compatible
Code snippit of projecting drawings from world to screen space, and returning most compatible gesture and its numeric value