Magic Leap Calligraphy Project

SUMMARY:

During the fall of 2018, I worked as an Interactive Media Researcher at NYU Shanghai.

My project for the semester was to create an application for the new Magic Leap mixed reality headset that used motion capture to teach Chinese calligraphy.

To accomplish this, we first used the university’s motion capture studio to record an expert calligrapher as she wrote several Chinese characters. We then extracted that motion capture data and displayed in 3D space via the Magic Leap. Finally, we implemented a simple painting UI that would allow users to draw characters using a virtual brush.

Ultimately, the project allowed students of calligraphy to trace the hand motions of an expert calligrapher as she drew several characters.

MY ROLES ON THIS PROJECT:

Programmer, Designer, and Motion Capture Tech Director.

Duties:

• Tech-directed several motion capture sessions (see videos and pictures above.)

• Designed the project’s UI.

• Programmed the project using the Magic Leap SDK.

Collaborators:

This project was a collaboration with Christian Grewell, Ariel Tao, Xincheng Huang, and several researchers from Tongji University, including Yinan Zhang.