IPN Students Develop 3D Facial Animation System

• Faced with the high cost of existing software, students from the Unidad Profesional Interdisciplinaria en Ingeniería y Tecnologías Avanzadas (UPIITA) developed a prototype capable of replicating facial movements and expressions

• The system can be applied to digital animation, video games, or adapted for use in vehicles to detect driver fatigue and help prevent accidents.

In response to the technological challenges of accurately animating facial expressions, students from the Instituto Politécnico Nacional (IPN) have developed a mechatronic system capable of capturing, analyzing, and reproducing facial gestures in a three-dimensional (3D) environment automatically.

The Polytechnic students noted that, despite advances in automating and refining facial movements in animation, significant limitations remain. In many cases, manual adjustments are still required for 3D models, and existing systems tend to be costly and time-consuming.

The project aligns with the vision of Claudia Sheinbaum Pardo, President of Mexico, and Mario Delgado Carrillo, Secretary of Public Education, who have emphasized science and technology as key pillars of educational development, with a focus on technological sovereignty.

Students Aleks Adrian Calderón Vázquez, Alejandro Campos Arroyo, and Daniel Cruz Ramírez, from UPIITA, adapted a depth-sensing camera mounted on a headband directed at the user’s face to capture facial expressions in three dimensions.

“These data are transferred to an Artificial Intelligence learning algorithm on a computer, where they are processed through a design application that enables the gestures to be mapped onto a facial model,” explained Aleks Calderón.

To ensure proper system performance, the team used open-source software to program the different modules, integrating the data captured by the camera into the 3D model. With guidance from professors Erick Huitrón Ramírez, David Abraham Morales Enríquez, and José Luis Cruz Mora, the students implemented a machine learning algorithm (neural network) capable of identifying specific facial regions, including the eyes, eyelids, eyebrows, and mouth.

“We trained the algorithm using our own facial data, but also relied on a large dataset from a Google computer vision library, which includes a wide range of facial features captured with depth-sensing cameras,” noted Alejandro Campos.

Using this data, the team generated a mesh that can be transferred onto a pre-designed 3D facial model, enabling it to replicate human facial movements and expressions in real time. The animation can also be stored for later use.

“We were able to leverage our knowledge and emerging technologies to carry out the entire rigging process, which involves configuring an internal skeletal structure that enables accurate facial expressions and precise technical adjustments,” explained Daniel Cruz.

The students, who developed the “System for Gesture Identification in Three-Dimensional Facial Models” at UPIITA’s Extended Reality Laboratory as part of their mechatronics engineering degree, highlighted that the project has applications in digital animation and video game development. It could also be adapted for use in vehicles to monitor driver behavior and issue alerts in cases of fatigue or drowsiness.

For more information, visit www.ipn.mx