AI method could create more lifelike hair for video games and animated movies

Hair is one of the most challenging aspects of computer graphics, especially for animated movies and video games. Hair consists of thousands of strands, each with its shape, color, texture, and movement. Simulating realistic hair requires a lot of computational power, memory, and sophisticated algorithms and models.

However, recent advances in artificial intelligence (AI) could make hair simulation easier and more realistic. AI is a branch of computer science that aims to create machines and systems that can perform tasks that normally require human intelligence, such as learning, reasoning, and creativity. AI can be applied to various domains and applications, such as image processing, natural language processing, speech recognition, computer vision, robotics, etc.

One of the applications of AI in computer graphics is hair simulation. Researchers from the University of Southern California, Pinscreen, and Microsoft have developed a deep learning-based method to generate full 3D hair geometry from single-view images in real time. Deep learning is a subset of machine learning that uses neural networks to learn from large amounts of data. Neural networks comprise layers of artificial neurons that can process and transmit information.

The researchers used a generative adversarial network (GAN) to create realistic hair models from input images. A GAN consists of two neural networks: a generator producing realistic outputs and a discriminator distinguishing between real and fake outputs. The generator and the discriminator compete, improving their performance over time.

The researchers trained their GAN on a large dataset of 3D hair models and then used it to generate hair geometry from 2D images. They also used a neural rendering technique to render the hair with realistic lighting and shading effects.

Their system takes smartphone photos as input and produces 3D hair models as output. The process is then divided into two stages: first, the system estimates the 2D orientation of each hair strand in the image; second, it reconstructs the 3D shape of each strand using a geometric model.

The system can handle various hairstyles, colors, lengths, and densities. It can also deal with occlusions, such as when the face or clothing partially hides hair. The system can generate 3D hair models on a standard GPU in less than a second.

The researchers claim their method is the first to produce realistic 3D hair geometry from single-view images in real-time. They also say their method outperforms previous methods in accuracy, speed, and visual quality.

The researchers hope their method can be used for various applications, such as virtual try-on, face swapping, avatar creation, and animation. They also plan to improve their method by incorporating more data sources, such as videos and depth maps.

The researchers presented their work at the ACM SIGGRAPH conference in August 2023.