Researchers at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) have introduced groundbreaking technology that enables users to create realistic virtual avatars in just seconds using a standard webcam. This innovation promises to revolutionize virtual communication by allowing individuals to interact as if they were physically present, capturing facial expressions and movements in real-time.

During a recent demonstration at the university’s Data Observatory, Professor Hao Li, a Computer Vision expert and the Director of MBZUAI’s Metaverse Centre, along with PhD student Ariana Bermudez, showcased two newly developed technologies: Voodoo XP and XMem++. Voodoo XP facilitates real-time facial reenactment from a single image, a significant advancement compared to other systems that require multiple cameras and extensive training. Professor Li noted that the technology allows him to control a digital avatar without any special equipment, just using an ordinary camera.

Bermudez elaborated on the advantages of Voodoo XP, highlighting its simplicity. Unlike Meta’s Codec Avatar system, which requires a complex setup of 171 cameras, Voodoo XP can create an avatar in seconds with just one webcam. This system effectively captures subtle expressions and movements, such as blinking and smiling, making virtual interactions more lifelike. The development team for Voodoo XP includes several researchers, such as Phong Tran and Egor Zakharov, among others.

Additionally, Bermudez introduced XMem++, an advanced video object segmentation method that enhances memory efficiency and accuracy. This technology is designed for long video sequences and integrates refined memory management strategies along with lightweight attention mechanisms. Bermudez explained that XMem++ is particularly beneficial for visual effects artists, as it simplifies the tedious processes often required in video editing and augmented reality applications.

Since its launch in 2023, XMem++ has been adopted by the visual effects community, seamlessly integrated into the industry-standard software Nuke. Bermudez provided a demonstration on her cell phone, showcasing how users can easily manipulate video elements using this tool. The open-source nature of XMem++ allows artists to utilize its features for tracking and refining visual effects efficiently. The development of XMem++ also involved contributions from researchers such as Maksym Bekuzarov and Joon-Young Lee from Adobe.