Guy Gafni

Video Diffusion Neural Rendering Human Avatars Pipio AI

I'm Co-founder and CTO at Pipio AI, where we're building a foundation video model for talking humans. A video diffusion model with perfect lip sync, designed for video editing workflows.

Before that, I was a PhD candidate at the Technical University of Munich, Visual Computing and AI Lab, where I was advised by Prof. Matthias Nießner (w/ Prof. Justus Thies) working on neural rendering, NeRFs, reconstruction and re-rendering of faces.

📍 Munich, Germany

Research

I'm interested in 3D Computer Vision and generative AI, specifically in video generation and editing of humans. I prefer generating and looking at pixels over improving a metric, and appreciate the use of 3D priors. Lately I've been heavily working on video diffusion models.

In 2023 I did a research internship at Meta Reality Labs in Zurich, working on 4D multiview reconstruction of humans. My last couple of projects have been around audio-driven facial re-enactment, hyper-realistic human avatars, dubbing and lip-sync. In the past, with CNNs and GANs, now heavily focused on DiTs and flow-matching.

EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers NEW
John Flynn, Wolfgang Paier, Dimitar Dinev, Sam Nhut Nguyen, Hayk Poghosyan, Manuel Toribio, Sandipan Banerjee, Guy Gafni
ArXiv 2026
project page · arXiv

EditYourself is a diffusion-based video editing model for talking heads, enabling transcript-driven lip-syncing, insertion, removal and retiming of speech while preserving identity and visual fidelity.

Semantify: Simplifying the Control of 3D Morphable Models using CLIP
Omer Gralnik, Guy Gafni, Ariel Shamir
ICCV 2023
project page · arXiv

A self-supervised method that utilizes the power of CLIP VLM to simplify the control of 3D morphable models, by learning a mapping from natural language to a 3DMM basis.

NeRFace: Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
Guy Gafni, Justus Thies, Michael Zollhöfer, Matthias Nießner
CVPR Oral 2021
project page · arXiv

The first method to combine NeRFs with 3DMM to reconstruct and re-render human faces.

Miscellanea

🎓 Academic Service	Reviewer for ICCV, CVPR, SIGGRAPH and SIGGRAPH ASIA
👩‍🏫 Teaching	Introduction to Deep Learning (TU Munich) Seminar on Neural Radiance Fields (TU Munich) Calculus 3 (TAU)
✨ Additional Stuff	Common-Ground AI (Acq. by Apple in 2024) - Advisor to the CTO and R&D teams, guiding the research roadmap for 3D human reconstruction from mobile devices: faces/heads, meshing, neural rendering, dynamic animation of textures, hair reconstruction, and re-animation of the reconstructed geometry in real-time. As a master student at TUM, I worked on neural rendering of point clouds for novel view synthesis (2019), and face replacement using CycleGAN in texture space of 3DMMs (2018) I obtained my Bsc in Mathematics and Computer Science from Tel Aviv University (2017) Worked at PTC on Subdivision Surfaces and processing of its control meshes, contributing to the Creo CAD software (2017)

Feel free to steal this website's source code from Jon Barron. Do not scrape the HTML from this page itself, as it includes analytics tags that you do not want on your own website — use the github code instead. Also, consider using Leonid Keselman's Jekyll fork of this page.

Research

Miscellanea

Contact

Impressum & Datenschutzerklärung