Nithin Gopalakrishnan Nair

Nithin Gopalakrishnan Nair

I am a Research Engineer at Apple, where I work on long-horizon agents for human interaction understanding and the simulation and evaluation infrastructure used to develop them. I completed my Ph.D. in Electrical and Computer Engineering at Johns Hopkins University, working under the supervision of Dr. Vishal M. Patel at the VIU Lab. Prior to JHU, I obtained my dual degree (B.Tech & M.Tech) in Electrical Engineering from the Indian Institute of Technology Madras, where I conducted research on image reconstruction with Dr. A.N. Rajagopalan at the IPCV Lab.

My research spans large-scale model training, generative modeling, multimodal learning, synthetic data, and 3D understanding. During my Ph.D., I worked on controlling and composing diffusion models and improving generalization with synthetic data.

I find the theory behind diffusion models fascinating. Fun fact: the basics were proposed by Einstein! Over the past several years, I have contributed to advances in image, video, and 3D generation using these models. I welcome collaborations with researchers who share an interest in generative modeling. Please feel free to reach out.

  1. A fine token grid is coarsened, then a transformer predicts the next finer scale. The arrows show the corresponding discrete diffusion steps.

    WACV 2027

    Scale-wise VAR is Secretly Discrete Diffusion

    Amandeep Kumar*, Nithin Gopalakrishnan Nair*, Vishal M Patel

    In Winter Conference on Applications of Computer Vision (WACV), 2027
    Accepted in 2026; forthcoming.
  2. The Tok-D transformer uses a style embedding to modulate source and target tokens before self-attention.

    ICCV 2025

    Scaling Transformer-Based Novel View Synthesis Models with Token Disentanglement and Synthetic Data

    Nithin Gopalakrishnan Nair, Srinivas Kaza, Xuan Luo, Vishal M Patel, Stephen Lombardi, Jungyeon Park

    In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025
  3. GenDeg takes clean images, text, and degradation statistics as inputs. Structure correction and intensity filtering select training pairs.

    CVPR 2025

    GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-in-One Image Restoration

    Sudarshan Rajagopalan, Nithin Gopalakrishnan Nair, Jay N Paranjape, Vishal M Patel

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025
  4. Two task-specific controllers merge and unmerge features before passing them to the Stable Diffusion decoder.

    ECCV 2024

    Maxfusion: Plug&play multi-modal generation in text-to-image diffusion models

    Nithin Gopalakrishnan Nair, Jeya Maria Jose Valanarasu, Vishal M Patel

    In European Conference on Computer Vision, 2024
  5. A denoising U-Net generates several density maps from a crowd photograph. The detections from those maps are fused.

    CVPR 2024

    Diffuse-Denoise-Count: Accurate Crowd-Counting with Diffusion Models

    Yasiru Ranasinghe, Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, Vishal M Patel

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Mar 2024
  6. A face prediction and a semantic map feed a parsing-based steering function. Its gradient adjusts the prediction.

    ICCV 2023

    Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis

    Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal M. Patel, Tim K. Marks

    In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2023
  7. A face sketch and a brown-hair prompt guide two diffusion predictions to generate a portrait.

    CVPR 2023

    Unite and Conquer: Plug & Play Multi-Modal Synthesis Using Diffusion Models

    Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, Vishal M Patel

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2023