cv

Undergraduate Research Assistant

P-square Lab, IIT Roorkee

Research on attention mechanisms for Prior-Data Fitted Networks (PFNs) and amortized kernel/hyperparameter inference for GP-style problems.

• Developed Decoupled-Value Attention (DVA) for PFNs to accelerate GP inference for physical equation solving (code: PSquare-Lab/DVA-PFN).
• Working on foundational architectures for amortized kernel hyperparameter inference and scaling PFNs.

Educator

Edufabrica Pvt. Ltd.

Delivered a 2-day workshop lecture to 200+ students across India on Generative AI.

Incoming Quantitative Research Intern (Quantitative Strategist)

Goldman Sachs

Selected for the quantitative strategist internship involving financial modelling, statistical analysis and quantitative research. (Upcoming)

B.Tech

Indian Institute of Technology Roorkee

Electrical Engineering

Micron AI Hackathon 2025 — Second Runner Up

Micron Technology

Second Runner Up at Micron AI Hackathon 2025.

JEE Advanced 2023

IIT / Joint Entrance Examination

Secured All India Rank 1624 among 100k+ applicants.

JEE Mains 2023

NTA

Secured All India Rank 1898 among 1.2M+ applicants.

NTSE Scholar 2021

NCERT

Awarded NTSE scholarship; among top ~2000 candidates from ~900k+ applicants.

KVPY Fellow 2022

IISc Bangalore / KVPY

Secured AIR 509 among ~200k+ applicants; KVPY fellowship recipient.

Dissecting Attention and MLP Roles: A Study of Domain Specialization in Large Language Models

Under review

First author (Kaustubh Sharma). Study on domain specialization in LLaMA 3-3B: forward pass profiling, probing, zero-out tests and analysis of fine-tuning shifts.

Decoupled-Value Attention for Prior-Data Fitted Networks: GP Inference for Physical Equations

arXiv / Under review

First author (Kaustubh Sharma). Proposed DVA attention to improve Gaussian process inference speed/accuracy inside PFNs for physical equation solving.

Image-Alchemy: Advancing Subject Fidelity in Personalized Text-to-Image Generation

DeLTa Workshop, ICLR 2025

First author (Kaustubh Sharma). Pipeline for personalizing Stable Diffusion XL improving subject fidelity using LoRA and segmentation-guided Img2Img to reduce overfitting/forgetting.

Explainable AI-Generated Image Forensics: A Low-Resolution Perspective with Novel Artifact Taxonomy

APAI Workshop, ICCV 2025 (Proceedings)

Solo author (Kaustubh Sharma). Developed an interpretable pipeline for AI-generated image detection at low (32×32) resolution with an artifact taxonomy and explainability framework.

Domain Circuit Discovery in LLMs - Mechanistic Interpretability

Investigating domain-specific knowledge emergence in LLaMA 3-3B; mapping domain-specific 'rooms' across transformer architecture and evaluating causal effects, probe separability, zero-out tests, hydra effect and fine-tuning shifts.

• Forward pass profiling
• Probe separability
• Zero-out tests
• Hydra effect analysis
• Fine-tuning shift evaluation

Decoupled-Value Attention (DVA) for PFNs

Designed and implemented DVA attention to speed up GP-like inference in Prior-Data Fitted Networks for physical equation solving; codebase and experiments developed with P-square Lab collaborators.

• DVA attention design and implementation
• GP inference time reduction for physical equation solving
• Scaling PFNs

Sparsity-Aware Representation Learning for Jet Image Generation via Guided Latent Diffusion

Sparsity-aware latent diffusion framework for generating high-energy physics jet images with a custom VAE and a mean-pulling mechanism to improve reconstruction quality.

• Custom VAE with sparsity reconstruction loss
• Latent diffusion mean-pulling mechanism