Kaustubh Sharma

Hi. I’m Kaustubh, a junior at IIT Roorkee studying Electrical Engineering.
I am a curious undergrad researcher working mainly on probabilistic modelling, mechanistic interpretability, foundation models, and a few projects in generative modelling.

I’m working in the P-square lab with Prof. Parikshit. Our recent work focuses on scaling prior-fitted networks for improved few-shot uncertainty estimation, using a new attention mechanism inspired by Gaussian Processes. (Preprint, Code).
We are also building a foundation model for amortized kernel hyperparameter discovery.

Mechanistic interpretability (something I really care about):
I lead a small project with friends in the Data Science Group on understanding the roles of attention and MLP components in LLMs at a domain level. (Preprint).
We initially began by searching for domain-specific circuits. (Project Website).

Other projects & explorations: As a curious starter, I’ve worked on exploratory projects including applying diffusion methods to structured physics data in Sparse diffusion models (CERN), diffusion models for personalized generation in Image Alchemy and low-resolution media forensics.

Outside of the lab and acads, I need good music to function. To take a break, I sing and play the piano. I’m also on the swimming team at IITR and I absolutely love travelling around!

If you’re curious about my work, feel free to explore my projects or CV. And if you’d like to collaborate, chat, or want to share a song recommendation - just say hi - my contact details are on the top left.