Hi! I am a PhD student at the Machine Learning Department, CMU, where I am co-advised by Prof. Andrej Risteski and Prof. Zachary Lipton. I am interested in the theory and empirics of Machine Learning.

My work focuses on using large models for Scientific Discovery. Towards this, I am especially interested in reliable and efficient modeling of multimodal data—with a special emphasis on continuous data (videos, PDEs, physical dynamics) and structured discrete data (graphs, and language). I am also working on improving the reasoning capabilities of large models for science, with a special emphasis on Math and Physics.

In Summer of 2024, I was an intern at Microsoft New England, where I worked with David Alvarez-Melis, Nicolo Fusi and Lester Mackey. Here, I worked on improving the zero-shot reasoning capabilities of large language models (LLMs) by utilizing context dependent embeddings!!

In Summer of 2022, I was an intern at the Blueshift Team at Google, where I had the pleasure to work with Guy Gur-Ari, Jascha Sohl-Dickstein, Yasaman Bahri and Behnam Neyshabur and worked on understanding the out-of-distribution generalization of large language models using synthetic data.

Previously, I completed my Masters in Robotics at CMU, during which I was advised by Prof. Kris Kitani and was a recipient of the Siebel Scholarship, 2019. I did my Bachelors with Honors in Electrical Engineering from Indian Institute of Technology, Hyderabad where I worked with Prof. Vineeth N. Balasubramanian.

—I am on the 2024-2025 Job market!—

---

Selected Publications

Chimera: State Space Models Beyond Sequences
Aakash Lahoti^*, Tanya Marwah^*, Albert Gu
In Submission

On the Benefits of Memory for Modeling Time-Dependent PDEs
Ricardo Buitrago Ruiz, Tanya Marwah, Albert Gu, Andrej Risteski
In Submission

Towards characterizing the value of edge embeddings in GNNs
Dhruv Rohatgi, Tanya Marwah, Zachary C. Lipton, Jianfeng Lu, Ankur Moitra, Andrej Risteski
In Submission

UPS: Towards Foundation Models for PDE Solving via Cross-Modal Adaptation
Junhong Shen, Tanya Marwah, Ameet Talwalkar
ICML AI4Science Workshop, 2024 (Spotlight)
Transactions on Machine Learning Research (TMLR), 2024

Deep Equilibrium Based Neural Operators for Steady-State PDEs
Tanya Marwah^*, Ashwini Pokle^*, J. Zico Kolter, Zachary C. Lipton, Jianfeng Lu, Andrej Risteski
Neural Information Processing Systems (NeurIPS), 2023

Neural Network approximations of PDEs Beyond Linearity: A Representational Perspective
Tanya Marwah, Zachary C. Lipton, Jianfeng Lu, Andrej Risteski
International Conference on Machine Learning (ICML), 2023

Disentangling the Mechanisms Behind Implicit Regularization in SGD
Zachary Novack, Simran Kaur, Tanya Marwah, Saurabh Garg, Zachary C. Lipton
International Conference on Learning Representations (ICLR), 2023

Parametric Complexity Bounds for Approximating PDEs with Neural Networks
Tanya Marwah, Zachary C. Lipton, Andrej Risteski
Neural Information Processing Systems (NeurIPS), 2021 (Spotlight)

Attentive Semantic Video Generation using Captions
Tanya Marwah^*, Gaurav Mittal^*, Vineeth N Balasubramanian
IEEE International Conference on Computer Vision (ICCV), 2017

---

Working Paper

Improving Zero-Shot Reasoning Using Dynamic Non-Verbal Tokens
Tanya Marwah, Zhili Feng, Lester Mackey, Nicolo Fusi, David Alvarez-Melis