Navid Rezazadeh
Machine Learning Research Engineer at Apple · Ph.D., UC Irvine
I work on the mathematical foundations of large language models and generative systems — LLM decoding, transformer verification, generative modeling, and GPU-accelerated scientific computing. My recent work on geometry-aware LLM decoding is an ICML 2026 spotlight.
Selected Research
Geometry-Aware Decoding for LLMs
An LLM decoding algorithm combining Wasserstein-regularized truncation with mass penalties — producing more reliable, better-calibrated generation.
Vertex-Softmax: Tight Transformer Verification
An exact softmax optimization method that yields strictly tighter certified bounds for transformer verification than standard relaxation-based approaches.
Backtracking Dynamics & Early Exit
Investigates how LLMs revise reasoning mid-generation, and proposes early-exit criteria that reduce wasted compute without sacrificing answer quality.
Learning Contraction Policies From Offline Data
Contraction-theoretic policy learning from offline trajectories with formal stability guarantees.
Experience
Apple, Machine Learning Research Engineer
- Vertex-Softmax. Exact softmax optimization yielding tighter certified bounds for transformer verification than relaxation-based methods. Open-sourced.
- Geometry-Aware Decoding. Co-developed Wasserstein-regularized truncation with mass penalties for LLM generation. ICML 2026 spotlight.
- Backtracking Dynamics. Investigated mid-generation reasoning dynamics in LLMs; proposed early-exit criteria for efficient inference.
- Transformer Forecasting. Designed transformer and Informer architectures for short-horizon forecasting on high-resolution signals; benchmarked against ARIMA / ARMA under rolling out-of-sample validation.
- Generative Modeling. Built GAN, diffusion, and copula-based surrogates for scientific simulators; validated with MMD, feature statistics, and chi-square distributional tests.
- GPU Scientific Computing. Re-architected scalar pipelines into batched PyTorch/CUDA with multi-GPU data parallelism — multi-week runs compressed to sub-hour with CPU-reference parity.
UC Irvine, Graduate Research Assistant
- Learning for Control. Jointly learned a neural control policy and its contraction metric from offline trajectory data, enforcing stability guarantees via contraction theory. NeurIPS SafeRL Best Paper (IEEE RA-L 2022).
- Probabilistic Estimation. Built Kalman, sigma-point, and particle filtering methods for probabilistic localization and state estimation in noisy networked systems.
- Distributed Optimization. Developed distributed algorithms for multi-agent submodular maximization and persistent monitoring with provable optimality bounds, using multilinear extensions, stochastic Pipage rounding, and receding-horizon policy design (Automatica 2023, Automatica 2021).
- Private Networked Systems. Designed additive-obfuscation privacy-preserving consensus protocols with formal deterministic privacy guarantees against network eavesdroppers while preserving convergence to the true average (IEEE TCNS 2024).
Publications & Manuscripts
- Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization. arXiv.
- Backtracking Dynamics and Early Exit. arXiv.
- Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for LLMs. ICML 2026 (spotlight).
- Learning Contraction Policies From Offline Data. IEEE RA-L 2022 · NeurIPS SafeRL Best Paper Award.
- A Study of Privacy Preservation in Average Consensus Algorithm via Deterministic Obfuscation Signals. IEEE TCNS, 2024.
- Distributed Strategy Selection: A Submodular Set Function Maximization Approach. Automatica, 2023.
- Distributed Submodular Maximization: Trading Performance for Privacy. IEEE CDC, 2022.
- A Sub-Modular Receding Horizon Solution for Mobile Multi-Agent Persistent Monitoring. Automatica, 2021.
- Multi-Agent Maximization of a Monotone Submodular Function via Maximum Consensus. IEEE CDC, 2021.
- A Sub-Modular Receding Horizon Approach to Persistent Monitoring for a Group of Mobile Agents Over an Urban Area. IFAC-PapersOnLine, 2019.
- Privacy Preservation in Continuous-Time Average Consensus Algorithm via Deterministic Additive Obfuscation Signals. arXiv, 2019.
- Privacy Preservation in a Continuous-Time Static Average Consensus Algorithm Over Directed Graphs. ACC, 2018.
Core Technical Skills
- ML / AI
- LLM decoding, transformer verification, sequence models, generative models (GANs, diffusion, copulas), simulator surrogates, time-series learning, calibration, model validation.
- Mathematics
- Optimization, stochastic processes, graph algorithms, control and estimation, linear algebra, Monte Carlo methods.
- Tools
- Python, PyTorch, CUDA, multi-GPU training, NumPy, pandas, CuPy, TensorFlow, MATLAB, C / C++, Git.
Education
Ph.D., Mechanical & Aerospace Engineering
University of California, Irvine · GPA 3.99 / 4.00
Thesis: Distributed Strategy Selection Over Graphs — Optimality and Privacy.
M.S., Mechanical & Aerospace Engineering
University of California, Irvine · GPA 4.00 / 4.00
B.S., Mechanical Engineering
Sharif University of Technology · GPA 3.85 / 4.00
Selected Coursework
- Mathematics
- Real Analysis; Advanced Calculus; Linear Algebra; Optimal Control.
- Probability & Statistics
- Stochastic Processes; Bayesian Data Analysis; Advanced Estimation and Detection; Classification, Parameter Estimation, and Filtering; Probabilistic Learning.
- ML & Optimization
- Deep Learning and Sequence Models; Convex Optimization; Advanced Optimization Methods; Algorithms.
Honors
- NeurIPS SafeRL Best Paper Award
- Holmes Fellowship, UCI Mechanical & Aerospace Engineering
- Samueli Endowed Fellowship, UCI
- Semifinalist, Iran National Physics & Mathematics Olympiads
- Ranked 250th among 300,000+ in Iran national university entrance exam