------------------->

Azalia Mirhoseini




I am an Assistant Professor in the Computer Science Department at Stanford University. My research interest is in developing capable, reliable, and efficient AI systems for solving high-impact, real-world problems. My work includes generalized learning-based methods for decision-making problems in systems and chip design, self-improving AI models through interactions with the world, and scalable deep learning optimization. I also spend time at Google DeepMind and prior to Stanford, I spent several years in industry AI labs, including Anthropic and Google Brain. At Anthropic, I worked on advancing the capabilities and reliability of large language models. At Google Brain, I co-founded/led the ML for Systems team, with a focus on automating and optimizing computer systems and chip design. I received my BSc degree in Electrical Engineering from Sharif University of Technology and my PhD in Electrical and Computer Engineering from Rice University. My work has been recognized through the MIT Technology Review’s 35 Under 35 Award, the Best ECE Thesis Award at Rice University, publications in flagship venues such as Nature, and coverage by various media outlets, including MIT Technology Review, IEEE Spectrum, The Verge, The Times, ZDNet, VentureBeat, and WIRED.

Research

Highlights:

  • Constitutional AI (CAI) is a novel approach to train Large Language Models (LLMs) that are helpful, harmless, and honest. CAI introduces a new strategy that relies on AI feedback for self-improvement. In CAI, the LLM criticizes its own outputs based on a set of principles (the constitution) and revises its outputs to follow the constitution. This technique is used to finetune the LLM with supervised training first and later with reinforcement learning (based on the preference model trained on AI feedback). CAI reduces reliance on human supervision for training, improves transparency, and enables a faster approach to new domain adaptation (paper'22).

  • FAST is a full-stack accelerator search framework with a broad design search space, including hardware datapath, software scheduling, and compiler passes such as operation fusion. When evaluated on vision and NLP benchmarks, custom designs generated by FAST show up to 6x improvement in Perf/TDP vs. TPU-v3. We also demonstrated that accelerators with 2x higher Perf/TCO can become ROI-positive at moderate datacenter deployments (ASPLOS'22).

  • Developed lossless accelerators using approximate multipliers for fast and energy-efficient inference on vision benchmarks (DATE'22).

  • Circuit Trainig introduces a new approach to chip floorplanning based on deep reinforcement learning. This work was published in Nature and used in production to design next generation Google TPUs, with potential to save thousands of hours of engineering effort for each next generaion. I cofounded/led this project, a cross-functional effort with 20+ engineers and researchers across different organizations, including Google Research and Platforms (Nature'21, Statement).

  • Developed deep reinforcement learning algorithms to do model parallelism that speed up deep network training by more than 60% over top-performing baselines (ICML’17, ICLR’18, and NeurIPS’20).

  • Introduced mixture-of-expert large language models that allow efficient training of massive neural networks with over 100 billion parameters, yielding state of the art results on established language modeling and translations tasks (ICLR’17).

    • Publications:

    For a full list of publications, please visit my Google Scholar page.

    >

    Teaching

  • Fall 2023 - CS229s: Systems for Machine Learning
    • >

    Recent Talks and Interviews

  • Keynote speaker at ASPLOS, 2023
  • Keynote speaker at the 32nd Microelectronics Design and Test Symposium, 2023
  • Keynote speaker at NeurIPS workshop on New Frontiers in Graph Learning, 2022
  • Invited speaker at AICoM workshop on AI-Enhanced Co-Design for Next-gen Microelectronics, 2022
  • Keynote speaker at CVPR workshop on Dynamic Neural Networks meets Computer Vision, 2021 (Slides, Video)
  • Keynote speaker at MLSys workshop on Graph Neural Networks and Systems, 2021
  • Invited speaker at IPAM 2021 Workshop on Deep Learning and Combinatorial Optimization
  • Invited speaker at NVIDIA GTC, 2021 (Video)
  • Panelist at IEEE Custom Integrated Circuits Conference ML for Chip Design Forum, 2021
  • Keynote Speaker at MIT EmTech China, 2020
  • Keynote Speaker at Ray summit, 2020 (Video)
  • Keynote Speaker at Open Data Science Conference, 2020
  • Keynote Speaker at International Supercomputing ML day, 2019
  • Keynote Speaker at ML in HPC Workshop at Supercomputing, 2018
  • Interviewed by MIT Technology Review, 2020
  • Interviewed by IEEE Spectrum, 2020
  • Interviewed by ACM Learning Center, 2020 (Interview)
  • Interviewed by Towards Data Science, 2019 (Interview, Video)
    • >

    Awards and Honors

  • Published in Nature Flagship Journal, 2021
  • MIT Technology Review 35 Innovators Under 35, 2019 (Article)
  • Best Thesis Award at Rice University, ECE Department, 2015
  • Fellowships and scholarships from Microsoft Research, IBM Research and Schlumberger, 2010-2015
  • Gold Medal in National Math Olympiad, Iran, 2004
    • >

    Media

    >

    Contact

    Prospective students and postdocs: Thanks for your interest in working in our lab!  Please email your CV along with a note briefly describing research directions you are excited about and up to two past projects.
    If you are applying to Stanford, please make sure to mention your interest in working in Prof. Mirhoseini's lab in the application form as well as in your research statement.  
    You can reach out by sending an email to FIRSTNAME (at) stanford.edu.