------------------->

Azalia Mirhoseini




I am a member of technical staff at Anthropic. My research focuses on advancing the capabilities and reliability of large language models.
Prior to Anthropic, I was a staff research scientist and manager at Google Brain. I co-founded Google Research's ML for Systems team, researching automating and optimizing systems and chip design through novel learning-based methods. Our work on deep reinforcement learning for floorplanning optimization has been used in multiple generations of Google's flagship AI accelerator (TPU).
I have worked on a diverse portfolio of AI and Systems research projects, with publications in Nature, ICML, ICLR, NeurIPS, UAI, ASPLOS, SIGMETRICS, DAC, DATE, and ICCAD.
I have received a number of awards, including the MIT Technology Review 35 Under 35, the Best Ph.D. Thesis at Rice University's ECE Department, and a Gold Medal in the National Math Olympiad in Iran. My work has been covered in various media outlets, including MIT Technology Review, IEEE Spectrum, The Verge, Times of London, ZDNet, VentureBeat, and WIRED.

Research

Highlights:

  • Developed FAST, a full-stack accelerator search framework with a broad design search space, including hardware datapath, software scheduling, and compiler passes such as operation fusion. When evaluated on vision and NLP benchmarks, custom designs generated by FAST show up to 6x improvement in Perf/TDP vs. TPU-v3. We also demonstrated that accelerators with 2x higher Perf/TCO can become ROI-positive at moderate datacenter deployments (ASPLOS'22).

  • Developed lossless accelerators using approximate multipliers for fast and energy-efficient inference on vision benchmarks (DATE'22).

  • Our work on deep reinforcement learning for chip floorplanning was published in Nature and used in production to design next generation Google TPUs, with potential to save thousands of hours of engineering effort for each next generaion. I cofounded/led this project, a larger team effort with 20+ engineers and researchers across different organizations, including Google Research and Platforms (Nature'21).

  • Developed deep reinforcement learning algorithms to do model parallelism that speed up deep network training by more than 60% over top-performing baselines (ICML’17, ICLR’18, and NeurIPS’20).

  • Introduced mixture-of-expert large language models that allow efficient training of massive neural networks with over 100 billion parameters, yielding state of the art results on established language modeling and translations tasks (ICLR’17).

    • Publications:

    For a full list of publications, please visit my Google Scholar page.

    >

    Recent Talks and Interviews

  • Keynote speaker at CVPR workshop on Dynamic Neural Networks meets Computer Vision 2021 (Slides, Video)
  • Keynote speaker at the workshop on Graph Neural Networks and Systems at MLSys, 2021
  • Invited speaker at IPAM 2021 Workshop on Deep Learning and Combinatorial Optimization
  • Invited speaker at NVIDIA GTC, 2021 (Video)
  • Panelist at IEEE Custom Integrated Circuits Conference ML for Chip Design Forum, 2021
  • Keynote Speaker at MIT EmTech China, 2020
  • Keynote Speaker at Ray summit, 2020 (Video)
  • Keynote Speaker at Open Data Science Conference, 2020
  • Keynote Speaker at International Supercomputing ML day, 2019
  • Keynote Speaker at ML in HPC Workshop at Supercomputing, 2018
  • Interviewed by MIT Technology Review, 2020
  • Interviewed by IEEE Spectrum, 2020
  • Interviewed by ACM Learning Center, 2020 (Interview)
  • Interviewed by Towards Data Science, 2019 (Interview, Video)
    • >

    Awards and Honors

  • Published in Nature Flagship Journal, 2021
  • MIT Technology Review 35 Innovators Under 35, 2019 (Article)
  • Best Thesis Award at Rice University, ECE Department, 2015
  • Fellowships and scholarships from Microsoft Research, IBM Research and Schlumberger, 2010-2015
  • Gold Medal in National Math Olympiad, Iran, 2004
    • >

    Misc

    >

    Contact

    You can reach out by sending an email to azaliamirh (at) gmail.com.