超越硬件实现高效AI推理的全栈优化

FuriosaAI Inc. AI Hardware Summit 2024Hyunsik Choi, Head of SW Platform, Jihoon Yoon,Product Marketing ManagerBeyond Just Hardware Full-stack Optimization Towards Efficient AI InferenceFuriosaAI Inc. AI Hardware Summit 2024FuriosaAI founded & Launch Gen 1 vision NPU RNGD raw silicon sample arrivalFirst LLM demo 2017-2021 2024 May2024 JulyGPT3 inspired RNGD 2021 RNGD DevelopmentKick off 2022 FuriosaAI Inc. AI Hardware Summit 202401Mass AI adoption is bottlenecked02 Energy efficient AI inference03 Full-stack optimization for achieving efficiency Key Points FuriosaAI Inc. AI Hardware Summit 2024Source: Masanet et al. (2020), Cisco, IEA, Goldman Sachs ResearchAI has broken energy efficiency V100Gaudi 1A100MI100MI250XH100Gaudi 2MI300XGaudi 3B200FuriosaAI Inc. AI Hardware Summit 2024Electricity is already a huge financial and environmental burden on data centersSource: HARTING White Paper (2024)FuriosaAI Inc. AI Hardware Summit 2024AI inference will be everywhere.But is our infrastruc