SK海力士人工智能专用计算内存解决方案从AiM设备到异构AiMX-xPU系统实现全面的LLM推理

SK hynix AI-Specific Computing Memory Solution: From AiMdevice to Heterogeneous AiMX-xPUSystem for Comprehensive LLM InferenceGuhyun Kim1, JinkwonKim1, NahsungKim1, WoojaeShin1, JongsoonWon1, HyunhaJoo1, HaerangChoi1, ByeongjuAn1, GyeongcheolShin1, DayeonYun1, JeongbinKim1, ChanghyunKim1, IlkonKim1, JaehanPark1, YosubSong1, ByeongsuYang1, HyeongdeokLee1, SeungyeongPark1, WonjunLee1, SeonghunKim1, YonghoonPark1, YousubJung1, Gi-Ho Park2, and EuicheolLim11SK hynix inc, 2Sejong University•Recap Accelerator-in-Memory (AiM) & AiMX•System Extensions of AiMXCard for Datacenter•AiM& AiMXfor On-device AI•Design Choices for Future AiM/AiMX•ConclusionRecap Accelerator-in-Memory (AiM) and AiMX© SK hynix Inc. This material is proprietary of SK hynix Inc. and subject to change without notice.3Is LLM Sustainable?Too Expensive Operating ExpenditureYearsCost© SK hynix Inc. This material is proprietary of SK hynix Inc. and subject to change without notice.4Large Language Model –Memory Bound(*) Assumptio