用于提高数据中心计算密度的散热技术

Thermal Techniques for Data Center Compute DensityTom GarvensSupermicroVP Hardware Solutions Better Faster Greener © 2024 SupermicroAgendaBetter Faster Greener © 2024 Supermicro•GenAI LLM Era•Data Center Power and Cooling Challenges•Solutions and TCO •Future Trends8/23/20242175BLLMsPetascale Data SetsMassive GPU ComputeChatGPTGemini=LLM Parameters (GPT-3 175B) are like adjustable dials in a complex machine. More adjustments = more optimization (for LLM’s this is more nuanced text)300 Billion Tokens(100 Tokens = 75 words)Better Faster Greener © 2024 SupermicroGenAI LLM Era8/23/20243Tensor Processing•Heavy linear algebra matrix multiplication with bulk data transfers between GPU’s•Partition size defined by GPU memory in coherent domain Better Faster Greener © 2024 SupermicroLLM Training: Tensor Parallelism & Model PipelinesGPU Server 1GPU Server 2Models and Data Sets must be sub-divided to fit into GPU memory for performance (time) optimizationGPT-3: 175 billion parameters and 300