弹性AI构建容错AI系统

Dan RabinovitsjVP, Engineering MetaResilient AIBuilding Fault-Tolerant AI SystemsArtificial intelligence (AI) is having quite a momentAI-enabled creation toolsText-to-image generationA hedgehog playing chessLarge Language Models(LLMs)Llama 3.1Source: Meta for Business. 'Culture Rising: 2023 Trends Report.' 2023. pushed our model training tonew heights, leveraging a significantly optimized full training stack16K H100 GPUsused to train Llama 3.1 405B>15T tokensTRAINED AT UNPRECEDENTED SCALEThe Challenge of Scale: Llama 3’s Infrastructure6K clustersJob size: 128-512 GPUs202216-24K clustersJob size: 16K GPUs2023AI jobs at scale: massive change in 2023AI jobs at scale: TodaySoftware InfraPhysical InfrastructureLlama2024Training @ Scale is Not Linear! Not scaling linearlyThroughput# of GPUsOne Small StepInterruptions302010025k50k75k100k250k500k750k1 millionInterruptions per HourNumber of GPUsMore GPUsequalsMore FailuresRoadmap to Resilient AI: Metrics Driven OutcomesEffective Training TimeE2