爱立信在实际应用中的大型语言模型实践

Measuring the Quality of GenAISystemsLiang YuGenAI: Generative Artificial IntelligenceA developer works at Ericsson Mobile Financial Services (EMFS)A researcher at BTH (50%)Our Journey•Dec 2022 to present•New use cases are emerging.•How to measure those emerging use cases?GenAIUse Case Use Case Use Case Use Case Use Case Use Case Use Case Use Case Use CaseAI adoptionYear2022NLP modelPoCingScenarioExperimentsLLMsAPIsUse CaseUse CaseUse Case Use Case Use Case Use CaseTry out20232024NLP: Natural Language Processing; LLM: Large Language Model2You can’t manage what you can’t measure.– Peter Drucker3Our researchMapping metrics to ISO quality characteristics.4Research finding (1/2)Metrics can be used to measure ISO quality characteristics.5Metric: a quantifiable measure to assess how well a system performs.Research finding (2/2)Baseline: the foundation for evaluating LLM’s outputs before implementing any adaptation or enhancements.6ProcessProcess: Identify metrics and metric data.Baseline<AI