组合多变量多臂土匪及其在情景强化学习等领域的应用-李帅

Shuai Li(https://shuaili8.github.io/)Shanghai Jiao Tong UniversityThis work is published at ICML 2024Oct13,2024https://arxiv.org/abs/2406.01386Driving questionCancombinatorialMABhandledecision-makingsystems with states?WhatistherelationshipbetweenRLandcombinatorialMAB?Main technicalresult●EpisodicRLisaninstanceofCMAB●Plug-inCMABalgorithms/analysisthatachievesnear-optimalregret●IntegratingRLstructureachievesminimax optimalregretConnectionPlug-inSolutionMinimaxOptimal•Background of multi-armed bandit (MAB) and combinatorial MAB (CMAB)•Episodic RL and its connection with CMAB•CMAB view of solving episodic RL•Background of multi-armed bandit (MAB) and combinatorial MAB (CMAB)•Episodic RL and its connection with CMAB•CMAB view of solving episodic RLmax𝑟(𝑖)𝑠.𝑡.𝑖∈[𝑚]●𝑟(𝑖):reward/utility fordecision𝑖●[𝑚]: mcandidatedecisionsDrivingRecSysLLMselectionObjective: maximize expectedtotalrewards𝔼σ𝑡=1𝑇𝑋𝑡●Anagentand𝒎arms●Eacharm𝑖∈𝒎hasarewarddistribution𝑷𝒊withunknownmean𝒓𝒊●Ineachro