Tay, S., Foo, C. S., Urano, D., Leong, R., & Low, B. K. H. (2023). Bayesian Optimization with Cost-varying Variable Subsets. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, & S. Levine (Eds.), Advances in Neural Information Processing Systems (Vol. 36, pp. 3008–3031)
Abstract:
We introduce the problem of Bayesian optimization with cost-varying variable subsets (BOCVS) where in each iteration, the learner chooses a subset of query variables and specifies their values while the rest are randomly sampled. Each chosen subset has an associated cost. This presents the learner with the novel challenge of balancing between choosing more informative subsets for more directed learning versus leaving some variables to be randomly sampled to reduce incurred costs. This paper presents a novel Gaussian process upper confidence bound-based algorithm for solving the BOCVS problem that is provably no-regret. We analyze how the availability of cheaper control sets helps in exploration and reduces overall regret. We empirically show that our proposed algorithm can find significantly better solutions than comparable baselines with the same budget.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Agency for Science, Technology and Research - Advanced Manufacturing and Engineering Programmatic Funds
Grant Reference no. : A20H6b0151
This research / project is supported by the Agency for Science, Technology and Research - Advanced Manufacturing and Engineering Industry Alignment Fund - Pre Positioning (IAF-PP)
Grant Reference no. : A19E4a0101