太阳成集团tyc7111cc学术报告

SCSG-BD: Improving the stochastically controlled stochastic gradient method by the bandwidth-based stepsize

报告时间：2024年7月12日星期五  9：30-10：10

报告人: 黄亚魁（河北工业大学）

报告地点：沙河校区国实E404

报告摘要: Stepsize plays an important role in both theoretical analysis and numerical performance of the stochastic gradient method. The bandwidth-based stepsize has the advantage of allowing us to adjust the stepsize within a banded region determined by some boundary functions. In this talk, based on the bandwidth-based stepsize, we introduce a new method, namely SCSG-BD, for smooth non-convex finite-sum optimization problems. For three different boundary functions, SCSG-BD converges sublinearly to a stationary point at a faster rate than the stochastically controlled stochastic gradient (SCSG) method under certain conditions. Moreover, SCSG-BD converges linearly to the solution if the objective function satisfies the Polyak-Lojasiewicz condition. We also introduce the 1/t-Barzilai-Borwein stepsize for practical computation. Numerical experiments demonstrate that SCSG-BD performs better than SCSG and its variants.

报告人简介：黄亚魁，河北工业大学准聘教授、硕士生导师，2015年博士毕业于西安电子科技大学，2015年7月至2017年5月在中国科学院数学与系统科学研究院从事博士后研究。主要研究兴趣包括梯度类算法理论及应用、大规模机器学习和分布式优化等领域的一阶算法，相关成果发表在SIAM Journal on Optimization、Journal of Scientific Computing、Computational Optimization and Applications等期刊，主持国家自然科学基金、河北省自然科学基金、中国博士后基金等科研项目。现任中国运筹学会数学规划分会理事、算法软件与应用分会常务理事，中国数学会计算数学分会理事，河北省运筹学会秘书长。

邀请人：崔春风