Double Inertial Stochastic Relaxed Forward-Backward-Forward Algorithm
Optimization Eruditorum, Volume 3, Issue 3, December 2026, Pages 183–208
Aviv Gibali
Department of Applied Mathematics, Holon Institute of Technology-HIT, 5810201, Holon, Israel
Grace Nnennaya Ogwo
Department of Mathematics, Shanghai Normal University, 100 Guilin Road, Shanghai 200234
Yekini Shehu
School of Mathematical Sciences, Zhejiang Normal University, Jinhua 321004, China
Abstract
We propose the double inertial stochastic relaxed forward-backward-forward (DI-SRIFBF) algorithm for solving monotone inclusions in Hilbert spaces. The method augments the stochastic FBF framework with two sequential inertial extrapolation steps, and it evaluates the single-valued operator via independent mini-batches at each forward step. This design avoids the correlation issues that arise when the same batch is reused, and it yields a stochastic conditioning that fits the Robbins--Siegmund template without hidden measurability assumptions. Under explicit, algebraically verifiable step-size restrictions that hold for every relaxation parameter \(\rho\in(0,2)\), we establish almost-sure weak convergence provided the inertial parameters and mini-batch variances are summable. We also prove an \(O(1/k)\) rate for the discrete velocity and, under strong monotonicity with geometrically growing batch sizes, linear convergence. Choosing the growth factor \(\tau=(1-\eta/2)^{-1}\) gives an \(\mathcal{O}(1/\varepsilon)\) oracle complexity. Finally, we derive explicit non-asymptotic bounds for biased stochastic oracles, showing that the iterates settle into an \(O(B^2/\mu^2)\) neighborhood of the unique solution. Numerical experiments on two-stage stochastic variational inequalities and group sparse learning confirm the theoretical predictions and demonstrate a practical advantage over single-inertial and non-inertial benchmarks.
Cite this Article as
Aviv Gibali, Grace Nnennaya Ogwo, and Yekini Shehu, Double Inertial Stochastic Relaxed Forward-Backward-Forward Algorithm, Optimization Eruditorum, 3(3), 183–208, 2026