有向网络中一种快速的分布式随机算法.pdf

资源描述

1、Advances in Applied Mathematics A?,2024,13(2),848-868Published Online February 2024 in Hans.https:/www.hanspub.org/journal/aamhttps:/doi.org/10.12677/aam.2024.132081k?DDD?n?U9vF2024c1?28FF2024c2?22FuF2024c2?29F?k?UNX?zK?8IL?k8Ik?/L|Nesterov E|?E LSVRG?J?k?AB-LSVRG1wr?8InyJ?5?)u68K?L?J?yk?Ly?JczUXk?A

2、 Fast Distributed Stochastic Algorithmover Directed NetworksXiuhui GongSchool of Science,Hebei University of Technology,TianjinReceived:Jan.28th,2024;accepted:Feb.22nd,2024;published:Feb.29th,2024AbstractThis paper considers the distributed optimization in a multi-agent system over bal-anced directe

3、d networks.The global objective function describes a finite sum of alllocal objective functions on the networks.Combining the distributed loopless variance:D.k?J.A?,2024,13(2):848-868.DOI:10.12677/aam.2024.132081Dreduction method with Nesterov momentum strategy,a fast distributed stochastic al-gorit

4、hm is developed,named ABN-LSVRG.For smooth and strongly convex objectivefunctions,it is proved that ABN-LSVRG has a linear convergence rate.Based on thedistributed logistic problem,simulation results show that ABN-LSVRG performs bet-ter in comparison with some distributed algorithms.KeywordsDistribu

5、ted Optimization,Multi-Agent System,Directed Networks,DistributedVariance Reduction,Momentum AccelerationCopyright c?2024 by author(s)and Hans Publishers Inc.This work is licensed under the Creative Commons Attribution International License(CC BY 4.0).http:/creativecommons.org/licenses/by/4.0/1.z?2A

6、uS 1,2,?X 3?4?+.?SK?z?8I?/,z8I fi=?UN i.?kUN?8IL&O?).?UNm?&?,)?k?.3?,UNm?&V?.?3SA,UNm?&k?.duk?zK?25.?k?z.N(5?J,XFe(DGD)5,FJl 68.u?,?k?p?z?u.du?8I?F,?!:3F?O?D(?Z6,(?FO.d,kF5Cq?F.DGD?CNFe DSGD 9,10 u)aK.?,Fl 11,12 DSGD(J?Fl(DSGT)13,TU?DSGD?.,?,duF?3,DSGD DSGT 3?eU?)?+S.8z?u 1416,Nk?z?J.z 17 3Fl?:e|?EJ

7、?GT-SAGA GT-SVRG.3?e5?).J?=u?-?k?,?I?EV?-?.,?,uk?zK,?EV?(J?.?)k?aK,push-sum 18 DSGD(J?gradient-push(SGP)19,20.TL|push-sum E|?1A?O,l?k?5.?,z 21 Fl SGP(J?SADDOPT,T3DOI:10.12677/aam.2024.132081849A?DFl?eU?SGP?.L1?-?,z 22J?S-AB.aq/,k?F?K,3?eU?)?+S.d,z 23|SAGA?E 14 J?15?)?Push-SAGA.?5,z 24 3 S-AB SAGA J?

8、AB-SAGA,T;?OA?,?O&.u SAGA?IFL5;F,?);?K.?,;?FJy?.z 25|V?E SVRG 15 J?Push-SVRG AB-SVRG,?u Push-SAGA AB-SAGA,?I;m.C,z 26?E 27 J?Push-LSVRG,Tk5?$?;?.?,k?z?u.z 28 GT-SAGA heavy-ball(J?kFl-?z.1wr?8I,Ty?.L S-AB Nesterov(,k?Nesterov 29?J.?u,|Nesterov E|,?J?k?z.?oNi1L,?Ni1L?.InL nn?,1nL n?1?.?X Y?S X Y,?x,xi

9、L?1i?,x x OL?,diag(x)L x?)?.?X,(X)L?XL?g(XJ3),=X=limkXk.?1?A,r 1nOL?AuA?1?mA?,=r1Tn=1 A=1nTr.u?B,k B=c1Tn.?x,y Rp?-S,hx,yir=xTdiag(r)y hx,yic=xTdiag(c)1y.kxkr kxkcL?-.|r|cOLd k kr k kcp?,=X Rnn,k|X|r=?diag(r)Xdiag(r)1?2|X|c=?diag(c)1Xdiag(c)?2.2.Kd n UN?k?,?kUN)ezK:minxRpF(x)=1nnXi=1fi(x),fi(x)=1min

10、Xj=1fi,j(x),(1)x Rp?C,8I fi:Rp R=?UN i.z8I?L mi8I?/.?kUN?8IL&O?).UN?&E?Lk?G=(V,E)?1.V=1,n L?UN?8,E L?UNm&?8.NiniLUN i?8,=UN i U?8zUNux?&E.n,NoutiLUN i?8,=UN i U?&Eux?8?zUN.?e5,?IO?b?.b?1.z?8I F r?,=i V x1,x2 Rp,3 0,kF(x2)F(x1)+F(x1)T(x2 x1)+2kx1 x2k2.DOI:10.12677/aam.2024.132081850A?Db?2.z8I fi L-1w

11、?,=i V x1,x2 Rp,3 L 0,kkfi(x1)fi(x2)k26 Lkx1 x2k2.b?3.G r?.b?1 L?8I3?).b?2 L?8I F L 1w?,=LL?8I F?.3.?!k0?(LSVRG),?S-AB,(Nesterov J?ABN-LSVRG.-ikO?F f?:.3zgS“,zUN i l 1,n?lik,:ik?#5K:1m?V?xik1,K ik=ik1.31 k gS“,LSVRG?S“:gik=fi,lik(xik)fi,lik(ik)+fi(ik).(2)?zK,z 22 1?-?J?S-AB,T SADDOPT;A?O,?O&?.3z,E|?

12、2?5,X:heavy-ball Nesterov.?LSVRG S-AB(,Nesterov,J?,ABN-LSVRG.31 k gS“zUN#oC=yik,sik,gik xik.?S“si0=xi0 gi1=yi1=f(x0)i V.J?ABN-LSVRG S“:yik=nXj=1bijyik1+gk gk1,sik+1=nXj=1aijxik yik,xik+1=sik+1+(sik+1 sik).(3)aij bijOve5aij=(0,j Nini,0,K,Pnj=1aij=1,i,bij=(0,i Noutj,0,K,Pni=1bij=1,j.(4)?-?A=aij Rnn1?,

13、B=bij Rnn?.sk,xk,gk,yk Rpn,i.e.sk=s1k,.,snkT,xk=x1k,.,xnkT,gk=g1k,.,gnkT,yk=DOI:10.12677/aam.2024.132081851A?Dy1k,.,ynkT,A=A Ip B=B Ip.|XJ ABN-LSVRG?d?yk=Byk1+gk gk1,sk+1=Ask yk,xk+1=sk+1+(sk+1 sk).(5)4.53?15c?yk=1n1Tnyk,f(xk)=1n1Tnf(xk),sk=Trsk,gk=1n1Tngk,f(xk)=f1(x1k),fn(xnk).en?1?-?5,y 22.n1.eb?3

14、,k 0,?-?AB?x Rp,kkAx Axkr6 Akx Axkr,kBx Bxkc6 Bkx Bxkc.(6)A=gA Agr 1 B=gB Bgc 0k yk=gk,E yk|Fk=E gk|Fk=f(xk).en5y-,z 24,29.n3.29 eb?2,dJ?)?S“xkk0.k 0,kkf(xk)F(sk)k26Lnksk 1Tn skk22+Lnk sk xk2.(8)n4.24 eb?1 2,?8I F r L 1w?.XJ 0 1L,s Rpkks F(s)xk26(1 )ks sk2.(9)n5.17-G RnpnpK?e G 0)k(G)0,e?Ekykk22 645

15、nkck22L2rEksk 1n skk2r+25n2kck22L2Ek sk xk22+45nkck22L22Eksk sk1k22+10kck22Evk+5cEkyk Bykk2c.(10)y:5?B=c1Tn,kkykk2=kyk c1Tnyk+c1Tnykk26 0.5ckyk c1Tnykkc+kc1Tnykk26 0.5ckyk c1Tnykkc+nkck2kgk f(xk)k2+nkck2kf(xk)F(sk)k2+nkck2kF(sk)F(x)k26 0.5ckyk Bykkc+nkck2kgk f(xk)k2+nkck2Lrksk 1n skkr+nkrk2Lksk sk1k

17、.eb?2 3.XJ 0 61kck2Lqh10n 0 6 0,?DOI:10.12677/aam.2024.132081853A?D?XeEksk+1 1n sk+1k2r 6(1+2A2+90nhkck22L21 2A2)Eksk 1n skk2r+50n2kck22L2r1 2A2Ek sk xk22+(1+2A)r+90nkck22L2r1 2A2)Eksk sk1k22+20kck22r1 2A2Evk+10rc1 2A2Ekyk Bykk2c,(14)h=r/r.y:5?A=1nTr xk=Trxk,d sk?ksk+1 1n sk+1k2r=kAxk Axk(In A)ykk2r

18、6kAxk Axkk2r+2kykk2r+2kAxk Axkkrkykkr6kAxk Axkk2r+2kykk2r+(kAxk Axkk2r+12kykk2r),(15)3?O-=12A22A 1 kksk+1 1n sk+1k2r61+2A22AkAxk Axkk2r+22r1 2Akykk22,(16)ksk+1 1n sk+1k2r6 2kAxk Axkk2r+2r2kykk22.(17)d sk?kAxk Axkk2r6 2Aksk 1n skk2r+22Arksk sk1k22+22Aksk 1n skkrksk sk1kr6 2A(1+)ksk 1n skk2r+22Arksk s

19、k1k22.(18)L(10)(18)(16)3?Eksk+1 1n sk+1k2r 6(1+2A2+90nhkck22L21 2A2)Eksk 1n skk2r+50n2kck22L2r1 2A2Ek sk xk22+(1+2A)r+90nkck22L2r1 2A2)Eksk sk1k22+20kck22r1 2A2Evk+10rc1 2A2Ekyk Bykk2c.(19)DOI:10.12677/aam.2024.132081854A?Dd?,e(10)(17)(16)3?Eksk+1 1n sk+1k2r 6(22A(1+)+90nhkck22L22)Eksk 1n skk2r+50n2

20、kck22L2r2Ek sk xk22+(42Ar+90nkck22L2r2)Eksk sk1k22+20kck22r2Evk+10rc2Ekyk Bykk2c.(20)XJ 0 61kck2Lqh10n,kEksk+1 1n sk+1k2r 6 11Eksk 1n skk2r+5nrEk sk xk22+13rEksk sk1k22+20kck22r2Evk+10rc2Ekyk Bykk2c.(21)du?Y?y.y?.n8.eb?1-3.XJ 0 6 min12nTrcL,15nTrc,14TrcL2 0 6 0,?mYXeEk sk+1 xk22 65TrcL2rEksk 1n skk2

21、r+(1 nTrc4)Ek sk xk22+5TrcL22Eksk sk1k22+2.5(Trc)22Evk+5krk22cnTrcEkyk Bykk2c.(22)y:d sk?k sk+1 xk22=k sk x nTrcyk Tr(yk Bgk)k226 k xk x gkk22 2+krk22c2kyk Bykk2c=I1+I2+krk22c2kyk Bykk2c.(23)=nTrc.3(23)?kEk sk+1 xk22 6 EI1|Fk+EI2|Fk+krk22c2Ekyk Bykk2c|Fk.(24)DOI:10.12677/aam.2024.132081855A?Dk,(25)?

23、F(sk)f(xk)k22+1n2Ekf(xk)gkk22|Fk.(26)|(13)n 3?EkF(sk)gkk22|Fk 610L2nrEksk 1n skk2r+4L2nEk sk xk22+10L2n2Eksk sk1k22+2n2Evk.(27)e 0 615,(27)(25)(kEI1|Fk 64L2nr Eksk 1n skk2r+(1 +4 2L2n)Ek sk xk22+4L2n 2Eksk sk1k22+2n2 2Evk.(28)y3 I2I2=2 6 2k sk x ykk2kTr(yk Byk)k26(2I1+krk22c2kyk Bykk2c)(29)3(29)-2=n

24、Trc43(29)?EI2|Fk 6 4EI1|Fk+4krk22cnTrcEkyk Bykk2c|Fk.(30)DOI:10.12677/aam.2024.132081856A?DXJ 0 6n4L2,(30),(29),(28)(24).5?1+4654(1+4)(1 2)6 1,nTrcO?Eksk+1 xk22 65TrcL2rEksk 1n skk2r+Ek sk xk22+5TrcL22Eksk sk1k22+2.5(Trc)22Evk+5krk22cnTrcEkyk Bykk2c.(31)XJ3(25)-1=1,K EI1|Fk LXeEI1|Fk=Ek sk x ykk22|F

25、k6 2Ek sk xk22+2 2EkF(sk)gkk22|Fk620L2nr 2Eksk 1n skk2r+(2+8L2n 2)Ek sk xk22+20L2n 22Eksk sk1k22+2n 2Evk(32)XJ 0 6n2L,?EI1|Fk 620L2nr 2Eksk 1n skk2r+4Ek sk xk22+20L2n 22Eksk sk1k22+2n 2Evk(33)3(29)-2=1kEI2 6 EI1+krk22c2Ekyk Bykk2c(34)(33)(34)(24)?Eksk+1 xk22 640L2nr 2Eksk 1n skk2r+8Ek sk xk22+40L2n

26、22Eksk sk1k22+4n2 2Evk+2krk22c2Ekyk Bykk2c.(35)O nTrc,y?.n9.eb?2.XJ 0 6 0,e?Eksk+1 skk226(16r+90nkck22L2r2)Eksk 1n skk2r+50n2kck22L22Ek sk xk22+2(2h+45nkck22L22)2Eksk sk1k22+20kck222Evk+10c2Ekyk Bykk2c.(36)DOI:10.12677/aam.2024.132081857A?Dy:d sk?ksk+1 skk22=kAxk yk skk226k(A In)(sk Axk)yk+A(sk sk1)

27、k2264r|A In|2rksk 1n skk2r+42h|A|2rksk sk1k22+22kykk226(16r+90nkck22L2r2)ksk 1n skk2r+50n2kck22L22k sk xk22+2(2h+45nkck22L22)2ksk sk1k22+20kck222vk+10c2kyk Bykk2c.(37)31?|?|A In|r6 2,3?y.n10.?b?2,k 0,e?Evk+1 64L2mrEksk 1n skk2r+2nL2mEk sk xk22+4L2m2Eksk sk1k22+(1 1m)Evk.(38)y:d vk?Evik+1|Fk=1mimiXj=

28、1Ekfi,j(i,jk+1)fi,j(x)k22|Fk=1mimiXj=1(1mkfi,j(xik)fi,j(x)k22+(1 1m)kfi,j(i,jk)fi,j(x)k22)64L2mrksik skk2r+2L2mk sk xk22+4L2m2ksik sik1k22+(1 1m)vik.(39)3?Evk+1 64L2mrEksk 1n skk2r+2nL2mEk sk xk22+4L2m2Eksk sk1k22+(1 1m)Evk.(40)y?.DOI:10.12677/aam.2024.132081858A?Dn11.eb?1-3.XJ0 6 min14L10nkck22(h+2

29、)+2n2(Trc)2,(12B)216L(15(h+2)+nkrk22)g 0 6 0,e?Ekyk+1 Byk+1k2c61946L2rc(1 2B)Eksk 1n skk2r+338nL2c(1 2B)Ek sk xk22+802hL2c(1 2B)Eksk sk1k22+44c(1 2B)Evk+3+24Ekyk Bykk2c.(41)g=c/c.y:d yk?Ekyk+1 Byk+1k2c|Fk=EkByk B+(In B)gk+1 gkk2c|Fk61+2B2Ekyk Bykk2c|Fk+21 2BEkgk+1 gkk2c|Fk.(42)3?|Young?.?e5,k(42)?1?

31、859A?D3?|e?Ekxk+1 xkk22 6 2(1+)2Eksk+1 skk22+22Eksk sk1k22.(44)e?(43)?,Ekgk f(xk)k22|Fk?L?Ekgk+1 f(xk+1)k22|Fk68L2rEksk+1 1n sk+1k2r+4nL2k sk+1 xk22+82L2Eksk+1 skk22+2Evk+16(168L2r+720nkck22L4r(h+2)2+160n2L4(Trc)2r2)Eksk 1n skk2r+(36nL2+400n2kck22L42(h+2)Ek sk xk22+(72hL2+720nkck22L4(h+2)2+160n2L4(T

32、rc)22)Eksk sk1k2r+(160kck22L2(h+2)2+16n(Trc)2L22+2 2m)Evk+(80(h+2)+8nkrk22)cL22Ekyk Bykk2c|Fk.(45)e 0?y.un 7-11?(J,?Xe5?.51.?b?1-3.XJ 0 6 min14L10nkck22(h+2)+2n2(Trc)2,15nLTrc,(12B)216L(15(h+2)+nkrk22)g 0 6 1,Ke?k6 G,k1,(50)k R5 G,R55Xek=Eksk 1n skk2rEk sk xk22Enksk sk1k22EvkEkyk Bykk2cDOI:10.12677/

33、aam.2024.132081861A?DG,=1+2A2+9npp1p2p52+5np1p22(1+2A)r+9np1p2p52)2p1p2210rp225pp315p322.5n(Trc)22p416p+9npp1p525np1p52(4h+9np1p52)22p1210c24p5mr2p5m4p5m21 1m0973pp5p6169p5p6401hp5p622p53+243?p=1r,p1=10kck22,p2=r12A,p3=TrcL2,p4=5krk22cTrc,p5=L2,p6=2c(12B).?e55 1(J,?(G,)1?X?.n12.ABN-LSVRG,?v0 6 mins(

34、1 2A)12(9npp1p2p51+5np1p22+2p1p24+10r5),22(5pp31+p45),s522n(Trc)24,s3 16p19npp1p51+5np1p52+2p14+10c5,14Lp10nkck22(h+2)+2n2(Trc)2,15nLTrc,(1 2B)216Lp(15(h+2)+nkrk22)g.(51)v0 6 min(1 2A)1 2(9npp1p2p51+5np1p22+2p1p24+10r5)22(1+(1+2A)r+9np1p22),s2(5pp31+p45)2.5n(Trc)2425p33,s3 16p1(9npp1p51+5np1p52+2p14

35、+10c5)2(4h+9np1p22)3,s4 4pp51 2p524p53,(1 2B)5 4(973pp5p61 169p5p62 22p54)401hp5p63,(52)Kk(G,)1.?ABN-LSVRG 5?)x.DOI:10.12677/aam.2024.132081862A?Dy:n 5,?X?.?=1,2,3,4,5TG,.?d?(1+(1+2A)r+9np1p22)1 2A21(9npp1p2p51+5np1p22+2p1p24+10r5)2,5p332 2(5pp31+p45)2.5n(Trc)242,(4h+9np1p22)32 3 16p1(9npp1p51+5np1p

36、52+2p14+10c5)2,4p5m321m44pp5m12p5m2,401hp5p63?,dk s(1 2A)12(9npp1p2p51+5np1p22+2p1p24+10r5)22(5pp31+p45),s522n(Trc)24,0,2 0,3 16p1,4 4pp51+2p52,5 4(973pp5p61 169p5p62 22p54).(55)DOI:10.12677/aam.2024.132081863A?D?,(53)?X?.(1 2A)1 2(9npp1p2p51+5np1p22+2p1p24+10r5)22(1+(1+2A)r+9np1p22),s2(5pp31+p45)2.

37、5n(Trc)2425p33,s3 16p1(9npp1p51+5np1p52+2p14+10c5)2(4h+9np1p22)3,s4 4pp51 2p524p53,(1 2B)5 4(973pp5p61 169p5p62 22p54)401hp5p63.(56)y?.5.?!L?5?yJ?k?5.Xe68K:minxRpF(x)=1nnXi=11mmiXj=1log1+exp(CTijx)yij)+2kxk22,Figure 1.Directed network with 30 agents 1.30 UN?k?fi(x)=1mmiPj=1log1+exp(CTijx)yij)+2kxk22

38、Cij RpA?,yij+1,1 ADOI:10.12677/aam.2024.132081864A?D?I?Kz.?8 a9a w8a l UCI S(?e1.?d 30!:?k?,X 1.?1?-?A=aij B=bij?-),=aij=1Ninii bij=1Noutij.b?8?!?zUN.3k?,1nnPi=1kxikxk2L?.?(,ABN-LSVRG?K.l 2uy,?OX?.3 a9a w8a 8,?=0.9,ABN-LSVRG Ly.d,3e?=0.9.Figure 2.The impact of the momentum coefficient for ABN-LSVRG

39、2.X ABN-LSVRG?K?,?J?k?5,?ABN-LSVRG yk?1?,X SADDOPT,S-AB,Push-SAGA,Push-LSVRG,AB-SAGA.?,k?LN?.l 3w,?J?3?z?.N?J?ABN-LSVRG?uk.Figure 3.Comparison of ABN-LSVRG with the existing methods 3.ABN-LSVRGyk?6.(u?E Nesterov E|,?J?k?DOI:10.12677/aam.2024.132081865A?D?.L1?-?,;?dA?O?)?O&.?E?F?)?I?;m,?;?.Nesterov?.

40、ny?J?5?).u68K,(J?y?J?yk?Ly.z1 Pu,S.,Olshevsky,A.and Paschalidis,I.(2020)Asymptotic Network Independence in Dis-tributed Stochastic Optimization for Machine Learning:Examining Distributed and Central-ized Stochastic Gradient Descent.IEEE Signal Processing Magazine,37,114-122.https:/doi.org/10.1109/MS

41、P.2020.29752122 Nedic,A.(2020)Distributed Gradient Methods for Convex Machine Learning Problems inNetworks:Distributed Optimization.IEEE Signal Processing Magazine,37,92-101.https:/doi.org/10.1109/MSP.2020.29752103?9,m,r.z?UNJ.nA,2019,36(11):1820-1833.4 L u,Q.,Liao,X.,Li,H.,et al.(2020)Achieving Acc

42、eleration for Distributed Economic Dis-patch in Smart Grids over Directed Networks.IEEE Transactions on Network Science andEngineering,7,1988-1999.https:/doi.org/10.1109/TNSE.2020.29659995 Nedic,A.and Ozdaglar,A.(2009)Distributed Subgradient Methods for Multi-Agent Opti-mization.IEEE Transactions on

43、 Automatic Control,54,48-61.https:/doi.org/10.1109/TAC.2008.20095156 Nedic,A.,Olshevsky,A.and Shi,W.(2017)Achieving Geometric Convergence for DistributedOptimization over Time-Varying Graphs.SIAM Journal on Optimization,27,2597-2633.https:/doi.org/10.1137/16M10843167 Gao,J.,Liu,X.,Dai,Y.,et al.(2022

44、)Achieving Geometric Convergence for DistributedOptimization with Barzilai-Borwein Step Sizes.Science China Information Sciences,65,149-204.8 Ghaderyan,D.,Aybat,N.,Aguiar,A.,et al.(2023)A Fast Row-Stochastic DecentralizedMethod for Distributed Optimization over Directed Graphs.IEEE Transactions on A

45、utomaticControl,69,275-289.https:/doi.org/10.1109/TAC.2023.32759279 Lian,X.,Zhang,C.,Zhang,H.,et al.(2017)Can Decentralized Algorithms Outperform Cen-tralized Algorithms?A Case Study for Decentralized Parallel Stochastic Gradient Descent.Proceedings of the 31st International Conference on Neural Inf

46、ormation Processing Systems,Long Beach,CA,4-9 December 2017,5336-5346.10 Lee,S.and Zavlanos,M.M.(2017)Approximate Projection Methods for Decentralized Op-timization with Functional Constraints.IEEE Transactions on Automatic Control,63,3248-3260.https:/doi.org/10.1109/TAC.2017.277869611 Qu,G.and Li,N

47、.(2017)Harnessing Smoothness to Accelerate Distributed Optimization.IEEE Transactions on Control of Network Systems,5,1245-1260.https:/doi.org/10.1109/TCNS.2017.2698261DOI:10.12677/aam.2024.132081866A?D12 Di Lorenzo,P.and Scutari,G.(2016)Next:In-Network Nonconvex Optimization.IEEETransactions on Sig

48、nal and Information Processing over Networks,2,120-136.https:/doi.org/10.1109/TSIPN.2016.252458813 Pu,S.and Nedic,A.(2021)Distributed Stochastic Gradient Tracking Methods.MathematicalProgramming,187,409-457.https:/doi.org/10.1007/s10107-020-01487-014 Defazio,A.,Bach,F.and Lacoste-Julien,S.(2014)SAGA

49、:A Fast Incremental GradientMethod with Support for Non-Strongly Convex Composite Objectives.Proceedings of the 27thInternational Conference on Neural Information Processing Systems,1,1646-1654.15 Tan,C.,Ma,S.,Dai,Y.,et al.(2016)Barzilai-Borwein Step Size for Stochastic GradientDescent.Advances in N

50、eural Information Processing Systems 29:Annual Conference on NeuralInformation Processing Systems 2016,5-10 December 2016,Barcelona.16 Nguyen,L.,Liu,J.,Scheinberg,K.,et al.(2017)SARAH:A Novel Method for MachineLearning Problems Using Stochastic Recursive Gradient.International Conference on MachineL

展开阅读全文