Local SGD: Unified Theory and New Efficient Methods

This work was supported by the KAUST baseline research grant of P. Richt´arik. Part of this work was done while E. Gorbunov was a research intern at KAUST. The research of E. Gorbunov was also partially supported by the Ministry of Science and Higher Education of the Russian Federation (Goszadaniye) 075-00337-20-03 and RFBR, project number 19-31-51001.