Parameter Server
Last updated
Last updated
The parameter server was proposed by for scalable machine learning.
Characters: client-server architecture, message-passing communication, and asynchronous.
(Note that MapReduce is bulk synchronous.)
Ray , an open-source software system, supports parameter server.
In practice, asynchronous algorithms are faster than the synchronous.
In theory, asynchronous algorithms has slower convergence rate.
Asynchronous algorithms have restrictions, e.g., a worker cannot be much slower than the others.
