dask_mpi.core.initialize(interface=None, nthreads=1, local_directory='', memory_limit='auto', nanny=False, dashboard=True, dashboard_address=':8787', protocol=None, worker_class='distributed.Worker', worker_options=None, comm=None, exit=True)[source]

Initialize a Dask cluster using mpi4py

Using mpi4py, MPI rank 0 launches the Scheduler, MPI rank 1 passes through to the client script, and all other MPI ranks launch workers. All MPI ranks other than MPI rank 1 block while their event loops run.

In normal operation these ranks exit once rank 1 ends. If exit=False is set they instead return an bool indicating whether they are the client and should execute more client code, or a worker/scheduler who should not. In this case the user is responsible for the client calling send_close_signal when work is complete, and checking the returned value to choose further actions.


Network interface like ‘eth0’ or ‘ib0’


Number of threads per worker


Directory to place worker files

memory_limitint, float, or ‘auto’

Number of bytes before spilling data to disk. This can be an integer (nbytes), float (fraction of total memory), or ‘auto’.


Start workers in nanny process for management (deprecated, use worker_class instead)


Enable Bokeh visual diagnostics


Bokeh port for visual diagnostics


Protocol like ‘inproc’ or ‘tcp’


Class to use when creating workers


Options to pass to workers

comm: mpi4py.MPI.Intracomm

Optional MPI communicator to use instead of COMM_WORLD

exit: bool

Whether to call sys.exit on the workers and schedulers when the event loop completes.

is_client: bool

Only returned if exit=False. Inidcates whether this rank should continue to run client code (True), or if it acts as a scheduler or worker (False).