- dask_mpi.core.initialize(interface=None, nthreads=1, local_directory='', memory_limit='auto', nanny=False, dashboard=True, dashboard_address=':8787', protocol=None, worker_class='distributed.Worker', worker_options=None, comm=None, exit=True)¶
Initialize a Dask cluster using mpi4py
Using mpi4py, MPI rank 0 launches the Scheduler, MPI rank 1 passes through to the client script, and all other MPI ranks launch workers. All MPI ranks other than MPI rank 1 block while their event loops run.
In normal operation these ranks exit once rank 1 ends. If exit=False is set they instead return an bool indicating whether they are the client and should execute more client code, or a worker/scheduler who should not. In this case the user is responsible for the client calling send_close_signal when work is complete, and checking the returned value to choose further actions.
Network interface like ‘eth0’ or ‘ib0’
Number of threads per worker
Directory to place worker files
- memory_limitint, float, or ‘auto’
Number of bytes before spilling data to disk. This can be an integer (nbytes), float (fraction of total memory), or ‘auto’.
Start workers in nanny process for management (deprecated, use worker_class instead)
Enable Bokeh visual diagnostics
Bokeh port for visual diagnostics
Protocol like ‘inproc’ or ‘tcp’
Class to use when creating workers
Options to pass to workers
- comm: mpi4py.MPI.Intracomm
Optional MPI communicator to use instead of COMM_WORLD
- exit: bool
Whether to call sys.exit on the workers and schedulers when the event loop completes.
- is_client: bool
Only returned if exit=False. Inidcates whether this rank should continue to run client code (True), or if it acts as a scheduler or worker (False).