stems.executor module

Dask/Distributed related helpers

stems.executor.executor_info(client, ip=True, bokeh=True, stats=True)[source]

Return a list of strings with info for a scheduler

Parameters

  • client (distributed.client.Client) – Scheduler (e.g., from client.scheduler)

  • ip (bool, optional) – Include scheduler IP

  • bokeh (bool, optional) – Include Bokeh visualization IP

  • stats (bool, optional) – Include stats on cluster (ncores, memory, etc)

Returns

Cluster information items

Return type

list[str]

stems.executor.setup_backend(scheduler, workers=None)[source]

Setup Dask to use threads or processes

stems.executor.setup_executor(address=None, n_workers=None, threads_per_worker=1, **kwds)[source]

Setup a Dask distributed cluster scheduler client

Parameters

  • address (str, optional) – This can be the address of a Scheduler server, like a string '127.0.0.1:8786'. If None, sets up a LocalCluster

  • n_workers (int, optional) – Number of workers. Only used if setting up a LocalCluster

  • threads_per_worker (int, optional) – Number of threads per worker

  • kwds – Additional options passed to distributed.Client()

Returns

Distributed compute client

Return type

distributed.Client