Utilities for multi-processing

cosmic.mp.mp.multiprocess_with_queues(nproc, func, inputs, verbose=False, **progress_kw)

Map a function over a list of inputs using multiprocess

This essentially duplicates multiprocess.map but allows for arbitrary functions (that aren’t necessarily importable)

Parameters

nproc : int

number of processes to use, if 1 is given, the current process is used, and no child processes are forked

func : callable

the function to call in each iteration, should take a single argument that is the next element from inputs

inputs : iterable

iterable (e.g. list) of inputs, each element of which is passed to func in one of the child processes

verbose : bool, str, optional

if True, print progress to the console as a bar, pass a str to customise the heading for the progress bar, default: False, (default heading 'Processing:' if ``verbose=True`)

Returns

outputs : list

the list of results from calling func(x) for each element of inputs