6.21.10 Parallel forms

The functions described in this section are available from

(use-modules (ice-9 threads))

They provide high-level parallel constructs. The following functions are implemented in terms of futures (see section Futures). Thus they are relatively cheap as they re-use existing threads, and portable, since they automatically use one thread per available CPU core.

syntax: parallel expr …

Evaluate each expr expression in parallel, each in its own thread. Return the results of n expressions as a set of n multiple values (see section Returning and Accepting Multiple Values).

syntax: letpar ((var expr) …) body1 body2 …

Evaluate each expr in parallel, each in its own thread, then bind the results to the corresponding var variables, and then evaluate body1 body2 ...

letpar is like let (see section Local Variable Bindings), but all the expressions for the bindings are evaluated in parallel.

Scheme Procedure: par-map proc lst1 lst2 …

Scheme Procedure: par-for-each proc lst1 lst2 …

Call proc on the elements of the given lists. par-map returns a list comprising the return values from proc. par-for-each returns an unspecified value, but waits for all calls to complete.

The proc calls are (proc elem1 elem2 …), where each elem is from the corresponding lst . Each lst must be the same length. The calls are potentially made in parallel, depending on the number of CPU cores available.

These functions are like map and for-each (see section List Mapping), but make their proc calls in parallel.

Unlike those above, the functions described below take a number of threads as an argument. This makes them inherently non-portable since the specified number of threads may differ from the number of available CPU cores as returned by current-processor-count (see section Processes). In addition, these functions create the specified number of threads when they are called and terminate them upon completion, which makes them quite expensive.

Therefore, they should be avoided.

Scheme Procedure: n-par-map n proc lst1 lst2 …

Scheme Procedure: n-par-for-each n proc lst1 lst2 …

Call proc on the elements of the given lists, in the same way as par-map and par-for-each above, but use no more than n threads at any one time. The order in which calls are initiated within that threads limit is unspecified.

These functions are good for controlling resource consumption if proc calls might be costly, or if there are many to be made. On a dual-CPU system for instance n=4 might be enough to keep the CPUs utilized, and not consume too much memory.

Scheme Procedure: n-for-each-par-map n sproc pproc lst1 lst2 …

Apply pproc to the elements of the given lists, and apply sproc to each result returned by pproc. The final return value is unspecified, but all calls will have been completed before returning.

The calls made are (sproc (pproc elem1 … elemN)), where each elem is from the corresponding lst. Each lst must have the same number of elements.

The pproc calls are made in parallel, in separate threads. No more than n threads are used at any one time. The order in which pproc calls are initiated within that limit is unspecified.

The sproc calls are made serially, in list element order, one at a time. pproc calls on later elements may execute in parallel with the sproc calls. Exactly which thread makes each sproc call is unspecified.

This function is designed for individual calculations that can be done in parallel, but with results needing to be handled serially, for instance to write them to a file. The n limit on threads controls system resource usage when there are many calculations or when they might be costly.

It will be seen that n-for-each-par-map is like a combination of n-par-map and for-each,

(for-each sproc (n-par-map n pproc lst1 ... lstN))

But the actual implementation is more efficient since each sproc call, in turn, can be initiated once the relevant pproc call has completed, it doesn’t need to wait for all to finish.

This document was generated on April 20, 2013 using texi2html 5.0.