Concurrency, Parallelism and Distributed Computing

Concurrency refers to running multiple computations more-or-less simultaneously, whereas parallelism refers to using multiple cores or OS-level threads to coordinate computation. We now know that the former is relatively safe and easy to reason about, whereas the latter is extremely difficult and causes many subtle bugs. OCaml currently supports concurrency elegantly, but parallelism support is not built in to the runtime.

Concurrency

• lwt: a monadic concurrency library. Concurrent code uses monads to express the higher-level abstractions of control flow.
• Async: another monadic concurrency library developed by Jane Street. This library is covered in Real World OCaml. While the concept is very similar to lwt, small discrepancies make compatibility between the libraries difficult.
• RWO-lwt: Real World OCaml code examples translated from Async to lwt.

Articles

• The blog post that introduced Async
• A user gives up on Async
• Cooperative Concurrency in OCaml: A Core.Std.Async Example.

Parallelism

As mentioned above, OCaml doesn't natively support multiple OS-level OCaml threads running simultaneously. A global lock prevents multiple OCaml threads from running simultaneously.

• The most hopeful and powerful way to use multicore is with the new OCaml multicore branch. This branch uses a parallel garbage collector, which means that OCaml will eventually be able to run on multiple cores in the same process. Note that this branch is not yet ready for real work, but it's rapidly advancing.
• By interfacing with external C code through the FFI, OCaml can pass off long-running computations to C threads running at the same time as the OCaml code. This is made easier nowadays due to CTypes (see Other Languages
• Parmap provides easy-to-use parallel map and fold functions. The library makes use of forking to create short-lived child processes, and memory mapping to feed the data back to the parent process.
• ForkWork is a simple library for forking child processes to perform work on multiple cores.
• Functory is a distributed computing library which facilitates distributed execution of parallelizable computations in a seamless fashion.
• Ocamlnet is an enhanced system platform library. It contains the /netmulticore/ library to compute tasks on as many cores of the machine as needed. This is probably the most advanced implementation currently available, as it is capable of creating a shared memory region, and running a custom-made garbage collector on said region, thus solving the problem of sharing memory with tracing garbage collectors.
• Nproc is a process pool implementation for OCaml.
• Parany is another parallel computation library.
• Sklml is a functional parallel skeleton compiler and programming system for OCaml programs.

Articles

• What is the state of OCaml's parallelization abilities?: Note that this article is quite a bit out-of-date.

Distributed Computing

• Rpc.Parallel is a library for spawning processes on a cluster of machines, and passing typed messages between them.
• MPI – Message Passing Interface bindings for OCaml.
• ocaml-rpc – Light library to deal with RPCs in OCaml.