Recently I was doing some data processing and since it fit nicely in a parallel setup, I changed my pipeline to use the Array.Parallel module. An easy and simple change that can speed up your task especially if it’s time consuming.
Nonetheless, I also needed Array.filter and to my surprise it wasn’t available in the Parallel module. After some searching, I realized that perhaps the common way to do what I wanted was to use the PSet module (see for example here).
However, I found this nice little example in F# Snippets. In short, a very simple extension of the Parallel module is done to include the filter operation and then it can be used like all others. The relevant code is summarized below:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|// Parallel filter for Arrays as seen in http://fssnip.net/hP|
|module Array =|
|module Parallel =|
|let filter predicate array =|
|array |> Array.Parallel.choose (fun x -> if predicate x then Some x else None)|
|// use example|
||> Array.Parallel.filter predicate|
To me this shows the elegance of F# in two simple aspects: 1) we are able to extended quite easily a standard module; 2) The use of Options. Initially, one could think that special care would be necessary to handle the case where no results are returned from collect. Using Some and None not only makes the coder shorter but also easy to understand and rather elegant.
4 thoughts on “Simple Parallel Array Filtering in F#”
[…] Jorge Tavares posted “Simple Parallel Array Filtering in F#“. […]
[…] Simple Parallel Array Filtering in F# (Slowing making our way toward complex parallel universe filtering) […]
You also could be interested in F# Parallel Sequences http://fsprojects.github.io/FSharp.Collections.ParallelSeq/
Thanks for the pointer Sergey! This looks very good.