Jorge Tavares

"It is sometimes an appropriate response to reality to go insane." — Philip K. Dick

Simple Parallel Array Filtering in F#

with 4 comments

Recently I was doing some data processing and since it fit nicely in a parallel setup, I changed my pipeline to use the Array.Parallel module. An easy and simple change that can speed up your task especially if it’s time consuming.

Nonetheless, I also needed Array.filter and to my surprise it wasn’t available in the Parallel module. After some searching, I realized that perhaps the common way to do what I wanted was to use the PSet module (see for example here).

However, I found this nice little example in F# Snippets. In short, a very simple extension of the Parallel module is done to include the filter operation and then it can be used like all others. The relevant code is summarized below:

To me this shows the elegance of F# in two simple aspects: 1) we are able to extended quite easily a standard module; 2) The use of Options. Initially, one could think that special care would be necessary to handle the case where no results are returned from collect. Using Some and None not only makes the coder shorter but also easy to understand and rather elegant.

About these ads

Written by Jorge Tavares

April 24, 2014 at 8:28

Posted in Programming

Tagged with , ,

4 Responses

Subscribe to comments with RSS.

  1. […] Jorge Tavares posted “Simple Parallel Array Filtering in F#“. […]

  2. […] Simple Parallel Array Filtering in F# (Slowing making our way toward complex parallel universe filtering) […]

  3. You also could be interested in F# Parallel Sequences http://fsprojects.github.io/FSharp.Collections.ParallelSeq/

    Sergey Tihon

    May 4, 2014 at 12:30


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: