News aggregator

Munich Haskell Meeting, 2015-03-30 < at > 19:30

haskell-cafe - Mon, 03/28/2016 - 10:55pm
Dear all, This week, our monthly Munich Haskell Meeting will take place again on Wednesday, March 30 at Cafe Puck at 19h30. For details see here: If you plan to join, please add yourself to this dudle so we can reserve enough seats! It is OK to add yourself to the dudle anonymously or pseudonymously. Everybody is welcome!
Categories: Offsite Discussion

Hspec + QuickCheck passing type

haskell-cafe - Mon, 03/28/2016 - 6:53pm
Hi, I'm writing the following Hspec + QuickCheck test for some type: context "Type1" $ it "decode inverses encode" $ property $ \x -> (decode . encode) x == Right (x::Type1) However, I have a number of these types, and I could write the same test for every type, but that isn't very DRY. Therefore I would like to put the test in a function and call that function for every type. Something like the following: typeTest name = do context name $ it "decode inverses encode" $ property $ \x -> (decode . encode) x == Right x Unfortunately this doesn't work, since QuickCheck doesn't know which Arbitrary instance to use. How can I pass the type, e.g. x::Type1, to this function so QuickCheck knows what Arbitrary instance to use? Kind regards, Martijn Rijkeboer
Categories: Offsite Discussion

[2nd CFP] Haskell Symposium 2016

General haskell list - Mon, 03/28/2016 - 6:44pm
======================================================================== ACM SIGPLAN CALL FOR SUBMISSIONS Haskell Symposium 2016 Nara, Japan, 22-23 September 2015, directly after ICFP ======================================================================== ** The Haskell Symposium has an early track this year ** ** See the Submission Timetable for details. ** The ACM SIGPLAN Haskell Symposium 2016 will be co-located with the International Conference on Functional Programming (ICFP 2016) in Vancouver, Canada. The Haskell Symposium aims to present original research on Haskell, discuss practical experience and future development of the language, and to promote other forms of denotative programming. Topics of interest include: * Language Design, with a focus on possible extensions and modifications of Haskell as well as critical discussions of the
Categories: Incoming News

[2nd CFP] Haskell Symposium 2016

haskell-cafe - Mon, 03/28/2016 - 6:44pm
======================================================================== ACM SIGPLAN CALL FOR SUBMISSIONS Haskell Symposium 2016 Nara, Japan, 22-23 September 2015, directly after ICFP ======================================================================== ** The Haskell Symposium has an early track this year ** ** See the Submission Timetable for details. ** The ACM SIGPLAN Haskell Symposium 2016 will be co-located with the International Conference on Functional Programming (ICFP 2016) in Vancouver, Canada. The Haskell Symposium aims to present original research on Haskell, discuss practical experience and future development of the language, and to promote other forms of denotative programming. Topics of interest include: * Language Design, with a focus on possible extensions and modifications of Haskell as well as critical discussions of the
Categories: Offsite Discussion

Joey Hess: type safe multi-OS Propellor

Planet Haskell - Mon, 03/28/2016 - 5:29am

Propellor was recently ported to FreeBSD, by Evan Cofsky. This new feature led me down a two week long rabbit hole to make it type safe. In particular, Propellor needed to be taught that some properties work on Debian, others on FreeBSD, and others on both.

The user shouldn't need to worry about making a mistake like this; the type checker should tell them they're asking for something that can't fly.

-- Is this a Debian or a FreeBSD host? I can't remember, let's use both package managers! host "" $ props & aptUpgraded & pkgUpgraded

As of propellor 3.0.0 (in git now; to be released soon), the type checker will catch such mistakes.

Also, it's really easy to combine two OS-specific properties into a property that supports both OS's:

upgraded = aptUpgraded `pickOS` pkgUpgraded type level lists and functions

The magick making this work is type-level lists. A property has a metatypes list as part of its type. (So called because it's additional types describing the type, and I couldn't find a better name.) This list can contain one or more OS's targeted by the property:

aptUpgraded :: Property (MetaTypes '[ 'Targeting 'OSDebian, 'Targeting 'OSBuntish ]) pkgUpgraded :: Property (MetaTypes '[ 'Targeting 'OSFreeBSD ])

In Haskell type-level lists and other DataKinds are indicated by the ' if you have not seen that before. There are some convenience aliases and type operators, which let the same types be expressed more cleanly:

aptUpgraded :: Property (Debian + Buntish) pkgUpgraded :: Property FreeBSD

Whenever two properties are combined, their metatypes are combined using a type-level function. Combining aptUpgraded and pkgUpgraded will yield a metatypes that targets no OS's, since they have none in common. So will fail to type check.

My implementation of the metatypes lists is hundreds of lines of code, consisting entirely of types and type families. It includes a basic implementation of singletons, and is portable back to ghc 7.6 to support Debian stable. While it takes some contortions to support such an old version of ghc, it's pretty awesome that the ghc in Debian stable supports this stuff.

extending beyond targeted OS's

Before this change, Propellor's Property type had already been slightly refined, tagging them with HasInfo or NoInfo, as described in making propellor safer with GADTs and type families. I needed to keep that HasInfo in the type of properties.

But, it seemed unnecessary verbose to have types like Property NoInfo Debian. Especially if I want to add even more information to Property types later. Property NoInfo Debian NoPortsOpen would be a real mouthful to need to write for every property.

Luckily I now have this handy type-level list. So, I can shove more types into it, so Property (HasInfo + Debian) is used where necessary, and Property Debian can be used everywhere else.

Since I can add more types to the type-level list, without affecting other properties, I expect to be able to implement type-level port conflict detection next. Should be fairly easy to do without changing the API except for properties that use ports.


As shown here, pickOS makes a property that decides which of two properties to use based on the host's OS.

aptUpgraded :: Property DebianLike aptUpgraded = property "apt upgraded" (apt "upgrade" `requires` apt "update") pkgUpgraded :: Property FreeBSD pkgUpgraded = property "pkg upgraded" (pkg "upgrade") upgraded :: Property UnixLike upgraded = (aptUpgraded `pickOS` pkgUpgraded) `describe` "OS upgraded"

Any number of OS's can be chained this way, to build a property that is super-portable out of simple little non-portable properties. This is a sweet combinator!

Singletons are types that are inhabited by a single value. This lets the value be inferred from the type, which came in handy in building the pickOS property combinator.

Its implementation needs to be able to look at each of the properties at runtime, to compare the OS's they target with the actial OS of the host. That's done by stashing a target list value inside a property. The target list value is inferred from the type of the property, thanks to singletons, and so does not need to be passed in to property. That saves keyboard time and avoids mistakes.

is it worth it?

It's important to consider whether more complicated types are a net benefit. Of course, opinions vary widely on that question in general! But let's consider it in light of my main goals for Propellor:

  1. Help save the user from pushing a broken configuration to their machines at a time when they're down in the trenches dealing with some urgent problem at 3 am.
  2. Advance the state of the art in configuration management by taking advantage of the state of the art in strongly typed haskell.

This change definitely meets both criteria. But there is a tradeoff; it got a little bit harder to write new propellor properties. Not only do new properties need to have their type set to target appropriate systems, but the more polymorphic code is, the more likely the type checker can't figure out all the types without some help.

A simple example of this problem is as follows.

foo :: Property UnixLike foo = p `requires` bar where p = property "foo" $ do ...

The type checker will complain that "The type variable ‘metatypes1’ is ambiguous". Problem is that it can't infer the type of p because many different types could be combined with the bar property and all would yield a Property UnixLike. The solution is simply to add a type signature like p :: Property UnixLike

Since this only affects creating new properties, and not combining existing properties (which have known types), it seems like a reasonable tradeoff.

things to improve later

There are a few warts that I'm willing to live with for now...

Currently, Property (HasInfo + Debian) is different than Property (Debian + HasInfo), but they should really be considered to be the same type. That is, I need type-level sets, not lists. While there's a type level sets library for hackage, it still seems to require a specific order of the set items when writing down a type signature.

Also, using ensureProperty, which runs one property inside the action of another property, got complicated by the need to pass it a type witness.

foo = Property Debian foo = property' $ \witness -> do ensureProperty witness (aptInstall "foo")

That witness is used to type check that the inner property targets every OS that the outer property targets. I think it might be possible to store the witness in the monad, and have ensureProperty read it, but it might complicate the type of the monad too much, since it would have to be parameterized on the type of the witness.

Oh no, I mentioned monads. While type level lists and type functions and generally bending the type checker to my will is all well and good, I know most readers stop reading at "monad". So, I'll stop writing. ;)


Thanks to David Miani who answered my first tentative question with a big hunk of example code that got me on the right track.

Also to many other people who answered increasingly esoteric Haskell type system questions.

Also thanks to the Shuttleworth foundation, which funded this work by way of a Flash Grant.

Categories: Offsite Blogs

Kevin Reid (kpreid): Representing the radio spectrum as spatial audio

Planet Haskell - Sun, 03/27/2016 - 8:17pm

In software-defined radio, there are well-established ways of visually representing the signal(s) in the entire bandwidth available from the hardware; we create a plot where the horizontal axis is frequency (using the Fourier transform to obtain the data). Then either the vertical axis is amplitude (creating an ordinary graph, sometimes called panorama) or the vertical axis is time and color is amplitude (creating a waterfall plot).

Here is an example of ShinySDR's spectrum display which includes both types (y=amplitude above and y=time below):

A further refinement is to display in the graph not just the most recent data but average or overlay many. In the above image, the blue fill color in the upper section is an overlay (both color and height correspond to amplitude), the green line is the average, and the red line is the peak amplitude over the same time interval.

We can see signals across an immensely wide spectrum (subject to hardware limitations), but is there a way to hear them meaningfully? Yes, there is, with caveats.

What's pictured above is a small portion of the band assigned to aviation use — they are used primarily for communication between aircraft in flight and air traffic control ground stations. The most significant thing about these communications is that there are a lot of different frequencies for different purposes, so if you're trying to hear “what's in the area”, you have to monitor all of them.

The conventional solution to this problem is a scanner, which is a radio receiver programmed to rapidly step through a range of frequencies and stop if a signal is detected. Scanners have disadvantages: they will miss the beginning of a signal, and they require a threshold set to trade off between missing weak signals and false-triggering on noise.

An alternative, specific to AM modulation (which is used by aircraft), is to make a receiver with very poor selectivity: the ability to receive only a specific channel and ignore other signals. (Historically, when RF electronic design was less well understood and components had worse characteristics, selectivity was a specification one would care about, but only if one lived in an area with closely-spaced radio stations — today, every receiver has good selectivity.)

I'm going to explain how to build an unselective receiver in software, and then refine this to create spatial audio — that is, the frequency of the signal shall correspond to the stereo panning of the output audio. This is the analogue of the spectrum display in audio.

Of course, this is an AM receiver and so it will only make intelligible sound for amplitude-modulated signals. However, many signals will produce some sound in an AM receiver. The exception is that a clean frequency-modulated (FM) or phase-modulated signal will produce silence, because its amplitude is theoretically constant, but this silence is still audibly distinct from background noise (if the signal is intermittent), and transmitted signals often do not have perfect constant amplitude.


A normal software AM demodulator has a structure like the following block diagram (some irrelevant details omitted). The RF signal is low-pass filtered to select the desired signal, then demodulated by taking the magnitude (which produces an audio signal with a DC offset corresponding to the carrier).

In order to produce an unselective receiver, we omit the RF filter step, and therefore also the downsampling — therefore demodulating at the RF sample rate. The resulting real signal must be low-pass filtered and downsampled to produce a usable audio sample rate (and because the high-frequency content is not interesting; see below), so we have now “just” swapped the two main components of the receiver.

This simple change works quite well. Two or more simultaneous AM signals can be received with clear stereo separation.

One interesting outcome is that, unlike the normal AM receiver, the audio noise when there is no signal is quieter (assuming AGC is present before the demodulator block in both cases) — this conveniently means that no squelch function is needed.

The reason for this is obvious-in-hindsight: loosely speaking, most of the noise power will be at RF frequencies and outside of the audio passband. In order to have a strong output signal, the input signal must contain a significant amount of power in a narrow band to serve as the AM carrier and sideband. (I haven't put any math to this theory, so it could be nonsense.)

Adding stereo

In order to produce the spatial audio, we want the audio signal amplitude, in a single stereo channel, to vary with frequency. And that is simply a filter with a sawtooth frequency response. The signal path is split for the two stereo channels, with opposite-slope filters. (AGC must be applied before the split.)

An undesired effect is that near the band edges, since the filter has a steep but not perfectly sharp transition from full-left to full-right, there is a lot of slope detection (output from frequency-modulated signals) that does not occur anywhere else. Of course,

This design can of course be applied to more than two audio channels; using surround sound would avoid the need for steepness of the filter at the edges and map the inherently circular digitized spectrum to a circular space, so it's worth trying.


I've implemented this in ShinySDR (and it is perhaps the first novel DSP feature I've put in). Just click the “AM unselective” mode button.

Some “directions for future research”:

As I mentioned above, this is useless for listening to FM signals. Is some technique which can do the same for FM? Naïvely creating an “unselective FM receiver” seems like it would be a recipe for horrible noise, because to a FM demodulator, noise looks like a very loud signal (because the apparent frequency is jumping randomly within the band, and frequency maps to amplitude of the output).

If we declare that the output need not be intelligible at all, is there a way to make a receiver that will respond to localized signal power independent of modulation? Can we make an unmodulated carrier act like an AM signal? (CW receivers do this using the BFO but that is dependent on input frequency.)

Categories: Offsite Blogs

Christopher Allen: Speeding up the automated building and testing of our Haskell projects

Planet Haskell - Sun, 03/27/2016 - 6:00pm

Alternate title: Arrest me for crimes against GNU Make.

I work for a company that uses Haskell for its primary application. We’re quite fond of having automatic builds fired off for each push to a branch and in our Github pull requests, so we were using CircleCI for our builds. Our circle.yml looked a bit like this initially:

machine: ruby: version: 2.1.7 services: - postgresql node: version: 5.1.0 dependencies: pre: - sudo apt-key adv --keyserver --recv-keys 575159689BEFB442 - echo 'deb trusty main' | sudo tee /etc/apt/sources.list.d/fpco.list - sudo apt-get update && sudo apt-get install stack -y - cd frontend && npm install - cd frontend && ./node_modules/.bin/bower install - npm install -g purescript - gem install bundler --pre - cd sql && bundle install override: - stack setup - cd app && make frontend: timeout: 3000 environment: UV_THREADPOOL_SIZE: 2 - make dirty: pwd: app test: override: - make sql - make -C sql migrate - stack test --jobs=1

However, I’m also a bit of a speed demon and impatient, so I was getting tired of the slow builds we had with CircleCI. 25-30 minutes for each build is just way too long when you’re waiting for the green checkmark to review code or merge it. My first pass was attempting to figure out what could be parallelized and what could be cached. Here’s what we added under dependencies for caching:

cache_directories: - "~/.stack" - ".stack-work" - "frontend/node_modules" - "frontend/bower_components" - "frontend/output"

But it wasn’t good enough.

Even with caching, our CircleCI builds were taking about 20 minutes.

We got a dedicated server and put on it instead of using CircleCI. We couldn’t have afforded the enterprise version of CircleCI and using something we can modify ourselves had a lot of appeal. Drone uses Docker to manage the build environment and after getting our build working and tests passing inside of the Docker containers, I was able to get Drone tracking our Github stuff pretty quickly. This got us down to about 6 minutes in order to do the following:

  1. Build frontend assets
  2. Build two different Haskell projects and run their respective tests

Here’s approximately what the .drone.yml looked like:

build: image: app environment: - POSTGRES_PORT_5432_TCP_ADDR=localhost commands: - make tests notify: slack: webhook_url: OUR_WEBHOOK_URL channel: dev username: drone template: > build #{{ build.number }} finished with a {{ build.status }} status. Commit message: {{build.message}} - See more at {{system.link_url}}/{{repo.owner}}/{{}}/{{build.number}} compose: db: image: postgres environment: - POSTGRES_USER=postgres - POSTGRES_PASSWORD=password app: image: app environment: - POSTGRES_PORT_5432_TCP_ADDR=localhost depends_on: - db

Note that we used Drone’s baked in Docker Compose’ish functionality so that we didn’t have to also configure a PostgreSQL server in the same container. make tests ends up running:

make -C sql/ test make -C lib/ test make -C app/ test

These tasks stood up and migrated the database, built our business logic library and ran the tests, and build our web app and ran the tests.

I wasn’t satisfied though, so I realized that since Drone uses Docker containers and our Haskell projects use Stackage LTS for our package versions, all we needed to do was specify our current LTS resolver and build a bunch of dependencies we knew we’d need. Here’s what I added to the Docker build container’s Dockerfile in order to pre-build the dependencies:

# Setup Stack RUN stack setup --resolver lts-5.8 # ADD global-stack.yaml ~/.stack/global-project/stack.yaml # RUN stack setup --resolver lts-5.8 # Cache some deps RUN stack --resolver lts-5.8 build lens-aeson yesod yesod-test esqueleto http-client free classy-prelude-yesod classy-prelude-conduit case-insensitive gravatar wreq xml-conduit warp hspec QuickCheck wai-logger persistent-postgresql HUnit uuid-aeson monad-logger mandrill email-validate yesod-auth yesod-newsfeed yesod-form haskell-src-exts cpphs polyparse xml-hamlet th-orphans either base-compat th-expand-syns th-lift MonadRandom

The fruit of my effort was:

Screenshot of a recent build & test run taking only 2 minutes or so. I elided the branch name.

In so doing, our build times dropped from 20 or 25 minutes with CircleCI, down to 2-3 minutes on our private Drone server.

Here’s the steps we took:

  1. Added caching to our CircleCI build. This got us: 25 minutes -> 20 minutes.

  2. Switched from CircleCI on their shared servers to Drone on a $100/month dedicated server. This got us: 20 minutes -> 6 minutes.

  3. Started pre-building dependencies. This got us: 6 minutes -> 2 or 3 minutes.

Note that these 2 or 3 minutes isn’t just building a Haskell project, that takes a couple seconds. About half the time is spent building frontend assets! Our test suite is relatively fast because we cache things like the Yesod application initialization. I believe we could get this down to a minute or two, but I don’t want to muck about with our PureScript build chain. Even with two Haskell developers, we think the $100/month and time to get it working paid off quickly.

I would estimate Drone took slightly longer than CircleCI originally did to get working initially, but I think the small difference is worth it and that it would’ve been faster for someone that dislikes Docker less.

I know this site is a bit of a disaster zone, but if you like my writing or think you could learn something useful from me, please take a look at the book I've been writing with my coauthor Julie. There's a free sample available too!

Posted on March 28, 2016

Categories: Offsite Blogs

HGamer3D has a new dependency installer

haskell-cafe - Sat, 03/26/2016 - 8:56pm
Dear All, it took a long while, but now a new solution has been found for dependency setup for HGamer3D. What do you think? Check here: . regards uotbw _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >
Categories: Offsite Discussion

build error using hsqml

haskell-cafe - Sat, 03/26/2016 - 7:15pm
Hello, I'm trying to build a program using HsQml. cabal 1.14.0 fails with this error: Building HsQmlTest-0.0.1... Preprocessing executable 'HsQmlTest' for HsQmlTest-0.0.1... [1 of 1] Compiling Main ( src/Main.hs, dist/build/HsQmlTest/HsQmlTest-tmp/Main.o ) Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Loading package array- ... linking ... done. Loading package deepseq- ... linking ... done. Loading package containers- ... linking ... done. Loading package filepath- ... linking ... done. Loading package tagged- ... linking ... done. Loading package bytestring- ... linking ... done. Loading package text- ... linking ... done. Loading package transformers- ... linking ... done. <command line>: can't load .so/.DLL for: /home/fabien/.cabal/lib/hsqml- ( cannot open shared object file: No
Categories: Offsite Discussion

[ANN] Csound-expression-5 is out (framework forcomputer music)

haskell-cafe - Sat, 03/26/2016 - 6:47pm
I'd like to announce a new version of Csound-expression library. It's a framework for computer music production. It compiles Haskell code to Csound. What's new? **The 5.0 is out! New features:** csound-expression * **Microtonal tunings**. We can use custom temperaments with insturments, patches, soundfonts and MIDI-instruments. Check out the guide on tuning and microtonal music (see also module `Csound.Tuning`). There are many predefined tunings (including ancient ones). Now we can play the authentic Bach music with Haskell! See [Custom temperament. Microtonal music]( ) for details. * **Functions for Csound API**. We can interface with generated code through many other languages. We can generate the code with Haskell and the use it in other environments. we can build UI with Python or Clojure, we can create an Android synthesizer. See the guide section on Csound API. See [Csound API. Using ge
Categories: Offsite Discussion

Sponsored: The 3 Week Diet - Fri, 03/25/2016 - 2:00am
8 Rules of Fat Loss. Warning: Fast Results! Click Here to Watch Video...
Categories: Offsite Blogs

Docs pending for a long time, on Hackage

haskell-cafe - Fri, 03/25/2016 - 1:31am
L.S., There is no documentation for the most recent version of package wx[0], even though it is on Hackage since December last year. It's Hackage page says: StatusDocs pending Build status unknown [no reports yet] Why is this and how can I get the build started? Regards, Henk-Jan van Tuyl [0]
Categories: Offsite Discussion

Functional Jobs: Haskell Engineer at Somatix Systems (Full-time)

Planet Haskell - Thu, 03/24/2016 - 2:48pm

Somatix Systems ( is re-imagining the way customers engage with the brands they love. We've got the DNA of a stealth startup and the deep resources of an established company. Our flagship product creates new interactions between customers and brands, and we're already reaching millions of customers with one of the world's most recognizable companies. We're building a team to sling the latest advancements in IoT, cloud infrastructure, mobility, big data analytics, and machine learning / AI to make this magic happen.

We work in small, self-sufficient, teams with the shared goal of delivering excellent software, anchored in an agile culture of quality, delivery, and innovation. We encourage our engineers to share responsibility for writing tested, high performance, maintainable code. We deliver working software in short sprints via iterative and incremental development practices. Individuals on our self-organizing teams are motivated, trusted and continuously focused on excellence. We share a common love for functional programming and using data to solve complex problems and create value.

About You
  • You've developed an appreciation for functional programming and revel at the opportunity to build production systems in Haskell
  • You welcome the responsibility and thrill that comes with being a member of a founding team
  • You court perfection, but you're grounded and practical
  • You understand that legacy code is the work you did yesterday
Desired Skills and Experience
  • Significant experience using Haskell, Scala, Erlang, Clojure (or other functional languages) to build production systems, complex personal projects, or make major OSS contributions
  • Real-world experience designing, developing, testing, and deploying systems based on SOA or micro-services
  • Skilled at designing and implementing SQL and NOSQL data persistence stores and caches
  • Experience building REST API services using functional languages and design principles
  • Ability to build infrastructure for real-time analytics and real-time predictive intelligence based on large, diverse, and dynamic data sets
  • Hardware hacking and pro-typing experience is welcome (e.g. Arduino, Raspberry Pi) !
How to Apply
  • Let us know why you think that you're the kind of person we're looking for
  • We value demonstrated expertise over a resume so show us what you've built

Get information on how to apply for this position.

Categories: Offsite Blogs

Usability of Programming Languages SIG at CHI'2016

Lambda the Ultimate - Thu, 03/24/2016 - 2:21pm

A special-interest group meeting during the ACM CHI 2016 conference in San Jose, CA on the topic of the usability of programming languages. People are invited to attend!

To attend you must be registered for the CHI'2016 conference, and early registration ends March 14:

For more information about the SIG, see:


Programming languages form the interface between programmers (the users) and the computation that they desire the computer to
execute. Although studies exist for some aspects of programming language design (such as conditionals), other aspects have received little or no human factors evaluations. Designers thus have little they can rely on if they want to make new languages highly usable, and users cannot easily chose a language based on usability criteria. This SIG will bring together researchers and practitioners interested in increasing the depth and breadth of studies on the usability of programming languages, and ultimately in improving the usability of future languages.

Categories: Offsite Discussion

[ANN] task-distribution: simple distribution of new code in a cluster computing setup

haskell-cafe - Thu, 03/24/2016 - 1:15pm
Hello cafe, I am happy to announce task-distribution, a framework written in order to distribute and execute tasks as fast as possible on a large data set. Its accentuation is that it supports distributing new logic to other nodes from within the program itself, thus avoiding both restarting slave node processes and a separated plugin-management infrastructure. An example use case is running continuously changing analytics logic on a given data set stored in HDFS. Avalailable on hackage and github: Cheers, Axel
Categories: Offsite Discussion

ARRAY 2016 2nd Call for Papers

General haskell list - Thu, 03/24/2016 - 9:44am
*************************************************************************** CALL FOR PAPERS ARRAY 2016 3rd ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming Santa Barbara, CA, USA June 14, 2016 Deadline: April 1, 2016 *************************************************************************** ARRAY 2016 is part of PLDI 2016 37th Annual ACM SIGPLAN Conference on Programming Language Design and Implementation June 13-17, 2016 *************************************************************************** About: Array-oriented programming is
Categories: Incoming News

Holden Karau: Early Release of High Performance Spark Now Available

Planet Haskell - Wed, 03/23/2016 - 6:20pm
Rachel and I are excited to announce the early release of the first four chapters of "High Performance Spark" just in time for Strata San Jose. Our first chunk includes:
  • Introduction to High Performance Spark
  • How Spark Works
  • DataFrames, Datasets & Spark SQL
  • Joins (SQL & Core)

You can buy it now from O'Reilly :)

Our planned future chapters are*:
  • Effective Transformations
  • Working with Key/Value data
  • Data Structure Design
  • Spark Tuning and Cluster Sizing
  • Getting Outside of the JVM
  • Debugging Techniques
  • Spark Components

We'd love your feedback to so we can make a super awesome finished book for you all. If you are going to Strata San Jose next week, I'll also be giving a talk on testing Spark & hosting office hours I'd love to see some of you there.


If your up in Seattle area Rachel and I are coming up for Data Day Seattle - hope to see some of you there too!

For our friends across the pond I'll also be speaking at Strata London and hopefully we will have an update with a few more chapters by then (but we might also need to take a quick break from writing to do our day jobs. Don't tell our editors :p).

*Subject to change depending on feedback and how much fun we have writing each chapter
Categories: Offsite Blogs

Warn about unwanted instances in a modular way

libraries list - Wed, 03/23/2016 - 2:16pm
I like to propose the following way to warn about instances that are unwanted by some programmers. First step is to mark the instances at their definition site like so: {-# WARN_INSTANCE tuple #-} instance Foldable ((,) a) where ... {-# WARN_INSTANCE tuple #-} instance Functor ((,) a) where ... {-# WARN_INSTANCE tuple #-} instance Foldable ((,,) a b) where ... {-# WARN_INSTANCE tuple #-} instance Functor ((,,) a b) where ... This way, all the above instances are collected in an instance group labelled 'tuple'. At the use sites we introduce a GHC warning option like -fwarn-instance=tuple. This warns about any place where any of the 'tuple' instances is used. We can either place GHC-Options: -fwarn-instance=tuple in a Cabal package description in order to issue warnings in a whole package or we can put {-# OPTIONS_GHC -fwarn-instance=tuple #-} at the top of a module in order to enable the warning per module. Another candidate for an instance group might be 'numeric' for numeric insta
Categories: Offsite Discussion

Brent Yorgey: Boltzmann sampling for generic Arbitrary instances

Planet Haskell - Wed, 03/23/2016 - 11:58am

tl;dr: I know how to generate random instances of data types in a generic way, and even have some old code that already does all the hard work, but won’t have time to polish and package it until this summer. If you’re interested in helping, let me know!

This morning Kenny Foner pointed out to me this tweet by Gabriel Gonzales, asking why there isn’t a default Arbitrary instance for types implementing Generic. It reminded me that I’ve been meaning for a while now (years, in fact!) to get around to packaging up some code that does this.

As several pointed out on Twitter, this seems obvious, but it isn’t. It’s easy to write a generic Arbitrary instance, but hard to write one that generates a good distribution of values. The basic idea is clear: randomly pick a constructor, and then recursively generate random subtrees. The problem is that this is very likely to either blow up and generate gigantic (even infinite) trees, or to generate almost all tiny trees, or both. I wrote a post about this three years ago which illustrates the problem. It also explains half of the solution: generate random trees with a target size in mind, and throw out any which are not within some epsilon of the target size (crucially, stopping the generation early as soon as the tree being generated gets too big).

However, I never got around to explaining the other half of the solution: it’s crucially important to use the right probabilities when picking a constructor. With the wrong probabilities, you will spend too much time generating trees that are either too small or too big. The surprising thing is that with exactly the right probabilities, you can expect to wait only time before generating a tree of size (approximately1) .2

So, how does one pick the right probabilities? Essentially, you turn the generic description of your data type into a mutually recursive system of generating functions, and (numerically) find their radii of convergence, when thought of as functions in the complex plane. Using these values it is straightforward to compute the right probabilities to use. For the intrepid, this is explained in Duchon et. al3.

I have some old Haskell code from Alexis Darrasse which already does a bunch of the work. It would have to be updated a bit to work with modern libraries and with GHC.Generics, and packaged up to go on Hackage. I won’t really have time to work on this until the summer—but if anyone else is interested in working on this, let me know! I’d be happy to send you the code and provide some guidance in figuring it out.

  1. The constant factor depends on how approximate you are willing to be.

  2. I wanted to put an exclamation point at the end of that sentence, because this is really surprising. But it looked like factorial. So, here is the exclamation point: !

  3. Duchon, Philippe, et al. “Boltzmann samplers for the random generation of combinatorial structures.” Combinatorics Probability and Computing 13.4-5 (2004): 577-625.

Categories: Offsite Blogs

Three Haskell dev roles at Standard Chartered

General haskell list - Wed, 03/23/2016 - 10:38am
Hi all I'm hiring another 3 devs to join Strats at Standard Chartered in London or Singapore. This is a good chance to join a growing, experienced team using Haskell for a wide set of projects in finance. More details in
Categories: Incoming News