News aggregator

Incredibly slow type-level Nub

haskell-cafe - Sun, 02/07/2016 - 3:37pm
Dear Cafe, I have faced a type-level performance problem (with GHC 7.10.3). I am trying to implement type-level Nub which removes duplicates from a type-level list. My implementation [1] with an example of a list of length 20 takes forever to compile. If you reduce the size of the example list (e.g. to 15 items), it will compile faster. Also, if you remove the Nub application from exampleVals, it compiles instantly (this is how I know the problem is in Nub). My question is how can I speed up type-level Nub? And less importantly why exactly is it this slow? The practical application for this is building automatic tests [2] for servant-swagger. There I am interested in generating exactly one test for each distinct type. Kind regards, Nick [1] [2] _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >
Categories: Offsite Discussion

Mark Jason Dominus: Four Victorian novels

Planet Haskell - Sun, 02/07/2016 - 1:37pm

I've read a bunch of 19-century English novels lately. I'm not exactly sure why; it just sort of happened. But it's been a lot of fun. When I was small, my mother told me more than once that people often dislike these books because they are made to read them too young; the books were written for adult readers and don't make sense to children. I deliberately waited to read most of these, and I am very pleased now to find that now that I am approaching middle age I enjoy books that were written for people approaching middle age.

Spoilers abound.

Jane Eyre

This is one of my wife's favorite books, or perhaps her very favorite, but I had not read it before. Wow, it's great! Jane is as fully three-dimensional as anyone in fiction.

I had read The Eyre Affair, which unfortunately spoiled a lot of the plot for me, including the Big Shocker; I kept wondering how I would feel if I didn't know what was coming next. Fortunately I didn't remember all the details.

  • From her name, I had expected Blanche Ingram to be pale and limp; I was not expecting a dark, disdainful beauty.

  • When Jane tells Rochester she must leave, he promises to find her another position, and the one he claims to have found is hilariously unattractive: she will be the governess to the five daughters of Mrs. Dionysus O'Gall of Bitternutt Lodge, somewhere in the ass-end of Ireland.

  • What a thrill when Jane proclaims “I am an independent woman now”! But she has achieved this by luck; she inherited a fortune from her long-lost uncle. That was pretty much the only possible path, and it makes an interesting counterpoint to Vanity Fair, which treats some of the same concerns.

  • The thought of dutifully fulfilling the duty of a dutiful married person by having dutiful sex with the dutiful Mr. Rivers makes my skin crawl. I imagine that Jane felt the same way.

  • Mr. Brocklehurst does not get one-tenth of what he deserves.

Jane Eyre set me off on a Victorian novel kick. The preface of Jane Eyre praises William Thackeray and Vanity Fair in particular. So I thought I'd read some Thackeray and see how I liked that. Then for some reason I read Silas Marner instead of Vanity Fair. I'm not sure how that happened.

Silas Marner

Silas Marner was the big surprise of this batch of books. I don't know why I had always imagined Silas Marner would be the very dreariest and most tedious of all Victorian novels. But Silas Marner is quite short, and I found it very sweet and charming.

I do not suppose my Gentle Readers are as likely to be familiar with Silas Marner as with Jane Eyre. As a young man, Silas is a member of a rigid, inward-looking religious sect. His best friend frames him for a crime, and he is cast out. Feeling abandoned by society and by God, he settles in Raveloe and becomes a miser, almost a hermit. Many years pass, and his hoarded gold is stolen, leaving him bereft. But one snowy evening a two-year-old girl stumbles into his house and brings new purpose to his life. I have omitted the subplot here, but it's a good subplot.

One of the scenes I particularly enjoyed concerns Silas’ first (and apparently only) attempt to discipline his adopted two-year-old daughter Eppie, with whom he is utterly besotted. Silas knows that sooner or later he will have to, but he doesn't know how—striking her seems unthinkable—and consults his neighbors. One suggests that he shut her in the dark, dirty coal-hole by the fireplace. When Eppie wanders away one day, Silas tries to be stern.

“Eppie must go into the coal hole for being naughty. Daddy must put her in the coal hole.”

He half expected that this would be shock enough and that Eppie would begin to cry. But instead of that she began to shake herself on his knee as if the proposition opened a pleasing novelty.

As they say, no plan survives contact with the enemy.

Seeing that he must proceed to extremities, he put her into the coal hole, and held the door closed, with a trembling sense that he was using a strong measure. For a moment there was silence but then came a little cry “Opy, opy!” and Silas let her out again…

Silas gets her cleaned up and changes her clothes, and is about to settle back to his work

when she peeped out at him with black face and hands again and said “Eppie in de toal hole!”

Two-year-olds are like that: you would probably strangle them, if they weren't so hilariously cute.

Everyone in this book gets what they deserve, except the hapless Nancy Lammeter, who gets a raw deal. But it's a deal partly of her own making. As Thackeray says of Lady Crawley, in a somewhat similar circumstance, “a title and a coach and four are toys more precious than happiness in Vanity Fair”.

There is a chapter about a local rustics at the pub which may remind you that human intercourse could be plenty tiresome even before the invention of social media. The one guy who makes everything into an argument will be quite familiar to my Gentle Readers.

I have added Silas Marner to the long list of books that I am glad I was not forced to read when I was younger.

The Old Curiosity Shop

Unlike Silas Marner, I know why I read this one. In the park near my house is a statue of Charles Dickens and Little Nell, on which my daughter Toph is accustomed to climb. As she inevitably asked me who it was a statue of, I explained that Dickens was a famous writer, and Nell is a character in a book by Dickens. She then asked me what the book was about, and who Nell was, and I did not know. I said I would read the book and find out, so here we are.

My experience with Dickens is very mixed. Dickens was always my mother's number one example of a writer that people were forced to read when too young. My grandfather had read me A Christmas Carol when I was young, and I think I liked it, but probably a lot of it went over my head. When I was about twenty-two I decided to write a parody of it, which meant I had to read it first, but I found it much better than I expected, and too good to be worth parodying. I have reread it a couple of times since. it is very much worth going back to, and is much better than its many imitators.

I had been required to read Great Expectations in high school, had not cared for it, and had stopped after four or five chapters. But as an adult I kept a copy in my house for many years, waiting for the day when I might try again, and when I was thirty-five I did try again, and I loved it.

Everyone agrees that Great Expectations is one of Dickens’ best, and so it is not too surprising that I was much less impressed with Martin Chuzzlewit when I tried that a couple of years later. I remember liking Mark Tapley, but I fell off the bus shortly after Martin came to America, and I did not get back on.

A few years ago I tried reading The Pickwick Papers, which my mother said should only be read by middle-aged people, and I have not yet finished it. It is supposed to be funny, and I almost never find funny books funny, except when they are read aloud. (When I tell people this, they inevitably name their favorite funny books: “Oh, but you thought The Hitchhiker’s Guide to the Galaxy was funny, didn't you?” or whatever. Sorry, I did not. There are a few exceptions; the only one that comes to mind is Stanisław Lem's The Cyberiad, which splits my sides every time. SEVEN!

Anyway, I digress. The Old Curiosity Shop was extremely popular when it was new. You always hear the same two stories about it: that crowds assembled at the wharves in New York to get spoilers from the seamen who might have read the new installments already, and that Oscar Wilde once said “one must have a heart of stone to read the death of Little Nell without laughing.” So I was not expecting too much, and indeed The Old Curiosity Shop is a book with serious problems.

Chief among them: it was published in installments, and about a third of the way through writing it Dickens seems to have changed his mind about how he wanted it to go, but by then it was too late to go back and change it. There is Nell and her grandfather on the one hand, the protagonists, and the villain is the terrifying Daniel Quilp. It seems at first that Nell's brother Fred is going to be important, but he disappears and does not come back until the last page when we find out he has been dead for some time. It seems that Quilp's relations with his tyrannized wife are going to be important, but Quilp soon moves out of his house and leaves Mrs. Quilp more or less alone. It seems that Quilp is going to pursue the thirteen-year-old Nell sexually, but Nell and Grandpa flee in the night and Quilp never meets them again. They spend the rest of the book traveling from place to place not doing much, while Quilp plots against Nell's friend Kit Nubbles.

Dickens doesn't even bother to invent names for many of the characters. There is Nell’s unnamed grandfather; the old bachelor; the kind schoolmaster; the young student; the guy who talks to the fire in the factory in Birmingham; and the old single gentleman.

The high point of the book for me was the development of Dick Swiveller. When I first met Dick I judged him to be completely worthless; we later learn that Dick keeps a memorandum book with a list of streets he must not go into, lest he bump into one of his legion of creditors. But Dick turns out to have some surprises in him. Quilp's lawyer Sampson Brass is forced to take on Swiveller as a clerk, in furtherment of Quilp's scheme to get Swiveller married to Nell, another subplot that comes to nothing. While there, Swiveller, with nothing to amuse himself, teaches the Brasses’ tiny servant, a slave so starved and downtrodden that she has never been given a name, to play cribbage. She later runs away from the Brasses, and Dick names her Sophronia Sphynx, which he feels is “euphonious and genteel, and furthermore indicative of mystery.” He eventually marries her, “and they played many hundred thousand games of cribbage together.”

I'm not alone in finding Dick and Sophronia to be the most interesting part of The Old Curiosity Shop. The anonymous author of the excellent blog A Reasonable Quantity of Butter agrees with me, and so does G.K. Chesterton:

The real hero and heroine of The Old Curiosity Shop are of course Dick Swiveller and [Sophronia]. It is significant in a sense that these two sane, strong, living, and lovable human beings are the only two, or almost the only two, people in the story who do not run after Little Nell. They have something better to do than to go on that shadowy chase after that cheerless phantom.

Today is Dickens’ 204th birthday. Happy birthday, Charles!

Vanity Fair

I finally did get to Vanity Fair, which I am only a quarter of the way through. It seems that Vanity Fair is going to live or die on the strength of its protagonist Becky Sharp.

When I first met Ms. Sharp, I thought I would love her. She is independent, clever, and sharp-tongued. But she quickly turned out to be scheming, manipulative, and mercenary. She might be hateful if the people she was manipulating were not quite such a flock of nincompoops and poltroons. I do not love her, but I love watching her, and I partly hope that her schemes succeed, although I rather suspect that she will sabotage herself and undo all her own best plans.

Becky, like Jane Eyre, is a penniless orphan. She wants money, and in Victorian England there are only two ways for her to get it: She can marry it or inherit it. Unlike Jane, she does not have a long-lost wealthy uncle (at least, not so far) so she schemes to get it by marriage. It's not very creditable, but one can't feel too righteous about it; she is in the crappy situation of being a woman in Victorian England, and she is working hard to make the best of it. She is extremely cynical, but the disagreeable thing about a cynic is that they refuse to pretend that things are better than they are. I don't think she has done anything actually wrong, and so far her main path to success has been to act so helpful and agreeable that everyone loves her, so I worry that I may come out of this feeling that Thackeray does not give her a fair shake.

In the part of the book I am reading, she has just married the exceptionally stupid Rawdon Crawley. I chuckle to I think of the flattering lies she must tell him when they are in the sack. She has married him because he is the favorite relative of his rich but infirm aunt. I wonder at this, because the plan does not seem up to Becky’s standards: what if the old lady hangs on for another ten years? But perhaps she has a plan B that hasn't yet been explained.

Thackeray says that Becky is very good-looking, but in his illustrations she has a beaky nose and an unpleasant, predatory grin. In a recent film version she was played by Reese Witherspoon, which does not seem to me like a good fit. Although Becky is blonde, I keep picturing Aubrey Plaza, who always seems to me to be saying something intended to distract you from what she is really thinking.

I don't know yet if I will finish Vanity Fair—I never know if I will finish a book until I finish it, and I have at times said “fuck this” and put down a book that I was ninety-five percent of the way through—but right now I am eager to find out what happens next.

Blah blah blah

This post observes the tenth anniversary of this blog, which I started in January 2006, directly inspired by Steve Yegge’s rant on why You Should Write Blogs, which I found extremely persuasive. (Articles that appear to have been posted before that were backdated, for some reason that I no longer remember but would probably find embarrassing.) I hope my Gentle Readers will excuse a bit of navel-gazing and self-congratulation.

When I started the blog I never imagined that I would continue as long as I have. I tend to get tired of projects after about four years and I was not at all sure the blog would last even that long. But to my great surprise it is one of the biggest projects I have ever done. I count 484 published articles totalling about 450,000 words. (Also 203 unpublished articles in every possible state of incompletion.) I drew, found, stole, or otherwise obtained something like 1,045 diagrams and illustrations. There were some long stoppages between articles, but I always came back to it. And I never wrote out of obligation or to meet a deadline, but always because the spirit moved me to write.

Looking back on the old articles, I am quite pleased with the blog and with myself. I find it entertaining and instructive. I like the person who wrote it. When I'm reading articles written by other people it sometimes happens that I smile ruefully and wish that I had been clever enough to write that myself; sometimes that happens to me when I reread my own old blog articles, and then my smile isn't rueful.

The blog paints a good picture, I think, of my personality, and of the kinds of things that make me unusual. I realized long long ago that I was a lot less smart than many people. But the way in which I was smart was very different from the way most smart people are smart. Most of the smart people I meet are specialists, even ultra-specialists. I am someone who is interested in a great many things and who strives for breadth of knowledge rather than depth. I want to be the person who makes connections that the specialists are too nearsighted to see. That is the thing I like most about myself, and that comes through clearly in the blog. I know that if my twenty-five-year-old self were to read it, he would be delighted to discover that he would grow up to be the kind of person that he wanted to be, that he did not let the world squash his individual spark. I have changed, but mostly for the better. I am a much less horrible person than I was then: the good parts of the twenty-five-year-old’s personality have developed, and the bad ones have shrunk a bit. I let my innate sense of fairness and justice overcome my innate indifference to other people’s feelings, and I now treat people less callously than before. I am still very self-absorbed and self-satisfied, still delighted above all by my own mind, but I think I do a better job now of sharing my delight with other people without making them feel less.

My grandparents had Eliot and Thackeray on the shelf, and I was always intrigued by them. I was just a little thing when I learned that George Eliot was a woman. When I asked about these books, my grandparents told me that they were grown-up books and I wouldn't like them until I was older—the implication being that I would like them when I was older. I was never sure that I would actually read them when I was older. Well, now I'm older and hey, look at that: I grew up to be someone who reads Eliot and Thackeray, not out of obligation or to meet a deadline, but because the spirit moves me to read.

Thank you Grandma Libby and Grandpa Dick, for everything. Thank you, Gentle Readers, for your kind attention and your many letters through the years.

Categories: Offsite Blogs

Pulling an applicative/apply out of a record

haskell-cafe - Sun, 02/07/2016 - 9:25am
Quick question for the experts out there. I need to process some records. The records come in in an unordered stream of record fields. From each stream of fields I want to produce a Record entry containing the fields I need to do my calculation. I was thinking this setup might work nice with something like as I could then have (1) a monoid version I could accumulate use as an accumulator over the stream and (2) a final version. This final version seems like it should easily come from the later by pulling the [] (or anything else that has the Apply/Applicative structure) to the outside and then taking the head of the resulting list. While I could write the boilerplate to do this, it seems like something I should be able to do more elegantly. The best I have come up with so far is to use the lens package to give me an isomorphism to a tuple. Then I need a uncurried zip for arbitrarily large tuples. Follow this by an isomorphism back. RecordF (f Type1) (f Type2) ... -> (f Type1, f Type2, ....) -
Categories: Offsite Discussion

generating object file for ARM

haskell-cafe - Sat, 02/06/2016 - 5:19pm
Hello, I would like to link my Haskell module with existing object files for armv7-m. I use ghc 7.4.1 on debian/amd64. This command seems incorrect or incomplete: ghc -pgmcarm-none-eabi-gcc -pgmParm-none-eabi-cpp -pgmaarm-none-eabi-as -pgmlarm-none-eabi-ld -keep-tmp-files HaskellModule.hs A lot of errors appeared: [1 of 1] Compiling HaskellModule ( HaskellModule.hs, HaskellModule.o ) /tmp/ghc13654_0/ghc13654_0.s: Assembler messages: /tmp/ghc13654_0/ghc13654_0.s:5:0: Error: unrecognized symbol type "" /tmp/ghc13654_0/ghc13654_0.s:17:0: Error: unrecognized symbol type "" /tmp/ghc13654_0/ghc13654_0.s:29:0: Error: bad instruction `leaq -40(%rbp),%rax' /tmp/ghc13654_0/ghc13654_0.s:30:0: Error: bad instruction `cmpq %r15,%rax' Any hints ?
Categories: Offsite Discussion

ANN: servant-swagger and swagger2

haskell-cafe - Sat, 02/06/2016 - 4:59pm
Dear Cafe, I am pleased to announce servant-swagger [1] and swagger2 [2] packages! Swagger [3] is a project used to describe and document RESTful APIs. Here's a servant blog post, introducing both packages: servant-swagger allows you to generate a Swagger specification from your Servant API. swagger2 is a standalone package for Swagger 2.0. This package introduces lenses and generic-based derivation to minimize the effort of constructing a Swagger specification for your API. Both packages are supposed to be easy to use and have great documentation! So if you have any troubles using/understanding them — feel free to report an issue or contact me directly! Kind regards, Nick [1] [2] [3] _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >
Categories: Offsite Discussion

Douglas M. Auclair (geophf): Graphing with Goats

Planet Haskell - Sat, 02/06/2016 - 9:43am
Slides, presented comme ça! Links at the end. Presentation posted on youtube of the meetup.



3 (I am known for teh kittehs)

4 (graphs-я-borin' is another talk I gave at SanFran GraphConnect 2015)

5. The Beach (not Plastic)



8. ACID means something very important to DBMSes(eses)(eses)

9. neo4j Graph of Amino Acids (data table, Haskell code)

10. links:,, links to geddit?)

11. "sed-butt, sed-butt, sed-butt" my daughters chant around the house all day
Now: Graph-applications:

12. Social Media

13. The Markets

14. The Markets (again) / Managing Complexity

15. Search / (Fraud) Detectione.g.: Regressive Imagery Dictionary

16. Scoping / (Requirements) Analysis

17. Clustering

18. Errybody say: "YAAAAAAYYYYYY!"

19. Links, en-text-ified:
20. Buh-bay!
Categories: Offsite Blogs

Combining Pipes with Callbacks

haskell-cafe - Sat, 02/06/2016 - 9:17am
Is it possible to shoehorn a function which uses send/receive callbacks in to the definition of a Pipe?: If so, how would I define `cb' here?
Categories: Offsite Discussion

How to handle recoverable errors in multi-stepcomputation?

haskell-cafe - Sat, 02/06/2016 - 4:58am
Dear haskellers, I have a multi-step computation. The steps are predefined, but users can choose which steps to execute (something like a recipe). Here is an example recipe: Step 1 - Get data from network (multiple HTTP calls) and put into a list Step 2 - Process the data (e.g. average, sum, median, etc.) Step 3 - Persist result to database Sometimes, Step 1 can fail for some of the HTTP calls. When this happens, Step 2 should continue as much as possible using whatever data that has been retrieved, but somehow indicate that an error has occurred and the result is partial. Q1: What is the idiomatic way of achieving this? Using throwError in Control.Monad.Except aborts the computation, which isn't what I want. Q2: (General software design) Furthermore, where should the error be logged? Logging it in both Step 1 and 2 preserves modularity for each of the steps, unfortunately it would result in duplicate error messages. What is the best practice for this? Regards, Hon _____________________________________
Categories: Offsite Discussion

New type of ($) operator in GHC 8.0 is problematic

haskell-cafe - Sat, 02/06/2016 - 1:14am
(Sorry you got this twice, Kyle!) I agree wholeheartedly with this: community; the newcomers. Sacrificing the 99% of beginner and intermediate haskellers for the 1%, I believe is a step in the wrong direction. But also with this: from unsuspecting users. We already get a steady trickle of bug reports stemming from confusion around hidden kinds. Users diligently try to make a minimal test case and then someone has to point out that the user is wrong. It's a waste of time and, I'm sure, is frustrating for users. I'm worried about this problem getting worse. I've been learning Haskell for about a year and a half, and using it in production for roughly a third of that. More than once I've run into a language construct which struck me as odd, asked about it, and was told it was that way for pedagogical reasons. I consider this a poor state of affairs on all sides, including pedagogy! I had a correct intuition that something was amiss, but here's the language itself sewing unjustified doubt about my understan
Categories: Offsite Discussion

Foldable/Traversable and Applicative/Monoid?

haskell-cafe - Fri, 02/05/2016 - 8:45pm
Hi all, I don't understand why Foldable is a necessary super-class of Traversable, and I suspect that the Applicative/Monoid duality, which I've just begun discovering in the literature, has something to do with why that is so. Can anyone give me a hint, without giving me the answer? Thanks! -db _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >
Categories: Offsite Discussion

Month in haskell-ide-engine January 2016

haskell-cafe - Fri, 02/05/2016 - 7:47pm
Welcome Haskell IDE Engine users, Haskell IDE Engine progress report for January 2016 [1] What is Haskell IDE Engine? Not an IDE. It is a common point for people in the Haskell community to pool their efforts with respect to tooling. For tool writers, provide tools as a HIE plugin, so it can be available on supported IDEs For IDE developers, integrate to HIE, and all the Haskell tools supported as plugins become available For users, it means the overall Haskell experience should improve. Important developments A new ghc-dump-tree plugin based on Current project focus The current focus is to get the initial version working well enough for an alpha release. To this end, there are some hardy developers using it in their daily work in emacs. Issues closed in January Querying a graph database instead of using GHC-API? #10 Decide how haskell-ide project is run #13 Rework the Console #20 Use an error handler in the dispatcher #50 Protocol definition #66 (emacs) "S
Categories: Offsite Discussion

Christopher Allen: Haskell is not trivial, but it's not unfair like Dark Souls either

Planet Haskell - Fri, 02/05/2016 - 6:00pm

Alternate title: Another brutal fisking for which I’ll forever be remembered as a monster

Don’t be a dick and don’t harass the original author please. They were just sharing their experience.

It took me five years to stop struggling with Haskell, I know a lot of the UX stuff sucks. I think there’s a lot of value in using Haskell for everyday work so it’s disappointing when things fall apart for a learner for unnecessary reasons.

I’ve since talked to the author of the original post and they’re cool with this post. Also they agree with me that wreq should get more airplay. On with the show.

Getting the ball rolling on talking to HTTP APIs

I want to collect some statistics from the GitHub API. Watch as I retrace my steps attempting the Tomb of the Dread HTTPS GET Request.

Okay, is there a reason we’re not going to use the Haskell Github client for that? Anyway, we’ll follow along with what the author’s doing for now.

Now I need to query the GitHub API. Not my first time to the rodeo, I generate a personal access token from GitHub and copy it to a local file. What query should I run first? How about the count for all ASM tetris repositories? Poking around the docs comes up with:

GET User-Agent: victim Authorization: token PUT_TOKEN_HERE

Cool so far. Think the author figured out Github’s docs more easily than I did.

Easy life. Now how do you GET a resource in Haskell? Ah, Network.HTTP! I copy the front page sample into src/Lib.hs

Okay first mistake. I know it sucks, but you want to be careful about using Hackage to find libraries for things unless you’re good at sniff-testing APIs. It’s generally better to ask what a good library to use is. The library named “HTTP” is a bit creaky and there are nicer, more up to date ways of doing HTTP in Haskell. Entirely not the author’s fault, but it’s pretty hard to get Hackage to do anything useful anyway. I know this is sucky implicit crap nobody should have to care about, but Haskell just doesn’t have the culture of design-focused self-promotion that other communities have. Not a value judgment, in the end it’s probably better for end-users if they can use design as a signal of how up to date or nice a library is, but that’s just how it is right now. It would probably help if there were more Haskellers that didn’t sneer at web devs.


-- the article's version module Lib ( someFunc ) where x = simpleHTTP (getRequest "") >>= fmap (take 100) . getResponseBody someFunc :: IO () someFunc = print x

Well. Yes that sucks. It’s also a weird way to write it. It’s like the code people write the first time they figure out how do syntax desugars into >>=, then they just start using >>= and point-free all over the place for the fuck of it. We’ll re-do it in wreq:

-- first.hs -- this is my version module FirstExample ( someFunc ) where -- Don't give me any shit about lens. -- You don't need to understand lens -- to know the ^. is for accessing a -- record field. The wreq tutorial -- lays out the common use-cases. import Control.Lens import qualified Data.ByteString.Lazy as BL import Network.Wreq -- Brand X -- simpleHTTP (getRequest "") >>= fmap (take 100) . getResponseBody -- someFunc :: IO () -- someFunc = -- print x -- our version someFunc :: IO () someFunc = do response <- get "" print $ BL.take 100 (response ^. responseBody)

To load this beast up:

-- yes this'll take a moment, but then you won't -- have to do it again because it's Stack. $ stack build lens wreq $ stack ghci Prelude> :l first.hs [1 of 1] Compiling FirstExample ( first.hs, interpreted ) Ok, modules loaded: FirstExample. Prelude> someFunc "<!DOCTYPE html>\n<html lang=\"en\" class=\"\">\n <head prefix=\"og: fb:"

Right-o, moving along.

Doesn’t compile. Durp, hackage is a package library, I need to add this to my cabal.

If you want to and you wanted a package for this, sure. I’ll typically use a stack template for new projects, but for initial exploration I’ll build the libraries as above I want and use them in stack ghci.

…author struggles with HTTP only supporting HTTP…

wreq and http-client-tls support HTTPS out of the box. YMMV. There’s a reason I don’t really recommend older Haskell libraries even if they’re maintained. The foundation of many libraries is http-client and it’s a pretty popular library to use. It’s used in http-conduit and pipes-http as well. The latter of which is a single 130 line module that has required almost zero maintenance in the past two years to add pipes streaming support to http-client. Things that use http-client are generally nice but you’ll often want to use something higher level than http-client itself, such as wreq.

Author moves on to use http-conduit, which uses http-client-tls under the hood

query :: IO String query = do initReq <- parseUrl "" let r = initReq { method = "GET" , requestHeaders = [(hUserAgent, "victim") , (hAuthorization, "token PUT_TOKEN_HERE")]} let request = setQueryString [("q", Just "tetris+language:assembly") ,("order", Just "desc") ,("sort", Just "stars")] r manager <- newManager tlsManagerSettings res <- httpLbs request manager return . show . responseBody $ res someFunc :: IO () someFunc = do query >>= putStrLn

They’re not using the streaming, might as well use wreq. Normally you’d have, such as in a web framework, an HTTP client pool which gets initialized with the web app once and then shared with the handlers. So the initial setup code would happen once. I do think the API staging out the parseUrl part is a bit pointless for the common-case but w/e. For the record, I wouldn’t consider this code to be bad.

As it happens, wreq has an example of how to talk to Github’s API in the tutorial here if you ctrl-f “most popular implementations of Tetris” you’ll find it.

Prelude> let opts = defaults & param "q" .~ ["tetris"] & param "language" .~ ["haskell"] Prelude> r <- getWith opts ""

As it happens, we can just skip past the explicit params thing and just do this:

Prelude> response <- get "" Prelude> response ^. responseBody

But uh, we’ll get back to what they’re trying to do.

Time to parse this mega JSON string. Aeson seems to be the biggest contender. To use Aeson and get the total_count value from the return, I needed the following additions:

{-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE DeriveGeneric #-} import GHC.Generics import Data.Aeson data ResultCount = ResultCount { total_count :: Int } deriving (Generic, Show) instance ToJSON ResultCount instance FromJSON ResultCount

Huh? No you don’t! Int already has a FromJSON instance.

Just to make the point, I’ll do it in GHCi again with no module.

$ stack build aeson lens-aeson $ stack ghci Prelude> import Network.Wreq Prelude> import Control.Lens Prelude> import Data.Aeson Prelude> import Data.Aeson.Lens Prelude> :set -XOverloadedStrings Prelude> response <- get "" Prelude> response ^? responseBody . key "total_count" Just (Number 354.0) Prelude> response ^? responseBody . key "total_count" . _Number Just 354.0

Don’t make it harder than it has to be. Ask for help!

I know this site is a bit of a disaster zone, but if you like my writing or think you could learn something useful from me, please take a look at the book I've been writing with my coauthor Julie. There's a free sample available too!

Posted on February 6, 2016

Categories: Offsite Blogs

servant: Announcing servant-swagger and swagger2

Planet Haskell - Fri, 02/05/2016 - 6:00pm
<section class="level2" id="swagger"> Swagger

Servant is not the first project to provide a unified way of documenting APIs. There is API Blueprint, RAML, Apiary, and finally swagger. While these Web API description languages are not also web frameworks , they are generally very mature, and have some amazing tooling. For example, take a look at what swagger-ui, a client-side HTML, CSS, and JS bundle, does with your swagger API description here.

As you can see, it’s a very convenient and approachable way of exploring your API. In addition to an easily-navigable structure, you can build up requests and send them to your server, and see its responses.

But it doesn’t end there. If you have a swagger specification of your API, you can also take advantage of the large variety of languages for which you can generate a client library automatically. You don’t even need to build the Java code - you can just use the “Generate Client” button in the beautiful swagger editor.

There are a wide array of other tools that support swagger. Obviously, having access to them would be a great boon. The problem so far has been that writing and maintaining a swagger specification, that you can be sure matches your service, is hard work.

</section> <section class="level2" id="swagger2-and-servant-swagger"> swagger2 and servant-swagger

Thankfully David Johnson and Nickolay Kudasov have written two Haskell libraries, swagger2 and servant-swagger, that automate nearly all of that process for servant APIs. They use the mechanism that guides most of the servant ecosystem — interpreters for the type-level DSL for APIs that is servant — to generate a swagger spec for that API.

Let’s see how it is used; as an example, we’re going to take the Gists part of the GitHub API v3. For the purpose of this post we will ignore authentication and consider only GET requests which do not require one. Furthermore, we’ll use simplified representation for the responses (i.e. we are also ignoring some fields of the response objects).

First the imports and pragmas (this is a literate haskell file):

{-# LANGUAGE DataKinds #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TypeOperators #-} module Gists where import Control.Lens import Data.Aeson import Data.Aeson.Types (camelTo2) import qualified Data.Aeson.Types as JSON import qualified Data.ByteString.Lazy.Char8 as BL8 import Data.HashMap.Strict (HashMap) import Data.Proxy import Data.Swagger import Data.Text (Text) import Data.Time (UTCTime) import GHC.Generics (Generic) import Servant import Servant.Swagger

The API:

type GitHubGistAPI = "users" :> Capture "username" Username :> "gists" :> QueryParam "since" UTCTime :> Get '[JSON] [Gist] :<|> "gists" :> GistsAPI type GistsAPI = "public" :> QueryParam "since" UTCTime :> Get '[JSON] [Gist] :<|> "starred" :> QueryParam "since" UTCTime :> Get '[JSON] [Gist] :<|> Capture "id" GistId :> GistAPI type GistAPI = Get '[JSON] Gist :<|> Capture "sha" Revision :> Get '[JSON] Gist api :: Proxy GitHubGistAPI api = Proxy

Data types:

newtype Username = Username Text deriving (Generic, ToText, FromJSON) newtype GistId = GistId Text deriving (Generic, ToText, FromJSON) newtype SHA = SHA Text deriving (Generic, ToText) type Revision = SHA data Gist = Gist { gistId :: GistId , gistDescription :: Text , gistOwner :: Owner , gistFiles :: HashMap FilePath GistFile , gistTruncated :: Bool , gistComments :: Integer , gistCreatedAt :: UTCTime , gistUpdatedAt :: UTCTime } deriving (Generic) data OwnerType = User | Organization deriving (Generic) data Owner = Owner { ownerLogin :: Username , ownerType :: OwnerType , ownerSiteAdmin :: Bool } deriving (Generic) data GistFile = GistFile { gistfileSize :: Integer , gistfileLanguage :: Text , gistfileRawUrl :: Text } deriving (Generic)

FromJSON instances:

modifier :: String -> String modifier = drop 1 . dropWhile (/= '_') . camelTo2 '_' prefixOptions :: JSON.Options prefixOptions = JSON.defaultOptions { JSON.fieldLabelModifier = modifier } instance FromJSON OwnerType instance FromJSON Owner where parseJSON = genericParseJSON prefixOptions instance FromJSON GistFile where parseJSON = genericParseJSON prefixOptions instance FromJSON Gist where parseJSON = genericParseJSON prefixOptions

So far this is what you would usually have when working with servant. Now to generate Swagger specification we need to define schemas for our types. This is done with ToParamSchema and ToSchema instances:

prefixSchemaOptions :: SchemaOptions prefixSchemaOptions = defaultSchemaOptions { fieldLabelModifier = modifier } instance ToParamSchema SHA instance ToParamSchema Username instance ToParamSchema GistId instance ToSchema Username instance ToSchema GistId instance ToSchema OwnerType instance ToSchema Owner where declareNamedSchema = genericDeclareNamedSchema prefixSchemaOptions instance ToSchema GistFile where declareNamedSchema = genericDeclareNamedSchema prefixSchemaOptions instance ToSchema Gist where declareNamedSchema = genericDeclareNamedSchema prefixSchemaOptions

These will give us a generically-derived Swagger schema (which is sort of a deterministic version of JSON Schema).

Part of the swagger2 package, Schema and ParamSchema can be quite useful in their own right if you want to e.g. respond with a schema in case of bad request bodies, or OPTIONS requests.

The next step will traverse the GitHubGistAPI, gathering information about it and swagger2 schemas to generate a Swagger value:

swaggerDoc1 :: Swagger swaggerDoc1 = toSwagger api

Now we can generate the swagger documentation:

genSwaggerDoc1 :: IO () genSwaggerDoc1 = BL8.putStr $ encode swaggerDoc1

You can attach more information to your Swagger doc quite easily, using the lenses provided by swagger2:

swaggerDoc2 :: Swagger swaggerDoc2 = swaggerDoc1 & host ?~ "" & info.title .~ "GitHub Gists API" & info.version .~ "v3" main :: IO () main = BL8.putStr $ encode swaggerDoc2

Which results in this.

There’s a lot more you can do with both servant-swagger and swagger2 — write manual ToSchema instances for more detailed information, conveniently add tags or change responses of parts of your API, use convenient lenses to modify any part of your schema, generate automatic tests, etc.

Check out the servant-swagger and swagger2 docs for more.

These two new packages vastly expand the landscape of tools within easy reach of application developers using servant. Time to explore that landscape!

On a related note, Masahiro Yamauchi has recently added Servant codegen for Swagger. So not only can you generate a swagger description for your servant server, but you can also generate the servant description from a swagger one too!

</section> Posted on February 6, 2016 by David Johnson, Nickolay Kudasov, Julian Arni
Categories: Offsite Blogs

Bug tracker for cpphs

libraries list - Sat, 01/30/2016 - 7:19am
Hi Malcolm, (I'm resending my old question to the libraries mailing list) Is there a bug tracker for cpphs? Best,
Categories: Offsite Discussion

New gtk2hs 0.12.4 release

gtk2hs - Wed, 11/21/2012 - 12:56pm

Thanks to John Lato and Duncan Coutts for the latest bugfix release! The latest packages should be buildable on GHC 7.6, and the cairo package should behave a bit nicer in ghci on Windows. Thanks to all!


Categories: Incoming News