News aggregator

Kevin Reid (kpreid): A Visual Introduction to DSP for SDR (video)

Planet Haskell - Thu, 11/20/2014 - 12:41pm

I have really failed to get around to blogging what I've been doing lately, which is all software-defined radio. Let's start fixing that, in reverse order.

Yesterday, I went to a Bay Area SDR meetup, “Cyberspectrum” organized by Balint Seeber and gave a presentation of visual representations of digital signals and DSP operations. It was very well received. This video is a recording of the entire event, with my talk starting at 39:35.

<lj-embed id="7">

Categories: Offsite Blogs

Four Haskell books

Haskell on Reddit - Thu, 11/20/2014 - 8:56am
Categories: Incoming News

[Please explain] Singleton pattern matching and exhaustiveness checker?

Haskell on Reddit - Thu, 11/20/2014 - 7:38am

Hello guys,

Here is a code snippet. When I am trying to compile it with:

$ ghc -fforce-recomp -fwarn-incomplete-patterns Test.hs

I am getting:

Test.hs:54:19: Warning: Pattern match(es) are non-exhaustive In a case alternative: Patterns not matched: SUnbound SSyntax

But having uncommented either line 56 or line 57 one can see:

Test.hs:56:21: Couldn't match type ‘'Unbound’ with ‘'Core’ Inaccessible code in a pattern with constructor SUnbound :: SLevel 'Unbound, in a case alternative In the pattern: SUnbound In a case alternative: SUnbound -> undefined In the expression: case levelOf x of { SCore -> reduce x SUnbound -> undefined }

Question:

What should be changed in the code to pass exhaustiveness checker? Please don't suggest:

_ -> error "impossible"

Thanks a lot :)

submitted by dima_mendeleev
[link] [7 comments]
Categories: Incoming News

Why does folding events and behaviors use so much memory? (x-post from stack overflow)

Haskell on Reddit - Thu, 11/20/2014 - 6:58am

Normally I avoid posting questions here. But in this case I am a little desperate and my guess is that the problem itself might be interesting to some.

So here goes nothing: http://stackoverflow.com/questions/27040190/why-does-folding-events-and-behaviors-use-so-much-memory

submitted by goliatskipson
[link] [comment]
Categories: Incoming News

Alternate formats for `The Monad Reader` issues?

Haskell on Reddit - Thu, 11/20/2014 - 1:56am

Is it available in epub or kindle format? I recently came across the markdown version of the typeclassopedia and I'm enjoying it on my kindle. I just thought it would be great to have all issues in accessible formats.

PS: If its not available yet, who would like to join me to convert them all? :)

submitted by jaseemabid
[link] [4 comments]
Categories: Incoming News

Austin Seipp: Outages and improvements...

Planet Haskell - Wed, 11/19/2014 - 10:34pm

This past week, the primary Haskell.org server hosting the wiki and mailing systems went down due to RAID failures in its underlying hardware. Thanks to some (real) luck and hard work, we've managed to recover and move off the server to a new system, avoided some faulty hardware, and hopefully got some improved reliability.

The rundown on what happened

Before we started using Rackspace, eveyrthing existed on a single machine hosted by Hetzner, in Germany. This machine was named rock and ran several VMs with IPs allocated to them, which ran Hackage, Mediawiki (AKA www) and ghc.haskell.org on top of KVM.

This server had a RAID1 setup between two drives. These drives were partitioned to have a primary 1.7TB partition for most of the data that was striped, and another small partition also striped.

We had degredation on both partitions and essentially lost one drive completely. This is why the system had been so slow the past week and grinding to a halt so often.

A quick move

We began to investigate this when the drives became unable to fairly service IO almost at all. We found something really really bad: we hadn't had one of the drives in nearly two weeks!

We had neglected to install SMART and RAID monitoring in our Nagios setup. An enormous and nearly disastrous blunder. And we didn't even look at our SFTP access for backups that Hetzner provided, but it was close to out of space last we checked. But we seemed to be OK in the read-only workload.

Overall, it was some amateur mistakes that almost cost us. We'd been so focused on moving things out, we didn't even take the time to check the server when we moved Hackage a few weeks prior and making sure the existing things kept working.

We'd planned to do this move earlier, but it obviously couldn't wait. So Davean spent most of his Tuesday migrating all the services over to Rackspace on a new VM, and we migrated the data to our shared MySQL server, and got the mail relay running again.

The whole process took close to 12 hours or so I'd say, but overall went quite smoothly without any kind of read errors or problems on the remaining drive, and restored service.

Improvements

In the midst of anticipating this move, I migrated a lot of download data - specifically GHC and the Haskell Platform - to a new server, https://downloads.haskell.org. It's powered by a brand new CDN via Fastly, and provides the Platform, and GHC downloads and documentation. Don't worry: redirects for the main server are still in place.

We've also finally fixed a few bugs stopping us from deploying the new main website. We've deployed https://try.haskell.org now as an official instance, as well as fixed https://new-www.haskell.org to use it.

We've also moved most the MediaWiki data to our consolidated MariaDB server. However, because we were so hastily moving data, we didn't put the new wiki in the same DC as the database! As a result, we have somewhat of a performance loss. In the mean time, we've deployed Memcached to try and help offset this a bit. We'll be moving things again soon, but it will hopefully be faster and much less problematic.

We've also taken the time to archive a lot of our data from Rock onto DreamHost, who have joined us and given us free S3-compatible object storage! We'll be moving a lot of data in there soon.

Future improvements

We've got several improvements we're going to deploy in the near future:

  • We're obviously going to overhaul our Nagios setup, which is clearly inadequate and now out of date with all the recent changes.
  • We'll be moving Fastly in front of Hackage soon, which should hopefully dramatically increase download speeds for all packages, and save us a lot of bandwidth in the long run.
  • Beyond memcached, our MediaWiki instance will be using Fastly as a CDN, as it's one of the most bandwidth-heavy parts of the site besides Hackage. We'll be working towards integrating purge support from Mediawiki with the upstream CDN.
  • We'll also be moving our new wiki server to a separate DC so it can talk to the MySQL server more efficiently.
  • We're planning on doing some other tech-stack upgrades: a move to nginx across the remaining servers using apache, upgrading MediaWiki, etc.
  • We'll be working towards better backups for all our servers, possibly using a solution like Attic and S3 synchronization.

And some other stuff I'll keep secret.

Conclusion

Overall, the past few weeks have seen several improvements but a lot of problems, and some horrible mistakes that almost cost us weeks of data if we hadn't been careful. Most of our code, repositories, configurations, and critical data was mirrored at the time - but we still got struck where we were vulnerable, on old hardware we had issues with before.

For that, I take responsibility on behalf of the administration team (as the de-facto lead, it seems) for the outage this past week, which cost us nearly a day of email and main site availability.

The past week hasn't been great, but here's to hoping the next one will be better. Upwards and onwards.

Categories: Offsite Blogs

5outh/Bang · GitHub

del.icio.us/haskell - Wed, 11/19/2014 - 9:27pm
Categories: Offsite Blogs

acowley.github.io

del.icio.us/haskell - Wed, 11/19/2014 - 7:07pm
Categories: Offsite Blogs

Christopher Done: Lucid: templating DSL for HTML

Planet Haskell - Wed, 11/19/2014 - 6:00pm

I’m not big on custom templating languages, for reasons I’ll write about another time. I prefer EDSLs. I preferred the xhtml package back when that was what everybody used. It looked like this:

header << thetitle << "Page title" thediv noHtml ! [theclass "logo"] << "…" thediv noHtml ! [identifier "login"]

Pretty line-noisy to read, write and hard to edit in a reasonable manner.

Later, blaze-html became the new goto HTML writing library. It improved upon the XHTML package by being faster and having a convenient monad instance. It looks like this:

page1 = html $ do head $ do title "Introduction page." link ! rel "stylesheet" ! type_ "text/css" ! href "screen.css" body $ do div ! id "header" $ "Syntax" p "This is an example of BlazeMarkup syntax." ul $ mapM_ (li . toMarkup . show) [1, 2, 3]

Much easier to read, write and edit thanks to the monad instance.

However, after several years of using that, I’ve come to write my own. I’ll cover the infelicities about Blaze and then discuss my alternative approach.

Reading back through what I’ve written below, it could be read as a bit attacky, and some of the issues are less philosophical and more incidental. I think of it more that the work on writing HTML in a DSL is incomplete and to some degree people somewhat gave up on doing it more conveniently at some point. So I’m re-igniting that.

The combination of having a need to write a few HTML reports and recent discussions about Blaze made me realise it was time for me to come at this problem a-fresh with my own tastes in mind. I also haven’t used my own approach much, other than porting some trivial apps to it.

Blaze Names that conflict with base

The first problem is that Blaze exports many names which conflict with base. Examples:

div, id, head, map

The obvious problem with this is that you either have to qualify any use of those names, which means you have to qualify Blaze, and end up with something inconsistent like this:

H.div ! A.id "logo" $ "…"

Where H and A come from importing the element and attribute modules like this:

import qualified Text.Blaze.Html5 as H import qualified Text.Blaze.Html5.Attributes as A

Or you don’t import Prelude and only import Blaze, but then you can’t do a simple map without qualification.

You might’ve noticed in the old xhtml package that thediv and identifier are used instead. The problem with using different names from the actual things they refer to is that they’re hard to learn and remember, both for regular Haskellers and newbies coming to edit your templates.

Names that are keywords

This is a common problem in DSLs, too. In Blaze the problem is: class or type (perhaps others I don’t recall). Blaze solves it with: class_ or type_

Again, the problem with this is that it is inconsistent with the other naming conventions. It’s another exception to the rule that you have to remember and makes the code look bad.

Conflicting attribute and element names

There are also names which are used for both attributes and elements. Examples are style and map. That means you can’t write:

H.head $ style "body { background: red; }" body $ p ! style "foo" $ …

You end up writing:

H.head $ H.style "body { background: red; }" body $ p ! A.style "foo" $ … Inconsistency is difficult and ugly

What the above problems amount to is ending up with code like this:

body $ H.div ! A.id "logo" ! class_ "left" ! hidden $ "Content"

At this point users of Blaze give up with second-guessing every markup term they write and decide it’s more consistent to qualify everything:

H.body $ H.div ! A.id "logo" ! A.class_ "left" ! A.hidden $ "Content"

Or, taken from some real code online:

H.input H.! A.type_ "checkbox" H.! A.checked True H.! A.readonly "true"

This ends up being too much. Inconvenient to type, ugly to read, and one more step removed from the HTML we’re supposed to be generating.

The Monad instance isn’t

The monad instance was originally conceived as a handy way to write HTML nicely without having to use <> or lists of lists and other less wieldy syntax.

In the end the monad ended up being defined like this:

instance Monad MarkupM where return _ = Empty {-# INLINE return #-} (>>) = Append {-# INLINE (>>) #-} h1 >>= f = h1 >> f (error "Text.Blaze.Internal.MarkupM: invalid use of monadic bind") {-# INLINE (>>=) #-}

And has been for some years. Let’s take a trivial example of why this is not good. You render some HTML and while doing so build a result to be used later:

do xs <- foldM (\c i -> …) mempty ys mapM_ dd xs

Uh-oh:

*** Exception: Text.Blaze.Internal.MarkupM: invalid use of monadic bind The overloaded strings instance is bad

The previous point leads onto this next point, which is that due to this phantomesque monad type, the instance is like this:

instance IsString (MarkupM a) where fromString = Content . fromString {-# INLINE fromString #-}

How can it make this value? It cannot. If you want to go ahead and extract that `a’, you get:

*** Exception: Text.Blaze.Internal.MarkupM: invalid use of monadic bind

Additionally, this instance is too liberal. You end up getting this warning:

A do-notation statement discarded a result of type GHC.Prim.Any

Suppress this warning by saying _ <- "Example" or by using the flag -fno-warn-unused-do-bind

So you end up having to write in practice (again, taken from a real Blaze codebase by one of the authors):

void "Hello!"

Which pretty much negates the point of using IsString in the first-place. Alternatively, you use -fno-warn-unused-do-bind in your module.

Working with attributes is awkward

The ! syntax seems pretty convenient from superficial inspection:

link ! rel "stylesheet" ! type_ "text/css" ! href "screen.css"

But in practice it means you always have the same combination:

div ! H.class_ "logo" $ "…"

Which I find—personally speaking—a bit distasteful to read, it’s not far from what we saw in the old xhtml package:

thediv ! [theclass "logo"] << "…"

Did we really save that much in the attribute department? Operators are evil.

But mostly presents an editing challenge. Operators like this make it tricky to navigate, format in a regular way and do code transformations on. All Haskell code has operators, so this is a general problem. But if your DSL doesn’t actually need these operators, I consider this a smell.

Attributes don’t compose

You should be able to compose with. For example, let’s say you want to define a re-usable component with bootstrap:

container inner = div ! class_ "container" $ inner

Now you can use it to make a container. But consider now that you also want to add additional attributes to it later. You can do that with another call to with:

container ! class_ "main" $ "zot"

In Blaze this produces:

λ> main "<div class=\"container\" class=\"main\">My content!</div>"

Browsers ignore the latter main, so the composition didn’t work.

Ceremony is tiring

Here’s the example from Blaze’s package, that’s introduced to users.

import Prelude hiding (head, id, div) import Text.Blaze.Html4.Strict hiding (map) import Text.Blaze.Html4.Strict.Attributes hiding (title) import Text.Blaze.Renderer.Utf8 (renderMarkup) page1 :: Markup page1 = html $ do head $ do title "Introduction page." link ! rel "stylesheet" ! type_ "text/css" ! href "screen.css" body $ do div ! id "header" $ "Syntax" p "This is an example of BlazeMarkup syntax." ul $ mapM_ (li . toMarkup . show) [1, 2, 3] main = print (renderMarkup page1)

Apart from the import backflips you have to do to resolve names properly, you have at least three imports to make just to render some HTML. Call me lazy, or stupid, but I never remember this deep hierarchy of modules and always have to look it up every single time. And I’ve been using Blaze for as long as the authors have.

Transforming

A smaller complaint is that it would sometimes be nice to transform over another monad. Simplest example is storing the read-only model information in a reader monad and then you don’t have to pass around a bunch of things as arguments to all your view functions. I’m a big fan of function arguments for explicit state, but not so much if it’s the same argument every time.

No Show instance

It would be nice if you could just write some markup in the REPL without having to import some other modules and wrap it all in a function just to see it.

Lucid

My new library, Lucid, attempts to solve most of these problems.

Naming issues

Firstly, all names which are representations of HTML terms are suffixed with an underscore _:

p_, class_, table_, style_

No ifs or buts. All markup terms.

That solves the following problems (from the issues described above):

  • Names that conflict with base: div_, id_, head_, map_, etc.
  • Names that are keywords: class_, type_, etc.
  • Conflicting attribute and element names: solved by abstracting those names via a class. You can write style_ to mean either the element name or the attribute name.
  • Inconsistency is difficult and ugly: there’s no inconsistency, all names are the same format.

No import problems or qualification. Just write code without worrying about it.

How it looks

Plain text is written using the OverloadedStrings and ExtendedDefaultRules extensions, and is automatically escaped:

λ> "123 < 456" :: Html () 123 &lt; 456

Elements nest by function application:

λ> table_ (tr_ (td_ (p_ "Hello, World!"))) <table><tr><td><p>Hello, World!</p></td></tr></table>

Elements are juxtaposed via monoidal append:

λ> p_ "hello" <> p_ "sup" <p>hello</p><p>sup</p>

Or monadic sequencing:

λ> div_ (do p_ "hello"; p_ "sup") <div><p>hello</p><p>sup</p></div>

Attributes are set using the with combinator:

λ> with p_ [class_ "brand"] "Lucid Inc" <p class="brand">Lucid Inc</p>

Conflicting attributes (like style_) work for attributes or elements:

λ> html_ (head_ (style_ "body{background:red}") <> with body_ [style_ "color:white"] "Look ma, no qualification!") <html><head><style>body{background:red}</style></head> <body style="color:white">Look ma, no qualification!</body></html> The Blaze example

For comparison, here’s the Blaze example again:

page1 = html $ do head $ do title "Introduction page." link ! rel "stylesheet" ! type_ "text/css" ! href "screen.css" body $ do div ! id "header" $ "Syntax" p "This is an example of BlazeMarkup syntax." ul $ mapM_ (li . toMarkup . show) [1, 2, 3]

And the same thing in Lucid:

page2 = html_ $ do head_ $ do title_ "Introduction page." with link_ [rel_ "stylesheet",type_ "text/css",href_ "screen.css"] body_ $ do with div_ [id_ "header"] "Syntax" p_ "This is an example of Lucid syntax." ul_ $ mapM_ (li_ . toHtml . show) [1,2,3]

I’m not into operators like ($) and swung indentation like that, but I followed the same format.

I’d write it in a more Lispy style and run my hindent tool on it:

page1 = html_ (do head_ (do title_ "Introduction page." with link_ [rel_ "stylesheet" ,type_ "text/css" ,href_ "screen.css"]) body_ (do with div_ [id_ "header"] "Syntax" p_ "This is an example of Lucid syntax." ul_ (mapM_ (li_ . toHtml . show) [1,2,3])))

But that’s another discussion.

It’s a real monad

Normal monadic operations work properly:

λ> (return "OK!" >>= p_) <p>OK!</p>

It’s basically a writer monad.

In fact, it’s also a monad transformer:

λ> runReader (renderTextT (html_ (body_ (do name <- lift ask p_ (toHtml name))))) ("Chris" :: String) "<html><body><p>Chris</p></body></html>" Overloaded strings instance is fine

The instance is constrained over the return type being (). So string literals can only be type HtmlT m ().

λ> do "x" >> "y" :: Html () xy λ> do x <- "x"; toHtml (show x) x() Attributes

Attributes are simply written as a list. That’s all. Easy to manipulate as a data structure, easy to write and edit, and automatically indent in a predictable way:

λ> with p_ [id_ "person-name",class_ "attribute"] "Mary" <p id="person-name" class="attribute">Mary</p>

No custom operators are required. Just the with combinator. If you want to indent it, just indent it like normal function application:

with p_ [id_ "person-name",class_ "attribute"] "Mary"

And you’re done.

Composing attributes

You should be able to compose with. For example, let’s say you want to define a re-usable component with bootstrap:

λ> let container_ = with div_ [class_ "container "]

Now you can use it to make a container:

λ> container_ "My content!" <div class="container ">My content!</div>

But consider now that you also want to add additional attributes to it later. You can do that with another call to with:

λ> with container_ [class_ "main"] "My content!" <div class="container main">My content!</div>

Duplicate attributes are composed with normal monoidal append. Note that I added a space in my definition of container anticipating further extension later. Other attributes might not compose with spaces.

Unceremonious

Another part I made sure was right was lack of import nightmare. You just import Lucid and away you go:

λ> import Lucid λ> p_ "OK!" <p>OK!</p> λ> p_ (span_ (strong_ "Woot!")) <p><span><strong>Woot!</strong></span></p> λ> renderBS (p_ (span_ (strong_ "Woot!"))) "<p><span><strong>Woot!</strong></span></p>" λ> renderToFile "/tmp/foo.html" (p_ (span_ (strong_ "Woot!")))

If I want to do more advanced stuff, it’s all available in Lucid. But by default it’s absolutely trivial to get going and output something.

Speed

Actually, despite having a trivial implementation, being a real monad and a monad transformer, it’s not far from Blaze. You can compare the benchmark reports here. A quick test of writing 38M of HTML to file yielded the same speed (about 1.5s) for both Lucid and Blaze. With such decent performance for very little work I’m already ready to start using it for real work.

Summary

So the point of this post was really to explain why another HTML DSL and I hope I did that well enough.

The code is on Github. I pushed to Hackage but you can consider it beta for now.

Categories: Offsite Blogs

FP Complete: Stackage server: new features and open source

Planet Haskell - Wed, 11/19/2014 - 6:00pm
Open source

We've been working on Stackage server for a while and now that the code has stabilized it's ready to be open source. You can fork it on Github! We're a responsive team, used to bringing pull requests forward and getting them deployed.

Since the last update we added a bunch of things. Here's a rundown:

Clarifying improvements
  • Recommended snapshots: The home page now lists recommended snapshots for GHC 7.8, GHC 7.8 Haskell Platform flavour, and GHC 7.6. Makes it a little more straight-forward where to start.

  • Snapshots: The snapshots page has been cleaned up a bit to cleanly separate times when snapshots are uploaded. Just an aesthetic improvement.

  • We also reorganized the Preparing your system to use Stackage wiki page to be more straight-forward to get started.

Packages

We now list all packages from Hackage on the packages page. A subset of these are included in Stackage snapshots. Hitting those links takes you to our new page package. Examples:

On this page we display:

  • Metadata for the most recently uploaded version of that package:

    • Name, version, license, authorship, maintainer
  • We also display the dependencies of that package. Notice that there are no version constraints. That information is stored in the snapshot itself.

  • Documentation versions: each snapshot that contains a version of this package will be listed along with documentation for that version.

README

We support showing a README from your package (see e.g. hlint example above), which is a really nice way to introduce people to your package. You can use the same one that you use for your Github README.md, just include it in your .cabal file with:

extra-source-files: README.md

If your filename is just README it'll be included as plain text. With the .md extension it will be rendered as markdown.

Please do go ahead and include your README.md in your .cabal file. If you don't have one, please write one and make it helpful and descriptive for any newbies to your package.

If no README file is found it falls back to the package description, which is displayed as plain text. As is well-known in the community at large, writing descriptive pros in Haddock syntax is not pleasant, whereas everyone is pretty much writing their READMEs in markdown anyway thanks to Github.

CHANGELOGs are also displayed as markdown if the file extension is .md, otherwise it's treated as plain text.

Specifying remote-repo in sandboxes

An issue with using Stackage in the past was that you had to either put your remote-repo field in your global cabal config, or setup an hsenv. Or by using a cabal.config with constraints in it. Now the feature has been merged to be able to specify remote-repo in the cabal.config file in your project root. Once this is released you'll be able to use a Stackage snapshot within a sandbox and keep the cabal.config file in your source repository.

Additional package page features

Additional features include:

  • Tagging: tagging a package is easy: you just click the + button, type something and hit return.

    We try to keep the tags of a simple format (a slug), so if you type characters that shouldn't be in there, it'll remove them and then prompt you to confirm.

    Related: if you click a tag it will take you to a page of all packages tagged with that, e.g. parsing.

    Finally, there is a list of all tags. Currently it's rather small because that's just what I populated myself.

  • Likes: just upvote a package if you like it. Hit the thumbs up icon.

  • Comments: comments are provided by Disqus. It's very easy to comment with Disqus, you can use your normal account.

    If you're interested in a package, or are an author, you can hit the Subscribe link displayed at the bottom of the page to subscribe to that discussion to get updates of future comments.

Additional to the other metadata, like Github, we make use of the files in your package, so we will also display:

Summary

Stackage just got easier to use:

  • The site is clearer now.
  • The wiki guide is easier.
  • Using Stackage from a sandbox will soon be very easy.
  • You can browse documentation of your snapshot on Stackage via the snapshot (e.g. here).
  • Or you can start from the package list and view an individual package, vote and tag it.
  • It's open source!
Categories: Offsite Blogs