It's been a few weeks since the last news bulletin - this is the result of mostly quietness on behalf of the list and developers, and some sickness on behalf of your editor for several days there. But now there's actually some things to write here!
The past few weeks, GHC HQ has been having some quiet meetings mostly about bugfixes for a 7.10.2 release - as well as noodling about compiler performance. Austin has begun compiling his preliminary notes on the wiki, under the CompilerPerformance page, where we'll be trying to keep track of the ongoing performance story. Hopefully, GHC 7.12.1 will boast a bit better performance numbers.
There are a lot of users who are interested in this particular pain point, so please file tickets and CC yourself on bugs (like #10370), or feel free to help out!7.10.2 status
There's been a bit of chatter about the lists about something on many peoples mind: the release of GHC 7.10.2. Most prominently, Mark Lentczner popped in to ask when the next GHC release will happen - in particular, he'd like to make a Haskell Platform release in lockstep with it (see below for a link to Mark's email).
Until recently, the actual desire for 7.10.2 wasn't totally clear, and at this point, GHC HQ hasn't firmly committed to the 7.10.2 release date. But if milestone:7.10.2 is any indicator, we've already closed over three dozen bugs, several of them high priority - and they keep coming in. So it seems likely people will want these fixes in their hands relatively soon.
Just remember: if you need a fix for 7.10.2, or have a bug you need us to look at, please email the ghc-devs list, file a ticket, and get our attention! Just be sure to set the milestone to 7.10.2.List chatter
- Herbert Valerio Riedel opened an RFC about a regression in GHC 7.10 relating to the update to Unicode 7. Any input from users of international languages or unicode users would be appreciated! https://mail.haskell.org/pipermail/ghc-devs/2015-May/008930.html
- Herbert Valerio Riedel also asked about a new C pre-processor implementation for GHC - but in particular, adopting the extant cpphs into the GHC codebase for this task itself. https://mail.haskell.org/pipermail/ghc-devs/2015-May/008934.html
- Austin Seipp emailed ghc-devs about the HCAR report, for which the GHC entry is due May 17th! Developers should get their edits in quickly. https://mail.haskell.org/pipermail/ghc-devs/2015-May/008939.html
- Niklas Hambüchen announced that he's backported the recent lightweight stack-trace support in GHC HEAD to GHC 7.10 and GHC 7.8 - meaning that users of these stable release can have informative call stack traces, even without profiling! FP Complete was interested in this feature, so they'd probably love to hear user input. https://mail.haskell.org/pipermail/ghc-devs/2015-April/008862.html
- David Terei has written up a proposal on reconciling the existence of Roles with Safe Haskell, which caused us a lot of problems during the 7.8 release cycle. In particular, concerning the ability to break module abstractions and requiring programmers to safeguard abstractions through careful use of roles - and David's written a proposal to address that. https://mail.haskell.org/pipermail/ghc-devs/2015-April/008902.html
- Mark Lentczner started a thread about the 7.10.2 release schedule - because this time, he wants to do a concurrent Haskell Platform release! The thread ended up with a good amount of discussion concerning if 7.10.2 is even needed - but at this rate, it looks like it will ship sometime soon. https://mail.haskell.org/pipermail/ghc-devs/2015-May/008904.html
- Mateusz Kowalczyk posted to ghc-devs hoping to get some help with a tricky, long-standing issue: #4012, which concerns the determinism of GHC binaries. It turns out GHC isn't entirely deterministic when it calculates package IDs, meaning things get really bad when you mix prebuilt binary packages for systems. This in particular has become a real problem for the Nix package manager and users of Haskell applications. Mateusz asks if anyone would be willing to help look into it - and a lot of people would appreciate the help! https://mail.haskell.org/pipermail/ghc-devs/2015-May/008992.html
- Commit f2d1b7fcbbc55e33375a7321222a9f4ee189aa38 - Support unboxing for GADT product types.
- Commit 51af102e5c6c56e0987432aa5a21fe10e24090e9 - Better hints when RTS options are not available.
- Commit 524ddbdad5816f77b7b719cac0671eebd3473616 - Make sure GHC.List.last is memory-efficient.
- Commit a1275a762ec04c1159ae37199b1c8f998a5c5499 - Improve improvement in the constraint solver.
- Commit 4efa421327cf127ebefde59b2eece693e37dc3c6 - Permit empty closed type families.
- Commit 477f514f6ebcf783810da93e2191e4b6ea65559b - rts: add "-no-rtsopts-suggestions" option
- Commit cf7573b8207bbb17c58612f3345e0b17d74cfb58 - More accurate allocation stats for :set +s
- Commit c4e8097ea8dd6e43eae7aadd6bae7e13272ba74d - Bump base version to 188.8.131.52
- Commit 1e8c9b81a819da8eb54405a029fc33a9f5220321 - Enable SMP and GHCi support for AArch64
#10293, #10273, #10021, #10209, #10255, #10326, #9745, #10314, #8928, #8743, #10182, #10281, #10325, #10297, #10292, #10304, #10260, #9204, #10121, #10329, #9920, #10308, #10234, #10356, #10351, #10364, #9564, #10306, #10108, #9581, #10369, #9673, #10288, #10260, #10363, #10315, #10389, #9929, #10384, #10382, #10400, #10256, #10254, #10277, #10299, #10268, #10269, #10280, #10312, #10209, #10109, #10321, #10285, #9895, #10395, #10263, #10293, #10210, #10302, #10206, #9858, #10045, and #9840.
FLOPS 2016, promoting cross-fertilization across the whole declarative programming and theory and practice
FLOPS has been established to promote cooperation between logic and functional programmers, hence the name. This year we have taken the name exceptionally seriously, to cover the whole extent of declarative programming, which also includes program transformation, re-writing, and extracting programs from proofs of their correctness. There is another strong emphasis: on cross-fertilization among people developing theory, writing tools and language systems using that theory, and the users of these tools. We specifically ask the authors to make their papers understandable by the wide audience of declarative programmers and researchers.
As you can see from the Program Committee list, the members have done first-rate theoretic work, and are also known for their languages, tools and libraries. PC will appreciate the good practical work. Incidentally, there is a special category, ``System Descriptions'' that FLOPS has always been known for. We really want to have more submissions in that category.
One can see even on LtU that there is some rift between theoreticians and practitioners: Sean McDermid messages come to mind. He does have many good points. We really hope that FLOPS will help repair this rift.
Whenever I'm reading over a codebase I like to start at the modules with the least in-project dependencies, so I wrote this simple script. Example usage:./explore ~/auto/src 0 Control.Auto.Blip.Internal 1 Control.Auto.Blip 1 Control.Auto.Interval 1 Control.Auto.Serialize 2 Control.Auto.Generate 2 Control.Auto.Process 2 Control.Auto.Run 5 Control.Auto.Effects 5 Control.Auto.Switch 9 Control.Auto.Time 11 Control.Auto.Process.Random 37 Control.Auto
Here, I'd start with Control.Auto.Blip and work my way up the abstraction hierarchy.
Hopefully this is useful to someone else! And if you can make this work for mutually recursive modules... I love you.submitted by MitchellSalad
[link] [7 comments]
We're happy to announce that all users of Haskell packages can now securely download packages. As a tl;dr, here are the changes you need to make:
- Add the relevant GPG key by following the instructions
- Install stackage-update and stackage-install: cabal update && cabal install stackage
- From now on, replace usage of cabal update with stk update --verify --hashes
- From now on, replace usage of cabal install ... with stk install ...
This takes advantage of the all-cabal-hashes repository, which contains cabal files that are modified to contain package hashes and sizes. The way we generate the all-cabal-hashes is interesting in its own right, but I won't shoehorn that discussion into this blog post. Wait for a separate blog post soon for a description of our lightweight architecture for this.
Note that this is an implementation of Mathieu's secure distribution proposal, with some details modified to work with the current state of our tooling (i.e., lack of package hash information from Hackage).How it works
The all-cabal-hashes repository contains all of the cabal files Hackage knows about. These cabal files are tweaked to have a few extra metadata fields, including cryptographic hashes of the package tarball and the size of the package, in bytes. (It also contains the same data in a JSON file, which is what we currently use due to cabal issue #2585.) There is also a tag on the repo, current-hackage, which always points at the latest commit and is GPG signed. (If you're wondering, we use a tag instead of just commit signing since it's easier to verify a tag signature.)
When you run stk update --verify --hashes, it fetches the latest content from that repository, verifies the GPG signature, generates a 00-index.tar file, and places it in the same location that cabal update would place it. At this point, you have a verified package index on your location machine, which contains cryptographic signatures and sizes for each package tarball.
Now, when you run stk install ..., the stackage-install tool handles all downloads for you (subject to some caveats, like cabal issue #2566). stackage-install will look up all of the hashes and sizes that are present in your package index, and verify them during download. In particular:
- If the server tries to send more data than expected, the download stops immediately and an exception is thrown.
- If the server sends less data than expected, an exception is thrown.
- If the hash does not match what was expected, an exception is thrown.
Only when the hash and size match does the file get written. In this way, tarballs are only made available to the rest of your build tools after they have been verified.What about Windows?
In mailing list discussions, some people were concerned about supporting Windows, in particular that Git and GPG may be difficult to install and configure on Windows. But as I shared on Google+ last week, MinGHC will now be shipping with both of those tools. I've tested things myself on Windows with the new versions of MinGHC, stackage-update, and stackage-install, and the instructions above worked without a hitch.
Of course, if others discover problems- either on Windows or elsewhere- please report them so they can be fixed.Speed and reliability
In addition to the security benefits of this tool chain, there are also two other obvious benefits. By downloading the package index updates via Git, we are able to download only the differences since the last time we downloaded. This leads to less bandwidth usage and a quicker download.
This toolchain also replaces connections to Hackage with two high reliability services: Amazon S3 (which holds the package contents) and Github. Using off the shelf, widely used services in place of hosting everything ourself reduces our community burden and increases our ecosystem's reliability.Caveats
There are unfortunately still some caveats with this.
- The biggest hole in the fence is that we have no way of securing distribution of packages from Hackage itself. While all-cabal-hashes downloads the package index from Hackage via HTTPS (avoiding MITM attacks), there are still other attack vectors to be concerned about (such as breaching the Hackage server itself). The improved Hackage security page documents many of these concerns. Ideally, Hackage would be modified to perform package index signing itself.
- Due to cabal issue #2566, it's still possible that cabal-install may download packages for you instead of stackage-install, though these situations should be rare. Hopefully integrating this download code directly with a build tool will eliminate that weakness.
- There is still no verification of package author signatures, so that if someone's Hackage credentials are compromised (which is unfortunately very probable), a corrupted package could be present. This is something Chris Done and Tim Dysinger are working on. We're looking for others in the community to work with us on pushing forward on this. If you're interested, please contact us.
What's great about this toolchain is how shallow it is. All of the heavy lifting is handled by Git, GPG, Amazon S3, Github, and (as you'll see in a later blog post) Travis CI. We mostly just wrap around these high quality tools and services. Not only was this a practical decision (reduce development time and code burden), but also a security decision. Instead of creating a Haskell-only security and distribution framework, we're reusing the same components that are being tried and tested on a daily basis by the greater software community. While this doesn't guarantee the tooling we use is bug free, it does mean that the "many eyeballs" principle applies.
Using preexisting tools also means that we open up the possibility of use cases never before considered. For example, someone contacted me (anonymity preserved) about a use case where he wanted to be able to identify which version of Hackage was being used. Until now, such a concept didn't exist. With a Git-based package index, the Hackage version can be identified by its commit.
I'm sure others will come up with new and innovative tricks to pull off, and I look forward to hearing about them.
Last week I wrote a post on integrating Persistent and Scotty, and received a lot of great feedback. I've been working on integrating the feedback since, and figured I'd post my findings:Part II: The Yak Shavening
In this episode, I discovered why it's critical to keep code compartmentalized. There was some language pragma that was required for Persistent that was conflicting with the scottyT function, and the result was a type error I couldn't understand (and seemingly unGoogleable). I'm planning on posting a complete minimal reproduction to get more insight.Part III: The It Worksening
And in this, I managed to wire up the Reader monad stack, working up my understanding on how these transformer things work.
The Github repository that hosts the code is here: https://github.com/parsonsmatt/scotty-persistent-example
I'd love any suggestions, feedback, etc. I personally found that writing out my learning experience was really helpful for solidifying what i was learning, and I hope that it can be useful for others.submitted by ephrion
[link] [5 comments]