This page mentions using an SMT solver for type-level naturals, but I've never SMT solvers discussed in the context of sums and products.
Either and (,) form commutative monoids withProductMonoid = (*, (,), ()) SumMonoid = (*, Either, Void)
but as far as the type checker is concerned, the type equalities ((a,b), c) ~ (a, (b,c)), and Either (Either a b) c ~ Either a (Either b c) don't hold. If they did, we'd have a really elegant way of constructing extensible data types, and we'd save a whole bunch fiddling around with constructors.
This is just blue-sky thinking on my part, but has anybody else thought of this? Has anyone ever tried to implement it? Are there any serious impediments? I imagine the runtime memory layout would be an issue, especially in the presence of commutativity.submitted by ForTheFunctionGod
[link] [6 comments]
I recently wrote my first serious Haskell program (a bot of sorts). It was also a big foray into learning Monad Transformers. With the exception of a handful of pure functions, the code mostly looks imperative. It's a series of sequential API calls executed throughEitherT e (StateT s (ReaderT h IO)) a
The bot works in a loop within IterT. It combines an mplus check of an MVar for a termination command along with a forever of the primary function.
Even though the code is largely strict and executed sequentially, I realised that the transformer stack lets me reason about my code in a way I never could in a language like Java or Python.
- When I wanted to encapsulate non-termination, I did it in an IterT
- When I needed to include a global constant (HTTP handler) I included it in a ReaderT
- When I needed to keep track of what the bot was doing I included it in a StateT
Then when I wanted to execute two API calls concurrently/asynchronously my transformer stack forced me to consider some things - what do I do about these two states that exist simultaneously? How do I handle an exception in only one thread? What do I do about the logs being generated?
Now when I write imperative code I lack those abstractions. If I make an asynchronous request there's no compiler telling me that I have just created two states. There's no pattern matching an Either for my exceptions. Where as each Monad in the stack extends me some imperative functionality. In that way each one explicitly tells me what functionality I've added.
But the way I think about imperative code has changed. It's no longer just a bunch of mutable objects being thrown around. I explicitly think about what it's doing. I often write out Haskell style type signatures in pseudocode in my planning. I separate out "pure" functions that don't mutate anything. I separate out different types of mutations based on what it is they are actually doing.
Funnily enough I could have written my bot faster in NodeJS, Python or even Java. I would have been done a lot sooner but I wouldn't have felt as secure. I wanted something that would crash gracefully, that would recover from errors and reverse actions it was in the middle of.
Haskell is fast becoming my favourite imperative language.submitted by TheCriticalSkeptic
[link] [54 comments]
WordPress was nice for some time. There were lots of nice themes to choose from, installation was easy, things mostly worked. However, the need to keep updating it with security patches, and the need to keep sifting through comment spam, became tiresome.
I always wish I had time to work on something in Haskell, so I decided to put the need for a new blog platform together with that: I’ve switched to Hakyll, at least for now.
This means I generate the site statically, as HTML pages. This means my site requires very little compute power to run, and it’s even easier for me to administer than the WP site was. It also means I don’t have support for comments, which is sad but saves me lots of time sifting through spam. My contact information is on the contact page, please reach out directly if you have comments. Maybe I’ll add some third-party discussion site pointers at a future date. Let me know if you have opinions about that.
Creating the site was fairly simple and took about a day, and took the following steps:
- Download and install stack (not cabal). This is the build tool we’ll use to fetch and install hakyll and its dependencies. Stack is now the preferred build tool for Haskell. They have binaries available for lots of platforms.
- Follow the hakyll installation tutorial, substituting “stack” for “cabal”.
Now you have a (very) bare bones site. I then:
- Added Bootstrap. I basically copied in the HTML from the template and modified it as needed to make it work with Hakyll.
- Added RSS and Atom feeds using the snapshots tutorial
- Changed the landing page to show teasers rather than full articles or just links. The teaser tutorial was very helpful for that. Definitely read this after getting the RSS feeds working, it will make more sense.
- Other minor things like adding favicons
All in all, quite a bit simpler than I expected. The fact that it was so easy to use stack instead of cabal was a nice surprise along the way. The tutorials for Hakyll are really helpful, though there are some details missing, so be patient.
It turns out not to need as much actual Haskell programming time as I expected, but maybe when I need to make something fancier I’ll get to work on an actual Haskell project.
I’m looking for a way to implement event-driven systems without having to deal with callbacks nor FRP. The former is way too imperative to me and the later way too experimental.
I’ve implemented a future Haskell package to abstract over callbacks from C systems, but I’d like to find another way to abstract reaction so that I can do things a more haskellish way, you know.
Any idea?submitted by _skp
[link] [16 comments]
Well-Typed is very happy to announce the first alpha release of the Hackage Security library, along with integration into both cabal-install and the Hackage server and a tool for managing file-based secure repositories. This release is not yet ready for general use, but we would like to invite interested parties to download and experiment with the library and its integration. We expect a beta release running on the central Hackage server will soon follow.
Hackage Security and related infrastructure is a project funded by the Industrial Haskell Group to secure Hackage, the central Haskell package server. A direct consequence of this work is that we can have untrusted Hackage mirrors (mirror selection is directly supported by the library). A minor but important additional side goal is support for incremental updates of the central Hackage index (only downloading information about new packages, rather than all packages).
TL;DR: Hackage will be more secure, more reliable and faster, and cabal update should generally finish in seconds.Security architecture
Security is notoriously difficult to get right, so rather than concocting something ad-hoc the Hackage Security framework is based on TUF, The Update Framework. TUF is a collaboration between security researches at the University of Arizona, the University of Berkeley and the Univerity of Washington, as well as various developers of the Tor project. It is a theory specifically designed for securing software update systems, and suits the needs of Hackage perfectly.
TUF covers both index signing and author signing. Index signing provides the means to verify that something we downloaded from a mirror is the same as what is available from the central server (along with some other security properties), thus making it possible to set up untrusted mirrors. It does not however deal with compromise of the central server. Author signing allows package authors to sign packages, providing a guarantee that the package you download is the one that the author uploaded. These two concerns are largely orthogonal, and the current project only adds support for index signing. Author signing will be the subject of a later project.
Very briefly, here is how it works. The bits in red refer to new features added as part of the Hackage Security work.
Hackage provides a file 00-index.tar.gz (known as “the index”) which contains the .cabal files for all versions of all packages on the server. It is this file that cabal update downloads, and it is this file that cabal install uses to find out which packages are available and what their dependencies are.Hackage additionally provides a signed file /snapshot.json (“the snapshot”), containing a hash of the index. When cabal downloads the index it computes its hash and verifies it against the hash recorded in the snapshot. Since mirrors do not have the key necessary to sign the snapshot (“ the snapshot key”), if the hash matches we know that the index we downloaded, and hence all files within, was the same as on the central server.
When you cabal install one or more packages, the index provides cabal with a list of packages and their dependencies. However, the index does not contain the packages themselves.The index does however contain package.json files for each version of each package, containing a hash for the .tar.gz of that package version. Since these files live in the index they are automatically protected by the snapshot key. When cabal downloads a package it computes the hash of the package and compares it against the hash recorded in the index; if it matches, we are guaranteed that the package is the same as the package on the central server, as the central server is the only one with access to the snapshot key.
- The client does not have built-in knowledge of the snapshot key. Instead, it can download /root.json (“the root metadata”) from the server, which contains the public snapshot key. The root metadata itself is signed by the root keys, of which the client does have built-in knowledge. The private root keys must be kept very securely (e.g. encrypted and offline).
This description leaves out lots of details, but the purpose of this blog post is not to give a full overview of TUF. See the initial announcement or the website of The Update Framework for more details on TUF; the Hackage Security project README provides a very detailed discussion on how our implemention of TUF relates to the official TUF specification.Software architecture
Most of the functionality is provided through a new library called hackage-security, available from github, designed to be used by clients and servers alike.Client-side
Although we have integrated it in cabal-install, the hackage-security library is expressly designed to be useable by different clients as well. For example, it generalizes over the library to use for HTTP requests; cabal uses hackage-security-HTTP, a thin layer around the HTTP library. However, if a client such as stack wants to use the hackage-security library to talk to Hackage it may prefer to use hackage-security-http-client instead, a thin layer around the http-client library.
Using the library is very simple. After importing Hackage.Security.Client three functions become available, corresponding to points 1, 2 and 3 above:checkForUpdates :: Repository -> CheckExpiry -> IO HasUpdates downloadPackage :: Repository -> PackageId -> (TempPath -> IO a) -> IO a bootstrap :: Repository -> [KeyId] -> KeyThreshold -> IO ()
- A Repository is an object describing a (local or remote) repository.
- The CheckExpiry argument describes whether we should check expiry dates on metadata. Expiry dates are important to prevent attacks where a malicious mirror provides outdated data (see A Look In the Mirror: Attacks on Package Managers, Section 3, Threat Model) but we may occassionally want to accept expired data (for instance, when the central server is down for an extended period of time).
The [KeyId] and KeyThreshold arguments to bootstrap represent the client’s “built-in” knowledge of the root keys we alluded to above. In the case of cabal-install these come from the cabal config file, which may contain a section such asremote-repo hackage.haskell.org url: http://hackage.haskell.org/ secure: True root-keys: 2ae741f4c4a5f70ed6e6c48762e0d7a493d8dd265e9cbc6c4037dfc7ceaec70e 32d3db5b4403935c0baf52a2bcb05031784a971ee2d43587288776f2e90609db eed36d2bb15f94628221cde558e99c4e1ad36fd243fe3748e1ee7ad00eb9d628 key-threshold: 2
(this syntax for specifying repositories in the cabal config is new.)
We have written an example client that demonstrates how to use this API; the example client supports both local and remote repositories and can use HTTP, http-client or curl, and yet is only just over 100 lines of code.Server-side
It is important to realize that servers need not be running the Hackage software; mirrors of the central Hackage server may (and typically will) be simple HTTP file servers, and indeed company-internal package servers may choose not to use the Hackage software at all, using only file servers. We provide a hackage-security utility for managing such file-based repositories; see below for details.Trying it out
There are various ways in which you can try out this alpha release, depending on what precisely you are interested in.
Resources at a glancehackage-security library github tag “pre-release-alpha” cabal-install github branch “using-hackage-security” hackage github branch “using-hackage-security” Secure Hackage snapshots
We provide two almost-complete secure (but mostly static) Hackage snapshots, located athttp://hackage.haskell.org/security-alpha/mirror1 http://hackage.haskell.org/security-alpha/mirror2
If you want to use cabal to talk to these repositories, you will need to download and build it from the using-hackage-security branch. Then change your cabal config and add a new section:remote-repo alpha-snapshot url: http://hackage.haskell.org/security-alpha/mirror1 secure: True root-keys: 89e692e45b53b575f79a02f82fe47359b0b577dec23b45d846d6e734ffcc887a dc4b6619e8ea2a0b72cad89a3803382f6acc8beda758be51660b5ce7c15e535b 1035a452fd3ada87956f9e77595cfd5e41446781d7ba9ff9e58b94488ac0bad7 key-threshold: 2
It suffices to point cabal to either mirror; TUF and the hackage-security library provide built-in support for providing clients with a list of mirrors. During the first check for updates cabal will download this list, and then use either mirror thereafter. Note that if you wish you can set the key-threshold to 0 and not specify any root keys; if you do this, the initial download of the root information will not be verified, but all access will be secure after that.
These mirrors are almost complete, because the first mirror has an intentional problem: the latest version of generics-sop does not match its signed hash (simulating an evil attempt from an attacker to replace the generics-sop library with DoublyEvil-0.3.142). If you attempt to cabal get this library cabal should notice something is amiss on this mirror, and automatically try again from the second mirror (which has not been “compromised”):# cabal get generics-sop Downloading generics-sop-0.1.1.2... Selected mirror http://hackage.haskell.org/security-alpha/mirror1 Downloading package generics-sop-0.1.1.2 Exception Invalid hash for .../generics-sop-0.1.1.2.tar45887.gz when using mirror http://hackage.haskell.org/security-alpha/mirror1 Selected mirror http://hackage.haskell.org/security-alpha/mirror2 Downloading package generics-sop-0.1.1.2 Unpacking to generics-sop-0.1.1.2/
(It is also possible to use the example client to talk to these mirrors, or indeed to a secure repo of your own.)Setting up a secure file-based repo
If you want to experiment with setting up your own secure repository, the easiest way to do this is to set up a file based repository using the hackage-security utility. A file based repository (as opposed to one running the actual Hackage software) is much easier to set up and will suffice for many purposes.
Create a directory ~/my-secure-repo containing a single subdirectory ~/my-secure-repo/package. Put whatever packages you want to make available from your repo in this subdirectory. At this point your repository might look like~/my-secure-repo/package/basic-sop-0.1.0.5.tar.gz ~/my-secure-repo/package/generics-sop-0.1.1.1.tar.gz ~/my-secure-repo/package/generics-sop-0.1.1.2.tar.gz ~/my-secure-repo/package/json-sop-0.1.0.4.tar.gz ~/my-secure-repo/package/lens-sop-0.1.0.2.tar.gz ~/my-secure-repo/package/pretty-sop-0.1.0.1.tar.gz
(because obviously the generics-sop packages are the first things to come to mind when thinking about which packages are important to secure.) Note the flat directory structure: different packages and different versions of those packages all live in the one directory.
Create public and private keys:# hackage-security create-keys --keys ~/my-private-keys
This will create a directory structure such as~/my-private-keys/mirrors/id01.private ~/my-private-keys/mirrors/.. ~/my-private-keys/root/id04.private ~/my-private-keys/root/.. ~/my-private-keys/snapshot/id07.private ~/my-private-keys/target/id08.private ~/my-private-keys/target/.. ~/my-private-keys/timestamp/id11.private
containing keys for all the various TUF roles (proper key management is not part of this alpha release).
Note that these keys are stored outside of the repository proper.
Create the initial TUF metadata and construct an index using# hackage-security bootstrap \ --repo ~/my-secure-repo \ --keys ~/my-private-keys
This will create a directory ~/my-secure-repo/index containing the .cabal files (extracted from the package tarballs) and TUF metadata for all packages~/my-secure-repo/index/basic-sop/0.1.0.5/basic-sop.cabal ~/my-secure-repo/index/basic-sop/0.1.0.5/package.json ~/my-secure-repo/index/generics-sop/0.1.1.1/generics-sop.cabal ~/my-secure-repo/index/generics-sop/0.1.1.1/package.json ...
and package the contents of that directory up as the index tarball ~/my-secure-repo/00-index.tar.gz; it will also create the top-level metadata files~/my-secure-repo/mirrors.json ~/my-secure-repo/root.json ~/my-secure-repo/snapshot.json ~/my-secure-repo/timestamp.json
The timestamp and snapshot are valid for three days, so you will need to resign these files regularly using# hackage-security update \ --repo ~/my-secure-repo \ --keys ~/my-private-keys
You can use the same command whenever you add any additional packages to your repository.
If you now make this directory available (for instance, by pointing Apache at it) you should be able to use cabal to access it, in the same way as described above for accessing the secure Hackage snapshots. You can either set key-threshold to 0, or else copy in the root key IDs from the generated root.json file.
If you are feeling adventurous you can also try to set up your own secure Hackage server. You will need to build Hackage from the using-secure-hackage branch.
You will need to create a subdirectory TUF inside Hackage’s datafiles/ directory, containing 4 files:datafiles/TUF/mirrors.json datafiles/TUF/root.json datafiles/TUF/snapshot.private datafiles/TUF/timestamp.private
containing the list of mirrors, the root metadata, and the private snapshot and timestamp keys. You can create these files using the hackage-security utility:
- Use the create-keys as described above to create a directory with keys for all roles, and then copy over the snapshot and timestamp keys to the TUF directory.
- Use the create-root and create-mirrors commands to create the root and mirrors metadata. The create-mirrors accepts an arbitrary list of mirrors to be added to the mirrors metadata, should you wish to do so.
Note that the root.json and mirrors.json files are served as-is by Hackage, they are not used internally; the snapshot and timestamp keys are of course used to sign the snapshot and the timestamp.
Once you have created and added these files, everything else should Just Work(™). When you start up your server it will create TUF metadata for any existing packages you may have (if you are migrating an existing database). It will create a snapshot and a timestamp file; create metadata for any new packages you upload and update the snapshot and timestamp; and resign the snapshot and timestamp nightly. You can talk to the repository using cabal as above.
If you a have Hackage server containing a lot of packages (a full mirror of the central Hackage server, for instance) then migration will be slow; it takes approximately an hour to compute hashes for all packages on Hackage. If this would lead to unacceptable downtime you can use the precompute-fileinfo tool to precompute hashes for all packages, given a recent backup. Copy the file created by this tool to datafiles/TUF/md5-to-sha256.map before doing the migration. If all hashes are precomputed migration only takes a few minutes for a full Hackage snapshot.Integrating hackage-security
If you want to experiment with integrating the hackage-security library into your own software, the example client is probably the best starting point for integration in client software, and the hackage-security utility is probably a good starting point for integration in server software.Bugs
Please report any bugs or comments you may have as GitHub issues.Roadmap
This is an alpha release, intended for testing by people with a close interest in the Hackage Security work. The issue tracker contains a list of issues to be resolved before the beta release, at which point we will make the security features available on the central Hackage server and make a patched cabal available in a more convenient manner. Note that the changes to Hackage are entirely backwards compatible, so existing clients will not be affected.
Hi guys, I'm trying to read/write a socket without having to close the handle. However, it seems like whatever I do, I cannot read the socket contents until the handle is closed.
Client code:import Network.Socket import Network.BSD import Control.Monad import System.IO as IO import Data.Text as T import Data.ByteString.Lazy as BSL import Data.MessagePack as MP main = withSocketsDo $ do sock <- socket AF_INET Stream 0 setSocketOption sock ReuseAddr 1 addr <- liftM hostAddresses $ getHostByName "localhost" connect sock $ SockAddrInet (fromInteger 8585) (Prelude.head addr) handle <- socketToHandle sock ReadWriteMode hSetBuffering handle NoBuffering replicateM_ 5 $ BSL.hPut handle $ MP.pack ("Hello host" :: Text) hFlush handle getLine replicateM_ 5 $ BSL.hPut handle $ MP.pack ("Hello host" :: Text) hFlush handle hClose handle
Server code:import Network.Socket import Network.Socket.ByteString as NSB import Network.Socket.ByteString.Lazy as NSBL import Data.MessagePack as MP import Data.Text as T import Data.ByteString.Lazy as BSL import Data.ByteString as BS import Data.HashMap as HM import System.IO as IO runServer :: Integer -> IO () runServer port = withSocketsDo $ do sock <- socket AF_INET Stream 0 setSocketOption sock ReuseAddr 1 bindSocket sock (SockAddrInet (fromInteger port) iNADDR_ANY) listen sock 100 servLoop sock servLoop :: Socket -> IO () servLoop sock = do connection <- accept sock onConnect connection servLoop sock onConnect :: (Socket, SockAddr) -> IO () onConnect (client, client_addr) = do IO.putStrLn "Got a connection" h <- socketToHandle client ReadWriteMode hSetBuffering h NoBuffering req <- BSL.hGet h 1024 IO.putStrLn "Got some contents: " IO.putStrLn $ show req
So the observed behavior is that the linereq <- BSL.hGet h 1024
doesn't evaluate until AFTER the client closes the handle. Is there a reason for that? and can I just have it continually stream data without closing the handle?submitted by fuzzyslippers42
[link] [8 comments]
Can any one tell me how to start learning Haskell? Any series of tutorials i tired but unable to find quality tutorials. That is might be because Haskell is not a popular language like others.submitted by mu_livecodingtv
[link] [5 comments]
Typically, when people explain monads, they begin with (>>=) and how it is used in desugaring the do notation. However, I've always found this very confusing because the data flows in the opposite direction from normal function application. Even LYAH only mentions (>>=).
I believe it is a good way to explain the different kinds of function applications by showing the following progression:
For a normal functionincByOne :: Int -> Int incByOne = (+1)
we apply it to the value twice as eitherincByOne (incByOne 1) > 3
or using ($)incByOne $ incByOne $ 1 > 3
We can apply the function to a value in functor context using (<$>)incByOne <$> incByOne <$> Just 1 > Just 3
EDIT. As rightly pointed out by /u/imz , the above should be written asincByOne <$> (incByOne <$> Just 1)
because (<$>) is left associative. See stack overflow question. Basically, my original example really is (incByOne <$> incByOne) <$> Just 1 and it only worked because -> is a functor.
Finally, we create a monadic versionincByOneM :: Int -> Maybe Int incByOneM n = if (n > 0) then Just $ n + 1 else Nothing
and we can apply it to the value in a monadic contextincByOneM =<< incByOneM =<< Just 1 > Just 3 incByOneM =<< incByOneM =<< Just (-1) > Nothing
Furhermore, we can mix the monadic and the non-monadic version as follows:incByOneM =<< incByOne <$> Just 1 > Just 3 incByOne <$> (incByOneM =<< Just 1) > Just 3
Only after that explanation, I believe it is instructive to introduce (>>=) by saying that it inverses the dataflow (from left to right) as follows:Just 1 >>= incByOneM >>= incByOneM > Just 3
Comments? Do you find it useful? Any existing resources using this style of explanation?
EDIT2: It seems the conclusion from the discussion is that supporting dataflow from left-to-right and right-to-left equally well is needed. To make all examples from this question equally well expressible using the left-to-right way, we can use the & operator:incByOne $ incByOne $ 1 1 & incByOne & incByOne
However, there's no standard operator for values in the functor context. Probably a good analogue could be <&>:incByOne <$> (incByOne <$> Just 1) (Just 1 <&> incByOne) <&> incByOne
EDIT5: /u/Peaker pointed out that &, <&>, and >>= are all left associative, so no parentheses needed forJust 1 <&> incByOne <&> incByOne
And, for values in monadic context we haveincByOneM =<< incByOneM =<< Just 1 Just 1 >>= incByOneM >>= incByOneM
So, the question is: has anybody considered adding something like <&> to base?
EDIT4: both (&) and (<&>) are defined in Control.Lens.Operators but it would be really great to have them both in base.EDIT5: CONCLUSION.
Thanks everybody for your insights. I've learned quite a lot here. Here's my summary after applying both styles (left-to-right and right-to-left) in my code.Using $, <$>, and =<< + applicative
This was my style. These operators shine in two cases:
[+] with datatype constructors (and applicative style)Person <$> getFirstName <*> getLastName
because it keeps the same shape as the datatype:data Person = Person String String
[+] point free stylelet f = incByOne $ incByOne
instead oflet f n = incByOne $ incByOne n
However, problems arise due to mixed infixity, since <$> is left associative whereas $ and =<< are right associative.
[-] requires careful placement of bracketsincByOne <$> (incByOne <$> Just 1) Using &, <&>, and >>=
These really shine in two cases:
[+] creating a pipe through which data flows left-to-right (which follows the direction of -> in type definition)1 & Just <&> incByOne >>= incByOneM
no brackets needed here since all of them are left-associative and have the same infixity (1).
[+] applying multiple modifications to the same object. This is particularly nice with lenses:Person "Alic" "Bck" & name .~ "Alice" & lastName .~ "Black"
[-] have to import Control.Lens ((&), (<&>))
[-] cannot use them point-free stylesubmitted by mallai
[link] [49 comments]