News aggregator

Injective type families for GHC

haskell-cafe - Mon, 02/09/2015 - 5:58pm
Haskellers, I am finishing work on adding injective type families to GHC. I know that in the past many people have asked for this feature. If you have use cases for injective type families I would appreciate if you could show them to me. My implementation has some restrictions and I want to see how severe these restrictions are from a practical point of view. Janek --- Politechnika Łódzka Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata. Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie. _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Categories: Offsite Discussion

4Yr PostDoc with McBride/Ghani

General haskell list - Mon, 02/09/2015 - 5:06pm
Please forward to anyone you think might be interested cheers neil Salary range: £30434 - £34,233 FTE: 1.0 Term: Fixed Term (4 years) Closing date: 23 February 2015 Applications are invited for a Research Associate to work under the supervision of Professor Neil Ghani and Dr Conor McBride on the EPSRC grant "Homotopy Type Theory: Programming and Verification". Homotopy Type Theory (HoTT) is a revolutionary new approach to type theory where types are interpreted as spaces, terms as points and equalities as paths. Decades of research in homotopy theory has uncovered the structure of such paths and HoTT uses this structure as the basis of a new theory of equality. Excitingly, within homotopy theory, one naturally studies higher homotopies of paths between paths and this gives the higher dimensional structure of equality we previously lacked. The objective of this grant is to translate the advances of HoTT into more concrete programming language and verification tools. You will join a team consis
Categories: Incoming News

Cabal: Missing field in record constructionconfigProf

haskell-cafe - Mon, 02/09/2015 - 3:00pm
L.S., I am trying to install a package with the new GHC 7.10 RC 2 (I recompiled cabal.exe with it, and now "cabal update" finally works again). The install fails, even after renaming the cabal config file: Config file path source is default config file. Config file C:\Users\X\AppData\Roaming\cabal\config not found. Writing default configuration to C:\Users\X\AppData\Roaming\cabal\config Resolving dependencies... Configuring random-1.1... cabal: Distribution\Client\Config.hs:(246,37)-(299,9): Missing field in record construction configProf What does this message mean? How can I correct this? Regards, Henk-Jan van Tuyl
Categories: Offsite Discussion

Philip Wadler: Say no to a Scottish national ID system

Planet Haskell - Mon, 02/09/2015 - 2:39pm
The Scottish government has opened for consultation plans of that would lead to database sharing among a vast range of organisations, and could lead to the introduction of de facto ID cards via the back door. Responses to the consultation are due by 25 February 2015. ORG Scotland writes:

A minor, barely noticed consultation is not the way to make a major change to Scottish citizens’ privacy and their relationship with the state. Creating a national ID register was rejected by the SNP and the UK, and the bare minimum should be for the Scottish Executive to introduce primary legislation whereby the public and MSPs can debate the nature of these changes and whether they are acceptable.

Respond to the consultation quickly, courtesy of ORG.

ORG is planning meetings to discuss how we can stop the Scottish Government's plans in EdinburghGlasgow and Aberdeen, and is tracking developments in their blog.
Here is the original consultation,  and a detailed response by ORG.
Categories: Offsite Blogs

Roman Cheplyaka: Dealing with broken Haskell packages

Planet Haskell - Mon, 02/09/2015 - 2:00pm

As we approach the release of GHC 7.10, there is a new wave of Haskell packages that require trivial fixes to build with the new versions of the compiler and standard libraries, but whose authors/maintainers are not around to apply the fixes. This is especially annoying when there is a pull request on GitHub, and all the maintainer would have to do is to press the green Merge button, and upload the new version on hackage.

If you are a responsible maintainer and don’t want this to happen to your packages in the future, you should appoint backup maintainers for your packages.

But what if you are a user of a package that remains broken on hackage, even though a fix is available? Here I review several ways to deal with this problem, including the new and promising Stackage snapshots.

Building the package locally

If all you care about is to get something installed locally (be it the broken package itself, or something that directly or indirectly depends on it), you can install the fixed version locally.

Non-sandboxed way

Check out the repository or branch with the fix, and cabal-install it:

% git clone -b ghc710 https://github.com/markus1189/feed.git % cabal install ./feed

(I prepend ./ to make sure cabal understands that I mean the directory, and not the package name on hackage.)

Sandboxed way

If you’re installing in the sandbox, then you can use add-source (although the non-sandboxed version will work in this case, too):

% git clone -b ghc710 https://github.com/markus1189/feed.git % cabal sandbox add-source feed % cabal install whatever-you-needed

If the package whatever-you-needed has feed among its transitive dependencies, cabal will automatically install it from the add-source’d directory.

Limitations

This approach doesn’t work well if:

  1. You are a maintainer of a package that depends on the broken library. It’s hard to ask your users to check out and build the fixed version by hand.

  2. You work on an in-house application that your coworkers should be able to build, for the same reason.

Forking the package

You cannot upload the fixed version of a package on hackage bypassing the maintainer. However, you can upload it under a new name. This works well if you are a maintainer of a package that directly depends on the broken package, because you can easily make your package depend on your fork.

Examples of this are tasty depending on regex-tdfa-rc (a fork of regex-tdfa) and tasty-golden depending on temporary-rc (a fork of temporary).

Limitations
  1. This doesn’t work well if there’s a chain of dependencies leading from your package to the broken one. You have to either persuade the other maintainer(s) to depend on your fork or fork the entire chain.

  2. If the broken package becomes actively developed again, you need to either move back to using it or backport the bugfixes from it to your fork. (I only fork packages when I find this scenario unlikely.)

  3. Other packages that depend on the broken package won’t automatically get fixed.

  4. Some people get upset when you fork packages.

Stackage snapshots

Instead of uploading the fixed version to hackage (which you can’t), you can upload it to Stackage instead, by creating a custom snapshot.

The procedure is described in Experimental package releases via Stackage Server. You create four files:

  • The fixed tarball (produced by cabal sdist). You probably want to bump the package’s version, so that it doesn’t conflict with the version already on hackage.
  • Two text files: desc and slug. The first one contains a human-readable description of the snapshot; the second contains an id that will become part of the snapshot’s URL.
  • A text file with the packages to be copied directly from hackage. For the purpose of this article, you probably want to leave this file empty. (I don’t know if it’s ok not to include it at all.)

Then you pack these four files into a tarball (that’s right, it’ll be a tarball with a tarball inside) and upload to stackage (after registering, if you haven’t registered before).

The outcome will be a custom hackage-like repository which will contain the single version of a single package — the one you’ve uploaded. (Of course, you can include multiple packages or versions if you like.)

The Stackage website will give you the remote-repo line that you can put into your cabal.config along with the hackage or stackage repos that are already there.

In contrast to building packages locally, you can easily tell your users or coworkers to add that repo as well.

Limitations
  1. If the new hackage release of the broken package will get the same version number as your stackage version, there will be a conflict. (I actually don’t know what happens in that case; my guess is that cabal will silently pick one of the two available versions.)

  2. If the package you maintain (which depends on the broken package) is a small one, or is deep down the dependency chain, it may be hard to tell your users to add the repository. If, on the other hand, you maintain a major web framework or other such thing, it would probably work.

Taking over a package

There’s a procedure for taking over a package described on the wiki. You’ll need to contact the current maintainer; wait an indefinite amount of time (there’s no consensus about it; estimates vary from 2 weeks to 6-12 months); ask again on the mailing list and wait again; finally ask Hackage admins to grant you the rights.

Limitations
  1. Since this procedure takes a long time, it’s almost never sufficient by itself, and you’ll need to resort to one of the other strategies until you’re given the upload rights.

  2. It’s not clear how long you actually need to wait.

  3. I find it odd that you need to jump through all these hoops in order to do a service to the community.

Categories: Offsite Blogs

GHC-7.8.4 for Mac OS X

haskell-cafe - Mon, 02/09/2015 - 1:36pm
Hi, Does anyone plan to put a binary release of GHC-7.8.4 for OS X? There isn't one here: https://www.haskell.org/ghc/download_ghc_7_8_4 Cheers! Ozgur _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe< at >haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Categories: Offsite Discussion

Help me develop tutorials for building Haskell applications.

Haskell on Reddit - Mon, 02/09/2015 - 1:33pm

I started working on Haskell projects (I am a product manager working for FP Complete) with developers and a pattern is emerging which is there isn't enough tutorials and samples teaching how to build an end-to-end Haskell application!

If you have an application that you developed in Haskell, I would love to hear from you and we can together make it into a tutorial to teach others and share the knowledge promoting and encouraging more developers to use Haskell for commercial applications.

Got an app? Let me know!

submitted by saldaoud
[link] [29 comments]
Categories: Incoming News

Use of STM in business?

haskell-cafe - Mon, 02/09/2015 - 1:19pm
From associate of mine: "I am quite curious if anyone actually uses Haskell (esp STM) successfully in business." Of course I can anwser the general question about usage of Haskell in business, but I don't know much about STM specifically. Can anyone provide me with any references to where STM has been used in production in a business setting? Thanks, Tom
Categories: Offsite Discussion

Welcome To LambdaCms-org!

del.icio.us/haskell - Mon, 02/09/2015 - 1:07pm
Categories: Offsite Blogs

Lee Pike: 15 Minutes

Planet Haskell - Mon, 02/09/2015 - 10:31am

Some of the work I lead at Galois was highlighted in the initial story on 60 Minutes last night, a spot interviewing Dan Kaufman at DARPA. I’m Galois’ principal investigator for the HACMS program, focused on building more reliable software for automobiles and aircraft and other embedded systems. The piece provides a nice overview for the general public on why software security matters and what DARPA is doing about it; HACMS is one piece of that story.

I was busy getting married when filming was scheduled, but two of my colleagues (Dylan McNamee and Pat Hickey) appear in brief cameos in the segment (don’t blink!). Good work, folks! I’m proud of my team and the work we’ve accomplished so far.

You can see more details about how we have been building better programming languages for embedded systems and using them to build unpiloted air vehicle software here.


Categories: Offsite Blogs

Yesod Web Framework: The brittle Haskell toolchain

Planet Haskell - Mon, 02/09/2015 - 10:16am

A few weeks ago I received a bug report against streaming-commons. Since then, the details of what we discovered when discussing this report have been bothering me quite a bit, as they expose a lot of the brittleness of the Haskell toolchain. I'm documenting all of these aspects now to make clear how fragile our tooling is, and thereby explain why I think Stackage is so vital to our community.

In this blog post, I'm going to describe six separate problems I've identified when looking into this issue, and explain how Stackage (or some similar deterministic build system) would have protected users against these problems had it been employed.

The story

streaming-commons is a library that provides helper utilities for a number of different streaming concepts, one of them being a streaming way to convert blaze-builder Builders to filled ByteString buffers. Since blaze-builder was released a few years ago, a new set of modules was added to the bytestring package in version 0.10 known as a "bytestring builder." I asked one of the engineers at FP Complete, Emanuel Borsboom, to start working on a new module for streaming-commons to provide similar functionality for bytestring builder.

And now we run into the first problem with the Haskell toolchain. You would think that we should just add a lower bound on bytestring >= 0.10 in the streaming-commons.cabal file. However, setting restrictive lower bounds on ghc-package dependencies can be a problem. Fortunately, Leon Smith already solved this problem for us with bytestring-builder, which provides a compatibility layer for older bytestrings (much like Ed's transformers-compat). The idea is that, when compiled against an older version of bytestring, the bytestring-builder package provides the necessary missing modules, and otherwise does nothing.

When Emanuel wrote his changes to streaming-commons, he added a dependency on bytestring-builder. We then proceeded to test this on multiple versions of GHC via Travis CI and Herbert's multi-ghc-travis. Everything compiled and passed tests, so we shipped the updated version.

However, that original bug report I linked to- reported by Ozgun Ataman- told us there was a problem with GHC 7.6. This was pretty surprising, given that we'd tested on GHC 7.6. Fortunately Lane Seppala discovered the culprit: the Cabal library. It turns out that installing a new version of the Cabal library causes the build of streaming-commons to break, whereas our tests just used the default version of Cabal shipped with GHC 7.6. (We'll get back to why that broke things in a bit.)

After some digging, Emanuel discovered the deeper cause of the problem: Bryan O'Sullivan reported an issue a year ago where- when using a new version of the Cabal library- bytestring-builder does not in fact provide it's compatibility modules. This leads us to our second issue: this known bug existed for almost a year without resolution, and since it only occurs in unusual circumstances, was not detected by any of our automated tooling.

The reason this bug existed though is by far the most worrisome thing I saw in this process: the Cabal library silently changed the semantics of one of its fields in the 1.18 (or 1.20? I'm not sure) release. You see, bytestring-builder was detecting which version of bytestring it was compiled against by inspecting the configConstraints field (you can see the code yourself on Hackage). And starting in Cabal 0.19.1 (a development release), that field was no longer being populated. As a result, as soon as that newer Cabal library was installed, the bytestring-builder package became worse than useless.

As an aside, this points to another problematic aspect of our toolchain: there is no way to specify constraints on dependencies used in custom Setup.hs files. That's actually causes more difficulty than it may sound like, but I'll skip diving into it for now.

The fix for this was relatively simple: use some flag logic in the cabal file instead of a complicated custom Setup.hs file. (Once this pull request was merged in and released, it did fix the original bug report.) But don't take this as a critique of Leon's choice of a complicated Setup.hs file. Because in reality, the flag trick- while the "standard" solution to this problem- broke cabal-install's dependency solver for quite a while. To be fair, I'm still not completely convinced that the bug is fixed, but for now that bug is the lesser of two evils vs the Cabal library bug.

And finally, based on the bug report from Ozgun, it seems like an internal build failed based on all of this occurring. This has been a constant criticism I've made about the way we generally do builds in the Haskell world. Rarely is reproducibility a part of the toolchain. To quote Ozgun:

We are in fact quite careful in dependency management with lower-upper bounds on most outside packages, so breakages like this are unexpected.

And many people feel that this is the way things should be. But as this discussion hopefully emphasizes, just playing with lower and upper bounds is not sufficient to avoid build failures in general. In this case, we're looking at a piece of software that was broken by a change in a library that it didn't depend on, namely Cabal, since our tooling makes an implicit dependency on that library, and we have no way of placing bounds on it.

The case for Stackage

So here are the toolchain problems I've identified above:

  1. Tight coupling between GHC version and some core libraries like bytestring.
  2. A known issue lasting undocumented for a corner case for over a year, without any indication on the Hackage page that we should be concerned.
  3. The Cabal library silently changed the semantics of a field, causing complete breakage of a package.
  4. cabal-install's solver gets confused by standard flag usage, at least in slightly older versions.
  5. Not all dependencies are actually specified in a cabal file. At the very least, the Cabal library version is unconstrained, and any other packages used by Setup.hs.
  6. Default Haskell toolchain doesn't protect us against these kinds of problems, or give us any concept of reproducibility.

Stackage completely solves (2), (3), (5), and (6) for end users. By specifying all library versions used, and then testing all of those versions together, we avoid many possible corner cases of weird library interactions, and provide a fully reproducible build. (Note the Stackage doesn't solve all such cases: operating system, system libraries, executables, etc are still unpinned. That's why FP Complete is working on Docker-based tooling.)

(1) is highly mitigated by Stackage because, even though the tight coupling still exists, Stackage provides a set of packages that take that coupling into account for you, so you're not stuck trying to put the pieces together yourself.

As for (4)... Stackage helps the situation by making the job of the solver simpler by pinning down version numbers. Unfortunately, there are still potential gotchas when encountering solver bugs. Sometimes we end up needing to implement terribly awkward solutions to work around those bugs.

Categories: Offsite Blogs

Does product function uses O(n) space?

Haskell on Reddit - Mon, 02/09/2015 - 10:02am

Hello, I am trying to learn haskell and try to understand the behavior of haskell programs. So I run following code in ghci Prelude> product [1..1000000]

I was expecting this code to use O(1) space. But when I run the code, I saw that ghc is using 250MB of ram to calculate this. Which suggests that space for the whole list is initialized in memory.

I believe this is the definition of product https://github.com/ghc/packages-base/blob/52c0b09036c36f1ed928663abb2f295fd36a88bb/Data/List.hs#L1010-L1021

In the definition, Comment explains that product function can calculate infinite lists. So there should be something that triggers full list initialization that I am not aware of.

I was just wondering what is the reason of that behavior. And how can I do this calculation in constant space.

Also I am using htop to see memory consumption of ghc. Is there a better way of doing this?

Thanks

submitted by yilmazhuseyin
[link] [20 comments]
Categories: Incoming News

Officially supported Android backend, GSoC?

haskell-cafe - Mon, 02/09/2015 - 9:56am
Hi Café For the past few years I have done quite a lot of work in functional languages targeting mobile platforms. I've tried every functional language I could find. GHC targets Android via Neurocyte's (and JoeyH's) backend [1], although, afaik, this is not officially supported. There have been recent discussions on strategies to improve the current state, especially wrt. TH. Nevertheless, the progress is amazing and the backend works very well. I haven't found any major bugs. We've successfully used this at Keera Studios to write multiple games [2,3,4], and I've also written small applications for Google Glass (yes, Haskell works on Glass!). Users are running the games on different Android devices (with different hardware and OS version), and none of them has reported any (ghc-related) bugs so far. Haskell's ability to 'write once, run anywhere' could be a major selling point. Soon Haskell might become one of the very few functional languages that can target any platform, be it web, desktop
Categories: Offsite Discussion

Entirely record based data

Haskell on Reddit - Mon, 02/09/2015 - 7:40am
Idea

I am wondering whether it would be feasible and moreover desirable to have an entirely record based language with row polymorphism. In this language the main mechanism for manipulation of records would be lenses. One can allow pattern matching using a Getter lens and a monoidal Binder type that represents bindings to variables.

With row polymorphism one can infer that (in some imaginary language)

showWithLabel @rec = (rec ^. 'label) <> ": " <> show (rec ^. 'content)

'label is a lens from any record with a label field that focuses on that field

@rec is not an as-pattern, but is actually a lens of the type Getter a (Binder {rec :: a |* rest})

has the type

showWithLabel :: (Show s) => { label :: String, content :: s |* rest} -> String

where

{ label :: String, content :: s |* rest}

asserts that rest is a record with a label field holding a String, a content field holding a type s that satisfies the Show s constraint and maybe some other fields. If rest was omitted, then the record could only contain those two fields. This allows product types with labels to be expressed without prior declaration and values of larger product types with more fields to be applied to the function.

Sum types are usually expressed with

data Either a b = Left a | Right b

There why can't sum types have the same record-like treatment?

newtype Either a b = Either { left :: a, right :: b |+}

There would be no Either constructor, but rather an 'Either Iso

with the + indicating a sum record and * indicating a product record

If we introduce row subtraction, then record prisms can of the form

'x :: Lens' {x :: a |+ rest} (Either {|+ rest - x} a) 'y :: Lens' {y :: a |+ rest} (Either {|+ rest - y} a)

By allowing lenses with Either types to be used as binders in multi-case pattern matching

-- the type of the first case and the actual type -- test :: {x :: Int, y :: String |+} -> String test '@x = "The number is: " <> show x -- the type of the second case -- test :: {y :: String |+} -> String test '@y = "The string is: " <> y --Since the type of '@y subsumes Getter a (Either {|+} b), the pattern matching is exhaustive and no error is given

'@identifier is a pun and in this context desugars to 'identifier . to (bimap id (^. '@identifier)

Use cases
  1. I think this would be especially useful for inversion of control. For example say you have some interpreted language, and you can add hooks to the parser and evaluator for someone else to extend. If that someone attaches a hook to the parser that has returns some new construct in the AST, then it will fail to compile until the new construct is also handled in the evaluator. Perhaps an entire language could be build in this manner, with a project structure consisting of a module for each feature rather than for each stage of processing.

  2. Since datatypes can be inferred, why not have a programmer not ever specify types directly. Rather have types in automatically managed header files. Have notifications when they change with interactive adjustments. If the old type is not subsumed by new type then they can either specify a bridge of the old value in terms of the new values, or mark it as a breaking change.

Disclaimers

I am still very much a Haskell noob. I have no experience in language design or type theory and such, so take this with a grain of salt. I am posting this in the /r/haskell subreddit because it has the readers with the best understanding of the set of concepts I use. Apologies if this is either completely trivial or non-nonsensical.

submitted by reuben364
[link] [8 comments]
Categories: Incoming News

Ian Ross: Non-diffusive atmospheric flow #12: dynamics warm-up

Planet Haskell - Mon, 02/09/2015 - 2:31am
Non-diffusive atmospheric flow #12: dynamics warm-up February 9, 2015

The analysis of preferred flow regimes in the previous article is all very well, and in its way quite illuminating, but it was an entirely static analysis – we didn’t make any use of the fact that the original <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> data we used was a time series, so we couldn’t gain any information about transitions between different states of atmospheric flow. We’ll attempt to remedy that situation now.

What sort of approach can we use to look at the dynamics of changes in patterns of <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics>? Our <semantics>(θ,ϕ)<annotation encoding="application/x-tex">(\theta, \phi)</annotation></semantics> parameterisation of flow patterns seems like a good start, but we need some way to model transitions between different flow states, i.e. between different points on the <semantics>(θ,ϕ)<annotation encoding="application/x-tex">(\theta, \phi)</annotation></semantics> sphere. Each of our original <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> maps corresponds to a point on this sphere, so we might hope that we can some up with a way of looking at trajectories of points in <semantics>(θ,ϕ)<annotation encoding="application/x-tex">(\theta, \phi)</annotation></semantics> space that will give us some insight into the dynamics of atmospheric flow.

Since atmospheric flow clearly has some stochastic element to it, a natural approach to take is to try to use some sort of Markov process to model transitions between flow states. Let me give a very quick overview of how we’re going to do this before getting into the details. In brief, we partition our <semantics>(θ,ϕ)<annotation encoding="application/x-tex">(\theta, \phi)</annotation></semantics> phase space into <semantics>P<annotation encoding="application/x-tex">P</annotation></semantics> components, assign each <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> pattern in our time series to a component of the partition, then count transitions between partition components. In this way, we can construct a matrix <semantics>M<annotation encoding="application/x-tex">M</annotation></semantics> with

<semantics>Mij=Ni→jNtot<annotation encoding="application/x-tex"> M_{ij} = \frac{N_{i \to j}}{N_{\mathrm{tot}}} </annotation></semantics>

where <semantics>Ni→j<annotation encoding="application/x-tex">N_{i \to j}</annotation></semantics> is the number of transitions from partition <semantics>i<annotation encoding="application/x-tex">i</annotation></semantics> to partition <semantics>j<annotation encoding="application/x-tex">j</annotation></semantics> and <semantics>Ntot<annotation encoding="application/x-tex">N_{\mathrm{tot}}</annotation></semantics> is the total number of transitions. We can then use this Markov matrix to answer some questions about the type of dynamics that we have in our data – splitting the Markov matrix into its symmetric and antisymmetric components allows us to respectively look at diffusive (or irreversible) and non-diffusive (or conservative) dynamics.

Before trying to apply these ideas to our <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> data, we’ll look (in the next article) at a very simple Markov matrix calculation by hand to get some understanding of what these concepts really mean. Before that though, we need to take a look at the temporal structure of the <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> data – in particular, if we’re going to model transitions between flow states by a Markov process, we really want uncorrelated samples from the flow, and our daily <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> data is clearly correlated, so we need to do something about that.

Autocorrelation properties

Let’s look at the autocorrelation properties of the PCA projected component time series from our original <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> data. We use the autocorrelation function in the statistics package to calculate and save the autocorrelation for these PCA projected time series. There is one slight wrinkle – because we have multiple winters of data, we want to calculate autocorrelation functions for each winter and average them. We do not want to treat all the data as a single continuous time series, because if we do we’ll be treating the jump from the end of one winter to the beginning of the next as “just another day”, which would be quite wrong. We’ll need to pay attention to this point when we calculate Markov transition matrices too. Here’s the code to calculate the autocorrelation:

npcs, nday, nyear :: Int npcs = 10 nday = 151 nyear = 66 main :: IO () main = do -- Open projected points data file for input. Right innc <- openFile $ workdir </> "z500-pca.nc" let Just ntime = ncDimLength <$> ncDim innc "time" let (Just projvar) = ncVar innc "proj" Right (HMatrix projsin) <- getA innc projvar [0, 0] [ntime, npcs] :: HMatrixRet CDouble -- Split projections into one-year segments. let projsconv = cmap realToFrac projsin :: Matrix Double lens = replicate nyear nday projs = map (takesV lens) $ toColumns projsconv -- Calculate autocorrelation for one-year segment and average. let vsums :: [Vector Double] -> Vector Double vsums = foldl1 (SV.zipWith (+)) fst3 (x, _, _) = x doone :: [Vector Double] -> Vector Double doone ps = SV.map (/ (fromIntegral nyear)) $ vsums $ map (fst3 . autocorrelation) ps autocorrs = fromColumns $ map doone projs -- Generate output file. let outpcdim = NcDim "pc" npcs False outpcvar = NcVar "pc" NcInt [outpcdim] M.empty outlagdim = NcDim "lag" (nday - 1) False outlagvar = NcVar "lag" NcInt [outlagdim] M.empty outautovar = NcVar "autocorr" NcDouble [outpcdim, outlagdim] M.empty outncinfo = emptyNcInfo (workdir </> "autocorrelation.nc") # addNcDim outpcdim # addNcDim outlagdim # addNcVar outpcvar # addNcVar outlagvar # addNcVar outautovar flip (withCreateFile outncinfo) (putStrLn . ("ERROR: " ++) . show) $ \outnc -> do -- Write coordinate variable values. put outnc outpcvar $ (SV.fromList [0..fromIntegral npcs-1] :: SV.Vector CInt) put outnc outlagvar $ (SV.fromList [0..fromIntegral nday-2] :: SV.Vector CInt) put outnc outautovar $ HMatrix $ (cmap realToFrac autocorrs :: Matrix CDouble) return ()

We read in the component time series as a hmatrix matrix, split the matrix into columns (the individual component time series) then split each time series into year-long segments. The we use the autocorrelation function on each segment of each time series (dropping the confidence limit values that the autocorrelation function returns since we’re not so interested in those here) and average across segments of each time series. The result is an autocorrelation function (for lags from zero to <semantics>

Categories: Offsite Blogs