# News aggregator

### Ian Ross: Non-diffusive atmospheric flow #14: Markov matrix calculations

This is going to be the last substantive post of this series (which is probably as much of a relief to you as it is to me…). In this article, we’re going to look at phase space partitioning for our dimension-reduced <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> PCA data and we’re going to calculate Markov transition matrices for our partitions to try to pick out consistent non-diffusive transitions in atmospheric flow regimes.

Phase space partitioningWe need to divide the phase space we’re working in (the unit sphere parameterised by <semantics>θ<annotation encoding="application/x-tex">\theta</annotation></semantics> and <semantics>ϕ<annotation encoding="application/x-tex">\phi</annotation></semantics>) into a partition of equal sized components, to which we’ll assign each data point. We’ll produce partitions by dividing the unit sphere into bands in the <semantics>θ<annotation encoding="application/x-tex">\theta</annotation></semantics> direction, then splitting those bands in the <semantics>ϕ<annotation encoding="application/x-tex">\phi</annotation></semantics> direction as required. The following figures show the four partitions we’re going to use here1:

In each case, the “compartments” of the partition are each of the same area on the unit sphere. For Partitions 1 and 2, we find the angle <semantics>α<annotation encoding="application/x-tex">\alpha</annotation></semantics> of the boundary of the “polar” components by solving the equation

<semantics>∫0αsinθdθ∫02πdϕ=4πC,<annotation encoding="application/x-tex"> \int_0^{\alpha} \sin \theta \, d\theta \int_0^{2\pi} \, d\phi = \frac{4\pi}{C}, </annotation></semantics>

where <semantics>C<annotation encoding="application/x-tex">C</annotation></semantics> is the number of components in the partition. For partition 1, with <semantics>N=4<annotation encoding="application/x-tex">N=4</annotation></semantics>, this gives <semantics>α1=π/3<annotation encoding="application/x-tex">\alpha_1 = \pi/3</annotation></semantics> and for partition 2, with <semantics>N=6<annotation encoding="application/x-tex">N=6</annotation></semantics>, <semantics>α2=cos−1(2/3)<annotation encoding="application/x-tex">\alpha_2 = \cos^{-1} (2/3)</annotation></semantics>.

Assigning points in our time series on the unit sphere to partitions is then done by this code (as usual, the code is in a Gist):

-- Partition component: theta range, phi range. data Component = C { thmin :: Double, thmax :: Double , phmin :: Double, phmax :: Double } deriving Show -- A partition is a list of components that cover the unit sphere. type Partition = [Component] -- Angle for 1-4-1 partition. th4 :: Double th4 = acos $ 2.0 / 3.0 -- Partitions. partitions :: [Partition] partitions = [ [ C 0 (pi/3) 0 (2*pi) , C (pi/3) (2*pi/3) 0 pi , C (pi/3) (2*pi/3) pi (2*pi) , C (2*pi/3) pi 0 (2*pi) ] , [ C 0 th4 0 (2*pi) , C th4 (pi-th4) 0 (pi/2) , C th4 (pi-th4) (pi/2) pi , C th4 (pi-th4) pi (3*pi/2) , C th4 (pi-th4) (3*pi/2) (2*pi) , C (pi-th4) pi 0 (2*pi) ] , [ C 0 (pi/2) 0 pi , C 0 (pi/2) pi (2*pi) , C (pi/2) pi 0 pi , C (pi/2) pi pi (2*pi) ] , [ C 0 (pi/2) (pi/4) (5*pi/4) , C 0 (pi/2) (5*pi/4) (pi/4) , C (pi/2) pi (pi/4) (5*pi/4) , C (pi/2) pi (5*pi/4) (pi/4) ] ] npartitions :: Int npartitions = length partitions -- Convert list of (theta, phi) coordinates to partition component -- numbers for a given partition. convert :: Partition -> [(Double, Double)] -> [Int] convert part pts = map (convOne part) pts where convOne comps (th, ph) = 1 + length (takeWhile not $ map isin comps) where isin (C thmin thmax ph1 ph2) = if ph1 < ph2 then th >= thmin && th < thmax && ph >= ph1 && ph < ph2 else th >= thmin && th < thmax && (ph >= ph1 || ph < ph2)The only thing we need to be careful about is dealing with partitions that extend across the zero of <semantics>ϕ<annotation encoding="application/x-tex">\phi</annotation></semantics>.

What we’re doing here is really another kind of dimensionality reduction. We’ve gone from our original spatial maps of <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> to a continuous reduced dimensionality representation via PCA, truncation of the PCA basis and projection to the unit sphere, and we’re now reducing further to a discrete representation – each <semantics>Z500<annotation encoding="application/x-tex">Z_{500}</annotation></semantics> map in our original time series data is represented by a single integer label giving the partition component in which it lies.

We can now use this discrete data to construct empirical Markov transition matrices.

Markov matrix calculationsOnce we’ve generated the partition time series described in the previous section, calculating the empirical Markov transition matrices is fairly straightforward. We need to be careful to avoid counting transitions from the end of one winter to the beginning of the next, but apart from that little wrinkle, it’s just a matter of counting how many times there’s a transition from partition component <semantics>j<annotation encoding="application/x-tex">j</annotation></semantics> to partition component <semantics>i<annotation encoding="application/x-tex">i</annotation></semantics>, which we call <semantics>Tij<annotation encoding="application/x-tex">T_{ij}</annotation></semantics>. We also need to make sure that we consider the same number, <semantics>Nk<annotation encoding="application/x-tex">N_k</annotation></semantics>, of points from each of the partition components. The listing below shows the function we use to do this – the function takes as arguments the size of the partition and the time series of partition components as a vector, and returns the transition count matrix <semantics>

### FP Complete: Announcing: LTS (Long Term Support) Haskell 2

The Stackage team is proud to announce the release of LTS Haskell 2. To quote the package page:

**LTS Haskell: Version your Ecosystem**

LTS (Long Term Support) Haskell is a curated set of packages which includes non-breaking point releases. It is a companion to Stackage Nightly: whereas Stackage Nightly releases include potentially breaking changes with each new release, LTS Haskell maintains major version stability for a longer period of time.

As usual, to start using LTS Haskell, you typically need to run the command wget https://www.stackage.org/lts/cabal.config in your package directory. More detailed instructions are available on the LTS Haskell 2 page itself.

This release is significant in that it is the first major version bump we've performed on LTS Haskell. I'm also happy to note that, despite some earlier concerns, both primitive 0.6 and blaze-builder 0.4 made it in, thanks to last minute patches by Emanuel Borsboom, Simon Meier, Edward Kmett, and Gregory Collins.

I'm also happy to announce that, in the three months since LTS 1 was released, there has been a significant surge in involvement from the community. For comparison:

MeasurementLTS 1.0LTS 2.0 Core packages2929 Non-core packages8331030 Total packages8621059 Unique maintainers6796I'm excited to see the community embrace this project so fully, and look forward to the trend continuing.

The road to 3.0The current plan is to target the LTS 3.0 release some time around August, depending on when the Hackage ecosystem updates to GHC 7.10 fully. The goal is to make sure the 3.0 is switched over to GHC 7.10.

In addition, Daniel Bergey sent an email which resulted in some questions from me about how we should plan and communicate around LTS major bumps. To summarize my goals and ideas:

- We need to make sure package authors understand when a release is coming out, and the importance of making their packages compatible with upstream dependencies. I believed previously that the issue tracker on the Stackage repo was sufficient to indicate this to authors, but Daniel's questions and other responses I received from package authors tells me that we need some more explicit communication. Perhaps there should be an email 1-2 weeks in advance of the release warning about restrictive upper bounds.
- How strictly should we adhere to a release schedule? I want to make sure that LTS Haskell is a reliable release, but perhaps providing a release window of 1-2 weeks instead of a hard release date will give package authors some necessary flexibility.
- Since Stackage Nightly is essentially the testing ground for new LTS major bumps, how aggressive should we be on removing packages with restrictive upper bounds? I've been pretty lenient until now. However, this is a two-edged sword. Allowing upper bounds to creep in makes the lives of some authors easier, but makes the lives of other authors (the ones updating their packages regularly) much more difficult.

I don't want to make any of these decisions myself, as they're pretty central to how the LTS ecosystem is going to operate. If you have thoughts on any of these points- or on points I haven't raised- please bring them up on the Stackage mailing list and/or Reddit.

### What would I need to know to work with haskell?

Ok, so I'm learning haskell. I know a fair bit of it - I can do the stuff on the 99 haskell problems page, etc etc. Took a university course in haskell, I know how recursion and folds work and stuff. I understand why monads are useful.

But I feel like I'm so, so far from actually making an application with Haskell. When I look at code for applications that actually do exist, I can't understand any of it. And I feel like if I wanted to work with haskell coding, I'd have to actually be able to program applications myself.

So what's the next step? What can I do to actually make something in haskell, and make myself interesting in the market?

submitted by Krexington_III[link] [14 comments]

### cabal haddock --executables

### How to implement a source-sink pattern

### Mark Jason Dominus: Your kids will love a cookie-decorating party

We had a party last week for Toph's 7th birthday, at an indoor rock-climbing gym, same as last year. Last year at least two of the guests showed up and didn't want to climb, so Lorrie asked me to help think of something for them to do if the same thing happened this year. After thinking about it, I decided we should have cookie decorating.

This is easy to set up and kids love it. I baked some plain sugar cookies, bought chocolate, vanilla, and strawberry frosting, several tubes of edible gel, and I mixed up five kinds of colored sugar. We had some colored sprinkles and little gold dragées and things like that. I laid the ingredients out on the table in the gym's side room with some plastic knives and paintbrushes, and the kids who didn't want to climb, or who wanted a break from climbing, decorated cookies. It was a great success. Toph's older sister Katara had hurt her leg, and couldn't climb, so she helped the littler kids with cookies. Even the tiny two-year-old sister of one of the guests was able to participate, and enjoyed playing with the dragées.

(It's easy to vary the project depending on how much trouble you want to take. I made the cookies from scratch, which is pretty easy, but realized later I could have bought prefabricated cookie batter, which would have been even easier. The store sold colored sugar for $3.29 for four ounces, which is offensive, so I went home and made my own. You put one drop of food coloring per two ounces of sugar in a sealed container and shake it up for a minute, for a total cost of close to zero; Toph helped with this. I bought my frosting, but my when my grandmother used to do it she'd make a simple white frosting from confectioners' sugar and then color it with food coloring.)

I was really pleased with the outcome, and not just because the guests liked it, but also because it is a violation of gender norms for a man to plan a cookie-decorating activity and then bake the cookies and prepare the pastel-colored sugar and so forth. (And of course I decorated some cookies myself.) These gender norms are insidious and pervasive, and to my mind no opportunity to interfere with them should be wasted. Messing with the gender norms is setting a good example for the kids and a good example for other dads and for the rest of the world.

I am bisexual, and sometimes I feel that it doesn't affect my life very much. The sexual part is mostly irrelevant now; I fell in love with a woman twenty years ago and married her and now we have kids. I probably won't ever have sex with another man. Whatever! In life you make choices. My life could have swung another way, but it didn't.

But there's one part of being bisexual that has never stopped paying dividends for me, and that is that when I came out as queer, it suddenly became apparent to me that I had abandoned the entire gigantic structure of how men are supposed to behave. And good riddance! This structure belongs in the trash; it completely sucks. So many straight men spend a huge amount of time terrified that other straight men will mock them for being insufficiently manly, or mocking other straight men for not being sufficiently manly. They're constantly wondering "if I do this will the other guys think it's gay?" But I've already ceded that argument. The horse is out of the barn, and I don't have to think about it any more. If people think what I'm doing is gay, that's a pill I swallowed when I came out in 1984. If they say I'm acting gay I'll say "close, but actually, I'm bi, and go choke on a bag of eels, jackass."

You don't have to be queer to opt out of straight-guy bullshit, and I
think I would eventually have done it anyway, but being queer made
opting out unavoidable. When I was first figuring out being queer I
spent a lot of time rethinking my relationship to society and its
gender constructions, and I felt that I was going to have to construct
my own gender from now and that I no longer had the option of taking
the default. I wasn't ever going to follow Rule Number One of Being a
Man (“do not under any circumstances touch, look at, mention, or think
about any dick other than your own”), so what rules *was* I going to
follow? Whenever someone tried to pull “men don't” on me, (or whenever
I tried to pull it on myself) I'd immediately think of all the Rule
Number One stuff I did that “men don't” and it would all go in the
same trash bin. Where (did I say this already?) it belongs.

Opting out frees up a lot of mental energy that I might otherwise
waste worrying about what other people think of stuff that is none of
their business, leaving me more space to think about how *I* feel
about it and whether I think it's morally or ethically right and
whether it's what I want. It means that if someone is puzzled or
startled by my pink sneakers, I don't have to care, except I might
congratulate myself a little for making them think about gender
construction for a moment. Or the same if people find out I have a
favorite flower (CROCUSES YEAH!) or if I wash the dishes or if I play
with my daughters or watch the ‘wrong’ TV programs or cry or apologize
for something I did wrong or whatever bullshit they're uncomfortable
about this time.

Opting out frees me up to be a feminist; I don't have to worry that a bunch of men think I'm betraying The Team, because I was never on their lousy team in the first place.

And it frees me up to bake cookies for my kid's birthday party, to
make a lot of little kids happy, and to know that that can only add
to, not subtract from, my identity. I'm Dominus, who loves programming
*and* mathematics *and* practicing the piano *and* playing with toy
octopuses *and* decorating cookies with a bunch of delightful girls.

This doesn't have to be a big deal. Nobody is likely to be shocked or even much startled by Dad baking cookies. But these tiny actions, chipping away at these vile rules, are one way we take tiny steps toward a better world. Every kid at that party will know, if they didn't before, that men can and do decorate cookies.

And perhaps I can give someone else courage to ignore some of that same bullshit that prevents all of us from being as great as we could and should be, all those rules about stuff men aren't supposed to do and other stuff women aren't supposed to do, that make everyone less. I decided about twenty years ago that that was the best reason for coming out at all. People are afraid to be different. If I can be different, maybe I can give other people courage and comfort when they need to be different too. As a smart guy once said, you can be a light to the world, like a city on a hilltop that cannot be hidden.

And to anyone who doesn't like it, I say:

### Ideas for an open source Finance or Math Haskell library

I want to contribute to an useful Finance or Math Haskell library, or start a new one. Do you haskellers have ideas on how I could make a meaningful contribution on those areas?

submitted by marcus_monteiro[link] [3 comments]