# News aggregator

### Ketil Malde: Can you trust science?

Hardly a week goes by without newspaper writing about new and exciting results from science. Perhaps scientists have discovered a new wonderful drug for cancer treatment, or maybe they have found a physiological cause for CFS. Or perhaps this time they finally proved that homeopathy works? And in spite of these bold announcements, we still don't seem to have cured cancer. Science is supposed to be the method which enables us to answer questions about how the world works, but one could be forgiven for wondering whether it, in fact, works at all.

As my latest contribution to my local journal club, I presented a paper by Ioannidis, titled *Why most published research findings are false* 1. This created something of a stir when it was published in 2005, because it points out some simple mathematical reasons why science isn't as accurate as we would like to believe.

Science is about finding out what is true. For instance, is there a relationship between treatment with some drug and the progress of some disease - or is there not? There are several ways to go about finding out, but in essence, it boils down to making some measurements, and doing some statistical calculations. Usually, the result will be reported along with a *p*-value, which is a by-product of the statistical calculations saying something about how certain we are of the results.

Specifically, if we claim there is a relationship, the associated *p*-value is the probability we would make such a claim *even if there is no relationship in reality*.

We would like this probability to be low, of course, and since we usually are free to select the *p*-value threshold, it is usually chosen to be 0.05 (or 0.01), meaning that if the claim is false, we will only accept it 5% (or 1%) of the times.

Now, the *p*-value is often interpreted as the probability of our (positive) claim being wrong. *This is incorrect!* There is a subtle difference here, which it is important to be aware of. What you must realize, is that the probability *α* relies on the *assumption that the hypothesis is wrong* - which may or may not be true, we don't know (which is precisely why we want to find out).

The probability of a claim being wrong after the fact is called the positive predictive value (PPV). In order to say something about this, we also need to take into account the probability of claiming there exists a relationship when the claim is true. Our methods aren't perfect, and even if a claim is true, we might not have sufficient evidence to say for sure.

So, take one step back and looking at our options. Our hypothesis (e.g., drug X works against disease Y) can be true or false. In either case, our experiment and analysis can lead us to reject or accept it with some probability. This gives us the following 2-by-2 table:

True False Accept 1-*β*

*α*Reject

*β*1-

*α*

Here, *α* is the probability of accepting a false relationship by accident (i.e., the *p*-value), and *β* is the probability of missing a true relationship -- we reject a hypothesis, even when it is true.

To see why *β* matters, consider a hypothetical really really poor method, which has **no** chance of identifying a true relationship, in other words, $\beta$=1. Then, **every** accepted hypothesis must come from the **False** column, as long as *α* is at all positive. Even if the *p*-value threshold only accepts 1 in 20 false relationships, that's all you will get, and as such, they constitute 100% of the accepted relationships.

But looking at *β* is not sufficient either. Let's say a team of researchers test hundreds of hypotheses, which all happen to be false? Then again, some of them will get accepted anyway (sneaking in under the *p*-value threshold *α*), and since there are no hypotheses in the **True** column, again **every** positive claim is false.

A *β* of 1 or a field of research with 100% false hypotheses are extreme cases2, and in reality, things are not quite so terrible. The Economist had a good article with a nice illustration showing how this might work in practice with more reasonable numbers. It should still be clear that the ratio of true to false hypotheses being tested, as well as the power of the analysis to identify true hypotheses are important factors. And if these numbers approach their limits, things can get quite bad enough.

Other factors also influence the PPV. Try as we might to be objective, scientists often try hard to find a relationship -- that's what you can publish, after all3. Perhaps in combination with a less firm grasp of statistics than one could wish for (and scientists who think they know enough statistics are few and far between - I'm certainly no exception there), this introduces bias towards acceptance.

Multiple teams pursuing the same challenges in a hot and rapidly developing field also decrease the chance of results being correct, and there's a whole cottage industry of scientist reporting spectacular and surprising results in high-ranking journals, followed by a trickle of failures to replicate.

Solving thisOne option is to be stricter - this is the default when you do multiple hypothesis testing, you require a lower *p*-value threshold in order to reduce *α*. The problem with this is that if you are stricter with what you accept as true, you will also reject more actually true hypotheses. In other words, you can reduce *α*, but only at the cost of increasing *β*.

On the other hand, you can reduce *β* by running a larger experiment. One obvious problem with this is cost, for many problems, a cohort of a hundred thousand or more is necessary, and not everybody can afford to run that kind of studies. Perhaps even worse, a large cohort means that almost any systematic difference will be found significant. Biases that normally are negligible will show up as glowing bonfires in your data.

Modern biology has changed a lot in recent years, and today we are routinely using high-throughput methods to test the expression of tens of thousands of genes, or the value of hundreds of thousands of genetic markers.

In other words, we simultaneously test an extreme number of hypotheses, where we expect a vast majority of them to be false, and in many cases, the effect size and the cohort are both small. It's often a new and exciting field, and we usually strive to use the latest version of the latest technology, always looking for new and improved analysis tools.

To put it bluntly, it is extremely unlikely that any result from this kind of study will be correct. Some people will claim these methods are still good for "hypothesis generation", but Ioannidis shows a hypothetical example where a positive result increases the likelihood that a hypothesis is correct by 50%. This doesn't sound so bad, perhaps, but in reality, the likelihood is only improved from 1 in 10000 to 1 in 7000 or so. I guess three thousand fewer trials to run in the lab is *something*, but you're still going to spend the rest of your life running the remaining ones.

You might expect scientists to be on guard for this kind of thing, and I think most scientists will claim they desire to publish correct results. But what counts for your career is publications and citations, and incorrect results are no less publishable than correct ones - and might even get cited more, as people fail to replicate them. And as you climb the academic ladder, publications in high-ranking journals is what counts, an for that you need spectacular results. And it is much easier to get spectacular incorrect results than spectacular correct ones. So the academic system rewards and encourages bad science.

ConsequencesThe bottom line is to be skeptical of any reported scientific results. The ability of the experiment and analysis to discover true relationships is critical, and one should always ask what the effect size is, and what the statistical power -- the probability of detecting a real effect -- is.

In addition, the prior probability of the hypothesis being true is crucial. Apparently-solid, empirical evidence of people getting cancer from cell phone radiation, or working homeopathic treatment of disease can almost be dismissed out of hand - there simply is no probable explanation for how that would work.

A third thing to look out for, is how well studied a problem is, and how the results add up. For health effects of GMO foods, there is a large body of scientific publications, and an overwhelming majority of them find no ill effects. If this was really dangerous, wouldn't some of these investigations show it conclusively? For other things, like the decline of honey bees, or the cause of CFS, there is a large body of contradictory material. Again - if there was a simple explanation, wouldn't we know it by now?

And since you ask: No, the irony of substantiating this claim with a scientific paper is not lost on me.↩

Actually, I would suggest that research in paranormal phenomena is such a field. They still manage to publish rigorous scientific works, see this Less Wrong article for a really interesting take.↩

I think the problem is not so much that you can't publish a result claiming no effect, but that you can rarely claim it with any confidence. Most likely, you just didn't design your study well enough to tell.↩

### Confusion regarding the differences between ByteString types

Hello /r/haskell. Currently I'm working on a server program that needs to parse incoming data from sockets. I'm a bit confused as to which data types I should be using. I believe I need to use Strict ByteStrings, and some of the incoming data might be unicode.

Basically, I parse the first N bytes following an established protocol, and a payload that might represent valid unicode follows. From what I understand, it makes sense to interpret the socket data as a ByteString, and convert the relevant unicode portion to Text.

When I hGet the data from my socket handle, which ByteString should I be choosing?

Data.ByteString(ByteString) Data.ByteString.Lazy(ByteString) Data.ByteString.Char8 (ByteString) ... etc.In addition, what difference will there be between strict and lazy bytestrings, and under what situation should I choose one or the other? Some libraries return lazy bytestrings when I am using strict ones, and vice-versa.

Thank you in advance for helping clear my confusion.

submitted by pythonista_barista[link] [10 comments]

### Haskell Platform Reboot Proposal

### Where does GHC spend most of it's time during compilation?

I'm just wondering what contributing factors result in long compile times for GHC (with the exception of Template Haskell). Is it things like type checking & analysis, is it the codegen process, or whatever.

submitted by gaymenonaboat[link] [15 comments]

### FP Complete: FP Complete's Hackage mirror

We have been running a mirror of Hackage package repository which we use internally for the FP Complete Haskell Centre's IDE, building Stackage, and other purposes. This has been an open secret, but now we're making it official.

To use it, replace the remote-repo line in your ~/.cabal/config with the following:

remote-repo: hackage.fpcomplete.com:http://hackage.fpcomplete.com/Then run cabal update, and you're all set.

This mirror is updated every half-hour. It is statically hosted on Amazon S3 so downtime should be very rare (Amazon claims 99.99% availability).

The mirror does not include the HTML documentation. However, Stackage hosts documentation for a large set of packages.

We have also released our hackage-mirror tool. It takes care of efficiently updating a static mirror of Hackage on S3, should anyone wish to host their own. While the official hackage-server has its own support for mirroring, our tool differs in that it does not require running a hackage-server process to host the mirror.

HTTPS for StackageOn a tangentially related note, we have enabled TLS for www.stackage.org. Since cabal-install does not support TLS at this time, we have *not* set up an automatic redirect from insecure connections to the https:// URL.

### citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.7733&rep=rep1&type=pdf

### citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.7733&rep=rep1&type=pdf

### What are your most persuasive examples of using Quickcheck?

I'm writing documentation for my Python Quickcheck-like library, Hypothesis and I'm looking for examples of using Quickcheck that are a little more persuasive than reversing a list or checking commutativity of numbers.

In particular I'm looking for examples that make people go "Oh, I could totally use this in $DAYJOB". I find most quickcheck examples seem to start from the assumption that you're writing a library, which most people aren't.

The examples I have so far are:

- The classic encoding/decoding example
- Getting date arithmetic wrong by thinking a year is always 365 days
- Demonstrating a topological sort is broken
- Testing timezone conversions
- Exhibiting a classic voting paradox (I can't really claim this one is practical but it was too cool not to include)
- Fuzzing an HTTP API

But I'd really like more, particularly ones from domains where you wouldn't necessarily think to have used Quickcheck, or ones with a certain "wow!" factor.

submitted by DRMacIver[link] [45 comments]

### Command Line Args passed to wxHaskell

### Wright Access - The Haskell gl package

### Wright Access - The Haskell gl package

### attoparsec and (binary|cereal)

Hi there, so I'm a beginner (don't blame :D) trying to write a parser for apache kafka log files. Those files are written in binary format containing messages with the following structure:

8 byte message offset number 4 byte size containing an integer N 4 byte CRC32 of the message 1 byte "magic" identifier to allow format changes, value is 2 currently 1 byte "attributes" identifier 4 byte key length, containing length K (K byte key) -- doesn't exist in our case for the moment 4 byte payload length, containing length V V byte payloadSince performance matters, I made a first attempt with attoparsec library which seems suitable at this place to me. Now I'm facing 2 problems: **laziness** and **error handling**.

For my questions, showing there the first two fields will be enough. Here is my first attempt with the *take N* function:

So first of all I have to decode the taken bytestrings into Int64 for offset field and Int32 for the length field, which relates to my first question. I tried to use cereal for lazy decoding. This didn't work because o <- take won't return a lazy bytestring. Is there any way to take those X bytes lazy? Is it even relevant whether its lazy or not...?

In a second step I tried to decode with binary library which seems to work fine. But now I'm facing the problem with error handling since decode returns *Either String a*.

What would be the appropriate way to handle errors in this scenario? Would it be appropriate to create some type, lets say Either String Offset, and pass this one layer up to my entryParser. Then again distinguish Left/Right and so forth? This doesn't seems really intuitive to me...

So any hints are appreciated a lot :)

submitted by mjnet[link] [6 comments]

### Ken T Takusagawa: [zyxbhqnd] Defining monads with do notation

If one happens to be most comfortable with "do" notation for monads ("What are monads? They are the things for which "do" notation works well."), so monads implicitly being defined in terms of bind (>>=) and return, here are the definitions of map and join, the "other" way of defining monads:

join :: (Monad m) => m (m a) -> m a;

join xs = do { x <- xs ; x }

map :: (Monad m) => (a -> b) -> m a -> m b;

map f xs = do { x <- xs ; return $ f x }

map is identical to liftM and slightly narrower than fmap which requires only the Functor typeclass instead of Monad. This redundancy is one of the motivations for the AMP proposal. Incidentally, map (as defined above) would work as well as the famous Prelude map function which operates only on lists, because a list is a Monad (and a Functor).

Just for completeness, here is bind defined in do notation:

(>>=) :: (Monad m) => m a -> (a -> m b) -> m b;

(>>=) xs f = do { x <- xs ; f x }

I sometimes like explicitly using the bind operator instead of do notation because the syntax, specifying xs then f, lines up well with the thought process "first prepare the input xs to a function, then call the function f on it". It also works well for longer chains. For example, the expression xs >>= f >>= g >>= h is equivalent to

do {

x <- xs;

y <- f x;

z <- g y;

h z;

}

but not having to name the intermediate results.

Inspired by the tutorial Monads as containers.

Update: added type signature for (>>=).

### cabal test and tests that overlap in covarage

### CFP: Erlang Workshop 2015

### The GHC Team: GHC Weekly News - 2015/03/24

Hi *,

It's time for the GHC weekly news. We've had an absence of the last one, mostly due to a lot of hustle to try and get 7.10 out the door (more on that shortly throughout this post). But now we're back and everything seems to be taken care of.

This week, in the wake of the GHC 7.10 release (which is occuring EOD, hopefully), GHC HQ met up for a brief chat and caught up:

- This week GHC HQ met for only a very short time to discuss the pending release - it looks like all the blocking bugs have been fixed, and we've got everything triaged appropriately. You'll hopefully see the 7.10 announcement shortly after reading this.

We've also had small amounts of list activity (the past 6-8 weeks have been very, very quiet it seems):

- Your editor sent out a call for developers to fill information in on the Status page about what they plan to do. If you're working on something, please add it there! https://mail.haskell.org/pipermail/ghc-devs/2015-March/008661.html

- Herbert Valerio Riedel asked about a possible regression regarding identifiers containing unicode subscripts - but nobody has replied! https://mail.haskell.org/pipermail/ghc-devs/2015-March/008503.html

- Doug Burke chimed in as a new contributor and wrote down some notes on what it took to write his first patch and submit it to us - and we really appreciate the feedback, Doug! https://mail.haskell.org/pipermail/ghc-devs/2015-March/008526.html

- Yitzchak Gale revived a thread he started a while back, which puttered out: bootstrapping GHC 7.8 with GHC 7.10. The long and short of it is, it should just about work - although we still haven't committed to this policy, it looks like Yitz and some others are quite adamant about it. https://mail.haskell.org/pipermail/ghc-devs/2015-March/008531.html

- Neil Mitchell uncovered a nasty bug in GHC 7.10.1 RC3, submitted it to us. He also wrote a fantastic blog post explaining the issue. And it was promply diagnosed, fixed, and taken care of by our own Joachim Breitner. Thank you for the fast response Joachim and Neil! https://mail.haskell.org/pipermail/ghc-devs/2015-March/008532.html

- Mark Lentczner has announced Alpha releases of the Haskell Platform 2015.2.0.0, containing GHC 7.10.1 RC3: https://mail.haskell.org/pipermail/ghc-devs/2015-March/008597.html

- Simon Peyton Jones asks: what's the current state about having simultaneous installations of a package? Simon is a bit confused as to why this is still a problem when we have all the tools to solve it, it looks like! (But read on for more): https://mail.haskell.org/pipermail/ghc-devs/2015-March/008602.html

- Michael Snoyman asks: can we get a new feature patch in GHC 7.10.2? The answer seems to be an unfortunate 'no', but with some tips, Michael may end up backporting the changes from HEAD to GHC 7.10 himself. https://mail.haskell.org/pipermail/ghc-devs/2015-March/008612.html

Some noteworthy commits that went into ghc.git in the past two weeks include:

- Commit 71fcc4c096ec0b575522e4c2d0104ef7a71a13c5 - GHC defaults to using the gold linker on ARM/Android and ARM/Linux targets.
- Commit 9dfdd16a61e79cb03c633d442190a81fe5c0b6b8 - Bump ghc-prim to version 0.4.0.0.
- Commit 42448e3757f25735a0a5b5e2b7ee456b5e8b0039 - GHC HEAD now always looks for LLVM 3.6 specifically.

Closed tickets this past week include: #9122, #10099, #10081, #9886, #9722, #9619, #9920, #9691, #8976, #9873, #9541, #9619, #9799, #9823, #10156, #1820, #6079, #9056, #9963, #10164, #10138, #10166, #10115, #9921, #9873, #9956, #9609, #7191, #10165, #10011, #8379, #10177, #9261, #10176, #10151, #9839, #8078, #8727, #9849, #10146, #9194, #10158, #7788, #9554, #8550, #10079, #10139, #10180, #10181, #10170, #10186, #10038, #10164, and #8976.