General News

Haskell Weekly News: August 23, 2005

Submitted by jgoerzen on Tue, 08/23/2005 - 5:32am.

Greetings, and thanks for reading the fourth issue of HWN, a weekly newsletter for the Haskell community. Each Tuesday, new editions will be posted (as text) to the Haskell mailing list and (as HTML) to The Haskell Sequence.

New Releases

  • ghc-src 0.2.0. Lemmih announced ghc-src. ghc-src is a Haskell parser with full support for every GHC extension. It is based on the GHC source and is meant as a replacement for haskell-src-exts, though it could of course have other ueses. ghc-src is available via a Darcs repository.
  • Cairo binding. Not a formal release, but great progress is being made on the binding to the Cairo vector graphics toolkit, including some working code.

Discussion

Category theory monads. Cale Gibbard began a discussion comparing the monads from category theory with the implementation of monads in Haskell. Michael Vanier suggested some of Phil Wadler's papers on monads. Michael went on to say that Haskell monads are very similar to those from category theory.

More on FFI and callbacks. The thread on FFI and callbacks was revived this week. Among other things, Duncan Coutts noted that there is no easy way to do a really correct binding to wxWidgets from Haskell when Haskell programs are multithreaded, due to limitations in Haskell's threading model.

Oracle on Haskell. Brian Strand asked about using Oracle on Haskell, and more generally, about the suitability of Haskell for database programming. Alistair Bayley mentioned that takusen has Oracle support. John Goerzen suggested using HSQL's ODBC support, with unixODBC on *nix platforms. He went on to say that HSQL has been used in production environments. Krasimir Angelov, author of HSQL, added that he's been wanting to add Oracle support to HSQL for awhile. Finally, Brian Strand later followed up and said that takusen has been working well for him so far.

Pros and cons of static typing. Keean Schupke revived an earlier discussion about the pros and cons of static typing and side effects.

Static typing and interactivity. On a similar note, Ketil Malde wrote about not being able to load modules with type errors into ghci. Bernard Pope suggested the type debugger in Chameleon.

Argument ordering. A thread about the order of arguments to functions entertained many different viewpoints this week. Too many to really summarize here.

Decoupling and encapsulation. Terrence Brannon wrote about decoupling program elements from presentation, as in HTML generators. He included a link to his document describing architectural flaws in Perl's HTML::Mason.

Future of The Monad.Reader. Shae Erisson wrote on #haskell today that he is looking for someone to either take over, or help with, editing The Monad.Reader, Haskell's monthly online magazine. Anyone that would like to help should contact him.

How is HWN material found? I've received some questions this week about how I find material for HWN. The most obvious way is if someone sends it to me; see the link at the bottom of each HWN for contribution information. Other than that, I read the main Haskell mailing lists, the Haskell Sequence, and IRC looking for things to write about. I prefer to have stories linked in at least one of these places before covering them in HWN, since it gives readers a convenient place to follow discussion.

Haskell Toolchain

Cabal design. Frederik Eaton started another discussion about Cabal, this time focusing on run time vs. configure time issues, the usage of custom package.conf files, and multiple cabal files in one package. Isaac Jones commented on most of those items. Duncan Coutts suggested the ability to register a package "in place".

Darcs Corner

Colordiff. Dmitriy Morozov asked about using colordiff with darcs. Timo Savola suggested a shell function to accomplish this.

Linus and git. Juliusz Chroboczek posted a link to a discussion about Git, and mentioned that it looks like they're re-inventing some Darcs features.

Success with trac. Pedro Melo posted about his success using trac (a SourceForge-like system) with Darcs.

Quotes of the Week

Seen on #haskell today...

<tuomov> define drug <Itkovian> anything that gets you addicted to <Itkovian> potentially <Itkovian> and that messes with yr senses <Lemmih> Haskell?

About Haskell Weekly News

Want to continue reading HWN? Please help us create new editions of this newsletter. Please see the contributing information, or send stories to hwn -at- complete -dot- org. There is also a Darcs repository available.

Haskell Weekly News: August 16, 2005

Submitted by jgoerzen on Tue, 08/16/2005 - 5:43am.

Haskell Weekly News: August 16, 2005

Greetings, and thanks for reading the third issue of HWN, a weekly newsletter for the Haskell community. Each Tuesday, new editions will be posted (as text) to the Haskell mailing list and (as HTML) to The Haskell Sequence.

New Releases

  • gtk2hs 0.9.9. Axel Simon announced the latest version of this binding to GTK, primarily containing bugfixes.
  • h4sh. Donald Bruce Stewart announced the new Haskell for shell scripts package. It exposes the Haskell Data.List library for use in shell scripting.
  • c2hs 0.14.3. Manuel M. T. Chakravarty released version 0.14.3 of c2hs. Improvements over 0.14.1 include support for cross-compilation, gcc's asm construct, better support for hierarchical module syntax, and new name translation functions.
  • magic-haskell. John Goerzen announced the availability of magic-haskell, a binding to C's libmagic. With it, you can determine the type of a file by looking at its contents rather than its name.
  • AVL 2.2. Adrian Hey released AVL 2.2. It introduces new set manipulation functions, a new type of zipper, and optimizations for functions that don't modify a tree.

Discussion

HWN format changes this week. Based on reader feedback, the New Releases section has been moved to the top of HWN. Please let me know what you think. Before you ask, the Darcs and Quotes of the Week sections are missing because there was no news for them this week.

Static typing. Minh Thu started a discussion about the pros and cons of static typing and side effects.

Language version pragmas. Bulat Ziganshin proposed a way to specify what language version and options are used in Haskell sources.

About Haskell Weekly News

Want to continue reading HWN? Please help us create new editions of this newsletter. Please see the contributing information, or send stories to hwn -at- complete -dot- org. There is also a Darcs repository available.

Haskell Weekly News: August 9, 2005

Submitted by jgoerzen on Tue, 08/09/2005 - 5:36am.

Greetings, and thanks for reading the second issue of HWN, a weekly newsletter for the Haskell community. Each Tuesday, new editions will be posted (as text) to the Haskell mailing list and (as HTML) to The Haskell Sequence.

Discussion

Practical Monads. Paul Moore started a discussion about Monads and resources for learning about them. Quite a few readers responded with suggestions.

STRef vs. IORef. Srinivas Nedunuri started a discussion by asking when to STRef and when to use IORef. Iavor Diatchki posted a helpful example, and many other helpful answers were posted as well.

Parsing Foreign Languages. The The ParsingForeignLanguagesInHaskell wiki page was the subject of a short discussion on the libraries mailing list. If you have any further information or would like to join or start a project to parse a particular language, see the wiki page.

Haskell Toolchain

Cabal was again a hot topic this week. There were discussions about data directories, running on Windows 98, and package description fields in general.

Darcs Corner

Darcs in FreshMeat. David Roundy is looking for volunteers to maintain the Darcs entry at FreshMeat.net. It wouldn't require much time, but the ability to summarize changes at release time.

Binary files and line endings. Phil Brooke asked how darcs handles line endings and binary files.

Uniqueness of patch names. On #darcs this week, a discussion about the uniqueness of low-level patch names in darcs. The consensus seemed to be that darcs needs an additional better-than-1-second component to patch names to eliminate a situation in which collisions can arise.

New Releases

  • Simon Marlow announced the release of Haddock version 0.7. Highlights of this version include improvements for linking across different packages, bug fixes, collapsable trees in HTML, and support for new output formats.
  • Einar Karttunen has released hsgnutls 0.1, a Haskell binding for the GnuTLS SSL/TLS library.
  • John Goerzen announced the release of a preliminary, but working, binding to OpenLDAP from Haskell.

Quotes of the Week

<CosmicRay> "Oh Lord, bless this thy holy IO monad, and use it for thy purposes that it may smash Java to tiny bits..." (with apologies to monty python)

<Pseudonym> If I ever write a GUI library for Haskell, I'm going to call it pointlesstif.

About Haskell Weekly News

Want to continue reading HWN? Please help us create new editions of this newsletter. Please see the contributing information, or send stories to hwn -at- complete -dot- org. There is also a Darcs repository available.

Haskell Weekly News: August 2, 2005

Submitted by jgoerzen on Tue, 08/02/2005 - 4:40am.

Greetings, and thanks for reading the first issue of HWN, a weekly newsletter for the Haskell community. HWN is an experiment inspired by Debian Weekly News and Linux Weekly News. Each Tuesday, new editions will be posted (as text) to the Haskell mailing list and (as HTML) to The Haskell Sequence.

Since this is the first issue, it covers a few items more than one week old.

Discussion

Updating the Haskell Standard? This question was posed on haskell-cafe and reaction was mixed.

Best way to assemble strings? Andy Gimblett inquired about building up strings. The discussion covered options such as printf, (++), concat, and even some sample code for interpolation inside strings.

FFI, Threading, and Callbacks. John Goerzen asked some questions about using FFI together with threading. Simon Marlow has written a paper on the topic that is useful background. Duncan Coutts described why some GUI toolkits presently do polling.

Haskell Toolchain

GHC 6.4.1 release candidate is available. Simon Marlow has announced the availability of GHC 6.4.1 release candidate and the beginning of testing for 6.4.1. 6.4.1 includes many fixes, including some performance enhancements, and also introduces support for a native code generator for amd64.

Results of GHC Performance Week. Simon Marlow posted a summary of the results of the GHC performance week. They found a number of things that improve the performance of GHC, and some are already fixed in 6.4.1.

Cabal was a hot topic this week. Brian Smith started a discussion about conditional code in Cabal. It seems to be a common problem when porting software to Windows. Duncan Couts asked about automated platform building of Haskell packages based on their Cabal descriptions.

GHC in Debian unstable. Due to a C++ transition going on, GHC is currently uninstallable in Debian unstable. If you want to use it on unstable, you can grab the libgmp3 package from stable. More details in Debian bug 319222.

Conferences

The 2005 Haskell Workshop is coming up on September 30 in Tallin, Estonia. David Roundy, author of darcs, will be a feature presenter this year. More information is available from the conference page.

Darcs Corner

Darcs 1.0.4pre2 released. David Roundy announced the availability of Darcs 1.0.4pre2. Major updates since 1.0.3 include reduced memory usage, and experimental support for git archives.

darcsweb. Alberto Bertogli announced darcsweb, a replcement for darcs.cgi modeled after gitweb.

Darcs Success Story. Mark Stosberg wrote about a success using Darcs for just-in-time branching.

Darcs on SourceForge. Eric S. Johansson wondered if any SourceForge-like Darcs-friendly sites existed. Thomas Zander suggested simply using public web space on SourceForge itself.

Centralized development with Darcs. A question was raised about using Darcs for centralized development in a specific scenario. Several solutions were mentioned. Remko Troncon linked to a recipe for centralized logging on the Darcs wiki. Mark Stosberg pointed out his article, Benefits from a real world switch from CVS to Darcs, and also pointed out the RSS support in Darcs.

New Releases

  • hsffig, a new FFI binding generator, was announced by Dimitry Golubovsky. Download via its Darcs repository. The main unique feature of hsffig is that it can parse C .h files without any human assistance whatsoever. Version 1.0 was also announced just yesterday.
  • c2hs version 0.14.1 is out. It has a new parser system and its build system is now based upon Cabal.
  • MissingH 0.11.3 is out, and now supports Windows. MissingH is a library of pure-Haskell utility functions relating to strings, logging, and I/O. Darcs repository also available.
  • MissingH LGPL/BSD branch was announced. This branch is a stripped-down version of MissingH, with all GPL'd code either re-licensed or removed. It is available from a Darcs repository only.

Quotes of the Week

<Speck> "That's like cheating. It isn't even programming. You just tell it what to do and it does it." -- My friend upon seeing some Haskell code

<autrijus> Perl: "Easy things are easy, hard things are possible"
<autrijus> Haskell: "Hard things are easy, the impossible just happened"

About Haskell Weekly News

Want to continue reading HWN? Please help us create new editions of this newsletter. Please see the contributing information, or send stories to hwn -at- complete -dot- org. There is also a Darcs repository available.

Contributing to Haskell Weekly News

Submitted by jgoerzen on Mon, 08/01/2005 - 4:14am.

Thank you for your interest in Haskell Weekly News.

We welcome and encourage contributed news items from the Haskell community.

Basics
The simplest way to contribute is to submit stories for HWN. You may e-mail your stories to hwn .at. complete .dot. org. To make things easiest for us to process (which makes it more likely that your submission gets processed quickly), here are some hints:

  • Include links to whatever you are talking about. Readers need to have a place to go to read more about it.
  • Keep it short and sweet. HWN gives people a summary of what's happening, and links to more information.
  • Plain text or hand-edited HTML is preferred. Don't send us HTML from FrontPage.

Mailing List Links
HWN presently uses Gmane.org for links into Haskell mailing list discussions. If you send us links into the mailing list, we prefer that you link to the appropriate items on Gmane. If not, you can link to them in the haskell.org archives and we'll find the corresponding gmane posts ourselves. But please at least link to them *somewhere*.

Darcs Repository
If you want to be really helpful, you can check out our darcs repository and send us patches directly. This will be the absolute quickest way to get something into HWN, especially if you do a good job :-)

Check it out with:

darcs get --partial http://darcs.complete.org/hwn

You should always make all changes against the file prep.html in that repository. Other files may be in various stages of being prepared for release. Entries in prep.html will be moved around if necessary.

When you're ready to submit your story, just run darcs send.

Netcat Tutorial

Submitted by Adam Palmer on Sat, 06/11/2005 - 2:20pm.

Netcat Tutorial
A LearnSecurityOnline.com Article By Adam Palmer

http://www.learnsecurityonline.com © Learn Security Online, Inc. 2004-2005

Contents

Introduction 3
Netcat Syntax 4
Netcat Installation 6
Simple File Transfer 7
Tar 9
Simple socket reply 10
inetd 11
talking to syslogd -r 12
Internetworking Basics 13
nc-e 14
Scanning 15
Spoofing 16
Simple response service 17
Advanced Uses 18
Windows Command Shell 19
Unauthorized Proxying 20
Cryptcat 20
Final Thoughts 21
Command cheat sheet 22

Introduction

What is Netcat?

"Netcat is a simple Unix utility which reads and writes data across network connections, using TCP or UDP protocol. It is designed to be a reliable "back-end" tool that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and exploration tool, since it can create almost any kind of connection you would need and has several interesting built-in capabilities. Netcat, or "nc" as the actual program is named, should have been supplied long ago as
another one of those cryptic but standard Unix tools." Taken from the README of the netcat source tree, this description sums up the uses of netcat perfectly.

Netcats homepage is: http://netcat.sourceforge.net

Throughout this tutorial, I will be giving examples on Linux systems. The official Netcat homepage makes no reference to Windows systems, however I have successfully built Netcat from source under Cygwin, and you can find a Win32 copy built by @Stake from:

http://www.atstake.com/research/tools/network_utilities/nc11nt.zip and all examples used
below are fully supported under Windows.

http://www.learnsecurityonline.com © Learn Security Online, Inc. 2004-2005

Netcat Syntax

adam@adamp:~$nc -h

[v1.10]
connect to

somewhere: nc [-options] hostname

port[s] [ports] ..
.
listen for
inbound: nc -l -p port [-options]
[hostname] [port]

options:
-e prog programto exec after connect [dangerous!!]
-b allow

broadcasts
-g gateway source-routing hop point[s], up to 8
-G num source-routing

pointer: 4, 8, 12, ...
-h this
cruft
-i secs delayinterval for lines sent, ports scanned
-l listen
mode, for inbound connects
-n numeric-onlyIP addresses, no DNS
-o file hex
dump of traffic
-p port localport number
-r randomize
local and remote ports
-q secs quitafter EOF on stdin and delay of secs
-s addr local
source address
-t answer
TELNET negotiation
-u UDP
mode

-v verbose
[use twice to be more verbose]

-w secs timeout
for connects and final net reads

-z zero-I/
O
mode [used for scanning]

port
numbers can be individual or ranges: lo-hi [inclusive]

http://www.learnsecurityonline.com © Learn Security Online, Inc. 2004-2005

Netcat Installation

I will cover here three installation methods.

a) On a debian or similar machine:

apt-get install netcat will do the trick:

adamp:~# apt-get install netcatReading Package Lists... DoneBuilding Dependency Tree... Done

The following NEW packages will be installed:

netcat
0 packages upgraded, 1 newly installed, 0 toremove and 0 not upgraded.
Need to get 63.3kB of archives. After unpacking190kB will be used.
Get:1 http://http.us.debian.org stable/mainnetcat 1.10-21 [63.3kB]
Fetched 63.3kB in 2s (27.9kB/s)
Selecting previously deselected package netcat.
(Readingdatabase ... 39433 files and directories currently installed.)
Unpacking netcat (from.../netcat_1.10-21_i386.deb) ...
Setting up netcat (1.10-21) ...
adamp:~#

b) And for those that prefer RPMs:

rpm Uvh netcat-version.rpm

c) And for those that prefer the source:

We will start by wgeting the source:

wget http://osdn.dl.sourceforge.net/sourceforge/netcat/netcat-0.7.1.tar.gz

We will now untar, cd to the directory we have untarred the source codeto, and run the configure script.

adam@adamp:~$ tar -xzf netcat-0.7.1.tar.gzadam@adamp:~$ cd netcat-0.7.1adam@adamp:~/netcat-0.7.1$ ./configure

The configure script should run through with no trouble, as netcat has very few dependencies.
We then run make:

adam@adamp:~/netcat-0.7.1$ make

This will run through and will compile your source, which again should complete simply and successfully. You
can then run make install if you have the necessary privileges, or you could simply run src/netcat which will have
been built after a successful make. At this point, you should now have a successful build of netcat
somewhere on your system.

http://www.learnsecurityonline.com © Learn Security Online, Inc. 2004-2005

Simple File Transfer

So as an example, I will start two copies of netcat on the same machine locally:

adam@adamp:~$ netcat -l -p 1111

Here, using the -l switch, we are able to specify that netcat should go into listen mode i.e. to listen on
the specified port. Using p 1111 we are able to specify that we are using port 1111. To summarize,
netcat will sit and listen for TCP connections on port 1111 and print any data it receives out to the
screen. In another window we start netcat as:

adam@adamp:~$ netcat 127.0.0.1 1111

This will connect to host 127.0.0.1 (Locally) on port 1111.
We are now able to have a full two way data transmission, in Window 1:

adam@adamp:~$ netcat -l -p 1111This message was typed in WINDOW1This message was typed in WINDOW2Now I'm going to end communication with ^C (Ctrl-C)
adam@adamp:~$

And in Window 2:

adam@adamp:~$ netcat 127.0.0.1 1111This message was typed in WINDOW1This message was typed in WINDOW2Now I'm going to end communication with ^C (Ctrl-C)
adam@adamp:~$

This is the most basic use of netcat described. Here, we are using a BASH shell, and thus we may pipe |
data to and from netcat, as well as using the redirection (>, >>, <, <<) to allow netcat to integrate into
the shell environment. We will now examine using netcat with one of the redirection operators. Lets
say we wanted to simply transmit a plaintext file. In one window, we will start netcat as:

adam@adamp:~$ netcat -l -p 1111 > outputfile

This will run netcat with the same parameters specified above, except it will redirect
all text received into outputfile.

adam@adamp:~$ echo > infile << EOF
> This is a test file.
> I am going to attempt to transmit this.
> Using Netcat.
> EOF
adam@adamp:~
$

Here, we have created some text in a file, and this is the file we are going to attempt to transmit:

adam@adamp:~$ cat infile | netcat 127.0.0.1 1111 q 10adam@adamp:~$

http://www.learnsecurityonline.com © Learn Security Online, Inc. 2004-2005

Hopefully this has now been transmitted to the otherside:
adam@adamp:~$ cat outputfile
This is a test file.

I am going to attempt to transmit this.
Using Netcat.
adam@adamp:~
$

And here we can confirm that it has. The -q 10 in the command line will quit after EOF (Otherwise
netcat will hang waiting for more input for cat and we will have to terminate it manually). The
parameter 10 causes it to quit after 10 seconds anyway.

Tar

Now, there is no reason why we can’t integrate tar and netcat together, and use this to transmit a
directory across a netcat socket:

On one side: tar zcfp - /path/to/directory | nc -w 3 127.0.0.1 1234

The tar statement before the pipe tars and compresses (using gzip) every file within that directory,
before printing its output to stdout (The screen). It is then caught by the pipe, and piped to nc which in
this example, connects to 127.0.0.1 on port 1234 and sends it the data which would normally hit the
screen. The w 3 switch causes nc to allow for a 3 second timeout (In the event of a temporary
disconnection or similar).

On the other side: nc -l -p 1234 | tar xvfpz

This will listen on port 1234 for a connection, and will pass any data received to tar. Using the option v
we can print out filenames to screen:

Simple Socket Reply

With what we have learned so far, we are easily able to get netcat to listen in on a socket, and pump out
any data we wish when it receives a connection.

As an example:

while true; do echo "Leave me alone" | netcat -l -p 1234 w10; done

Consider this line. Firstly lets examine echo "Leave me alone" | netcat -l -p 1234 -w10

What we are doing here, is listening in on port 1234 with a wait time of 10 seconds. If/when we receive
a connection, pipe the results of echo "Leave me alone" to netcat. The w 10 is necessary, as otherwise
any connection made in will remain open forever. We can also optionally add a v in to the netcat
command line which will give us verbose information, i.e. who is connecting.

Every time a connection times out (either with the w 10 command line switch, or because a connection
has been made and then closed), netcat will exit. As this is not what we want, we put the command line
within a standard BASH: while CONDITION; do STATEMENT; done clause, which when the
condition is set to true will run forever.

Inetd

If you build netcat with GAPING_SECURITY_HOLE defined, you can use it as an "inetd" substitute
to test experimental network servers that would otherwise run under "inetd".

A script or program will have its input and output hooked to the network the same way, perhaps sans
some fancier signal handling.

Given that most network services do not bind to a particular local address, whether they are under
"inetd" or not, it is possible for netcat avoid the "address already in use" error by binding to a specific
address.

This lets you [as root, for low ports] place netcat "in the way" of a standard service, since inbound
connections are generally sent to such specifically-bound listeners first and fall back to the ones bound
to "any".

This allows for a one-off experimental simulation of some service, without having to screw around
with inetd.conf. Running with -v turned on and collecting a connection log from standard error is
recommended.

Netcat as well can make an outbound connection and then run a program or script on the originating
end, with input and output connected to the same network port.

This "inverse inetd" capability could enhance the backup-server concept described above or help
facilitate things such as a "network dialback" concept.

The possibilities are many and varied here; if such things are intended as security mechanisms, it may
be best to modify netcat specifically for the purpose instead of wrapping such functions in scripts.

Speaking of inetd, netcat will function perfectly well *under* inetd as a TCP connection redirector for
inbound services, like a "plug-gw" without the authentication step.

This is very useful for doing stuff like redirecting traffic through your firewall out to other places like
web servers and mail hubs, while posing no risk to the firewall machine itself.

Put netcat behind inetd and tcp_wrappers, perhaps thusly:

www stream tcp nowait nobody /etc/tcpd /bin/nc -w 3 realwww 80

and you have a simple and effective "application relay" with access control and logging. Note use of
the wait time as a "safety" in case realwww isn't reachable or the calling user aborts the connection --
otherwise the relay may hang there forever.

Inetd/tcp_wrappers and netcat information, courtesy of: http://www.spyder-fonix.com/netcat.html

Talking to syslogd -r

Syslog Daemons running with the r switch log not only their own hosts data but accept remote UDP
broadcasts. They listen in on UDP port 514.

"echo '<0>message' | nc -w 1 -u loggerhost 514"

If loggerhost is running syslogd r and can accept your messages.

Note the -u switch here, to put netcat into UDP mode. Specifying the <0> before your message ensures
that your message receives top priority within syslog (kern.emerg)

Internetworking Basics

For the purposes of this section, machine refers to an x86 compatible PC with a connection to the
Internet through some means, terminated by a standardized TCP/IP stack.

Each machine on the Internet today comes shipped with a standard, compatible TCP/IP stack. This
stack guarantees the use of 65535 ports, and IPv4 protocol compatibility.

Below we can see the OSI model. This explains in terms of 7 layers, how data is constructed at one
host and received at the next.

In short; Data is constructed on the left by an application, encodes it with a transport (TCP) which takes
it over the network (IP), resolves MACs of local devices (Data Link) and then passes a constructed
packet to the network card which transmits (Physical) it over the wire (at which point the opposite
happens at the other end).

You may have intelligent devices such as switches along the way. These for example may be wise up
to layer 5 for example and not only route according to MAC address (Layer 2) but inspect and firewall
packets based on findings up to Layer 5 (Simple firewalling) or even Layer 7 (Packet inspection).

"The OSI, or Open System Interconnection, model defines a networking framework for implementing
protocols in seven layers. Control is passed from one layer to the next, starting at the application layer
in one station, proceeding to the bottom layer, over the channel to the next station and back up the
hierarchy." (Courtesy of: http://webopedia.internet.com/quick_ref/OSI_Layers.asp)

nc -e

We have already discussed the basics of redirection with netcat. Netcat has a e switch which we can
use to execute a program on connection. There are a couple of viable and legitimate uses for this, i.e.
running as nc e v called by the inetd wrapper, which we can use to view traffic and information on
users connecting to wrapped daemons, however the most common use which we will explore here is
using it to redirect to and from /bin/bash or similar shell, for both good and bad.

One method could be this:

adam@adamp:~$ nc -v -e '/bin/bash' -l -p 1234 –tlistening on [any] 1234 ...
connect to [127.0.0.1] from localhost [127.0.0.1] 51210

In one window, and a simple telnet localhost 1234 in another window:

adam@adamp:~$ telnet 127.0.0.1 1234
Trying 127.0.0.1..
.
Connected to 127.0.0.1.
Escape character is '^]'
.
echo Test
Test
^
]
telnet>

Scanning

The scanning features of netcat can be used against yours or your friends networks to get useful
information about which hosts have certain ports open. You can also send a precompiled data file to
each. For example:

Echo EXIT | nc -w 1 127.0.0.1 20-250 500-600 5990-7000

Will scan 127.0.0.1 on ports 20-250, 500-600 and 5990-7000. Every port that it finds is open, it will
pipe the output of echo "EXIT" being the word "EXIT" to that port.
The results are as follows:

(For the sanity of my server, I have blocked out a number of parts from certain service banners.)
And now with UDP scanning: nc -v -w 1 127.0.0.1 u 20-250 500-600 5990-7000 we receive:

adam@adamp:~$ nc -u -v -w 1 127.0.0.1 20-250 500-600 5990-7000localhost [127.0.0.1] 250 (?) openadam@adamp:~$

-v was to put netcat into verbose mode, and u was telling netcat to fall into UDP mode.

Spoofing

"Your TCP spoofing possibilities are mostly limited to destinations you can source-route to, while
locally bound to your phony address.

Many sites block source-routed packets these days for precisely this reason.

If your kernel does oddball things when sending source-routed packets, try moving the pointer
around with -G. You may also have to fiddle with the routing on your own
machine before you start receiving packets back.

Warning: some machines still send out traffic using the source address of the outbound interface,
regardless of your binding, especially in the case of localhost.

Check first. If you can open a connection but then get no data back from it, the target host is probably
killing the IP options on its end [this is an option inside TCP wrappers and several other packages],
which happens after the 3-way handshake is completed.

If you send some data and observe the "send-q" side of "netstat" for that connection increasing but
never getting sent, that's another symptom. Beware: if Sendmail 8.7.x detects a source-routed SMTP
connection, it extracts the hop list and sticks it in the Received: header!"

http://www.spyder-fonix.com/netcat.html

Spoofing is a useful technique, as is source routing.

Source routing is almost obsolete now, and the majority of routers filter out source routed packets.
Source routing in a nutshell is basically setting the route that the packet will take at the source, and
storing that information along with the packet.

Normally, each router makes its own mind up as to where a packet will get routed, and follows its
predefined routing tables. If we have access to all routers between our device and the target device
(which can be one machine if youre talking about your local LAN server), then we are able to modify
the routing entries on those devices, bind a phoney address to our machine and source route packets to
the intended destination.

Spoofing is where we modify the source address of a packet so that the recipient believes it came from
a different address. There are two problems with this;


A number of clever ISP routers will drop packets with incorrect source addresses.

If the destination host does get to receive your spoofed packet, it will send data back to the
spoofed address (instead of ours). This does have a number of uses however in the example of
ICMP ping flooding a host and spoofing the source address to Microsoft.com (as a theoretical
example).

Simple Response Service

echo -e "GET http://www.google.com HTTP/1.0\n\n" | nc w 5www.google.com 80

We make a connection to google.com on port 80 (Web server port), and put in an HTTP request for
http://www.google.com. At this point, we are presented with the HTML spurted out by the web server.
We can pipe this to "| less" or similar or even our favourite HTML interpreter.

Take a look at this example, and you will see what we have done here. In one instance we have created
an HTML file webfrontend and we now pipe that HTML to any incoming connection to netcat on port
1111. We then make a connection on the larger window, using lynx http://127.0.0.1:1111 and we have
made ourselves a tiny http server, possibly could be used as a holding page server or something similar.

Advanced Uses

Now we'll set up a server netcat to listen on port 1111. We'll also set up a client netcat to talk to the real
web server on port 81. By getting them to pass all data they receive to each other, together they form a
proxy; something that sits in the middle of a network connection. Here are the commands we use:

mknod backpipe p

nc -l -p 1111 0backpipe

Because bash pipes only carry data in one direction, we need to provide a way to carry the responses as
well. We can create a pipe on the local filesystem to carry the data in the backwards direction with the
mknod command; this only needs to be run once.

Requests coming into the proxy from the client arrive at the first nc, listening on port 1111. They get
handed off to the "tee" command, which logs them to the inflow file, then continue on to the second nc
command which hands them off to the real web server. When a response comes back from the server, it
arrives back at the second nc command, gets logged in the second tee command to the outflow file, and
then gets pushed into the backpipe pipe on the local filesystem. Since the first netcat is listening to that
pipe, these responses get handed to that first netcat, which then dutifully gives them back to the original
client.

While the above example is for watching tcp streams going to and from a web server, the above
technique is useful for watching any tcp connection. In fact, since nc also works with udp packets
something telnet can't do - it should be possible to even set up udp proxies this way.

Windows Command Shell

As we can see from the image above, we have started netcat with options of l p 1234 e
"c:\windows\system32\cmd.exe". These are the same options as with the Unix shell, and this should
theoretically start a cmd.exe shell listening in on port 1234:

As you see from above, this has succeeded. Netcat and program execution for Windows can be used in
exactly the same way.

Unauthorised Proxying

Assume you are an administrator of a Linux router. Using the methods above, as well as your iptables
software, you can proxy a users outgoing connection through your nc proxy. Using iptables with the j
DNAT target and the j REDIRECT target, you can transparently proxy outgoing connections through
to any other ports you want, and what better to use than your nc proxy?

Cryptcat

Cryptcat can be found at: http://sourceforge.net/projects/cryptcat/ and is the ultimate companion for
Netcat. It includes a lightweight version of Netcat, featuring encrypted transport properties. (Just for
those superbly paranoid!). Useful for encrypting communications out of a network.

Final Thoughts

If I was given one tool on a freshly installed PC, I would ask for Netcat. Due to its versatility and its
huge range of uses, it can be used as a transfer tool, a scanning tool, a server, a proxy and so much
more. I have put down everything useful I can think of, and welcome any further suggestions directed
to adam@apnicsolutions.com

Command Cheat Sheet

The following are the most useful uses of netcat:

For windows nc d can be used to detach from the console.

nc -l -p [port] will create a simple listening tcp port. Add u to put into UDP mode.
nc -e [program] To redirect stdin/stdout from program.
nc -w [timeout] To set a timeout before netcat automatically quits. (Used within a loop usually)
program | nc To pipe output of program to netcat
nc | program To pipe output of netcat to program
nc -h Help sheet
nc -v To put into verbose mode, or use v v to put into ultra-verbose mode!
nc -g or nc G Source routing flags
nc -t Use telnet negotiation (If performing telnet negotiation)
.
nc -o [file] Hex dump traffic to file
nc -z No I/O (Used for scanning ports)

Comment Spam Problems Fixed, I think

Submitted by jgoerzen on Sun, 04/17/2005 - 9:43pm.

I have switched this site from using a Bayesian spam filter to using Captchas. I've had a lot of success with this method against comment spam in the past, and I think it will make the site much more pleasant.

More details can be found in my blog.

Call for Contributions: HCAR

Submitted by jgoerzen on Fri, 04/15/2005 - 6:21pm.

It's that time of year again: the call for contributions to the Haskell Communities and Activities Report has been posted. If you develop any Haskell software, participate in the Haskell community, etc., you ought to submit something for the report.

Plea for participation in GHC survey

Submitted by jgoerzen on Fri, 04/15/2005 - 6:18am.

Simon Peyton-Jones posted a request for all GHC users to complete the GHC survey. He's only received 230 responses so far, and would like to see much better participation with the survey.

Please do the right thing and participate. Takes only 5 minutes.

GHC survey announced

Submitted by simonmar on Fri, 03/04/2005 - 9:02am.

The GHC Team announced a user survey giving you the chance to comment on all aspects of GHC from your favourite features, wishlist items, to the development model.