World's Dumbest TCP Service

Posted in programming - Saturday, May 4, 2013

I feel like a bash scripting Dr. Frankenstein. Last week, I wrote about the world’s dumbest instant messaging tool. This week I’ve moved on to the world’s dumbest TCP server. In my quest for a deep understanding of networking, it’s a bit of a detour, a roadside America attraction. I’m not so much learning new stuff here as figuring out how to explain what I know. And having fun pushing the limits of bash in the process.

Part of this is just the challenge of the absurd: Could I make this particular bear dance? But the serious point is two-fold: to make the idea of network services more accessible, and to show that bash is more powerful than you might think. People don’t think of bash as a programming language: A bash script usually starts life as a bunch of commands you executed one at a time on the command line, then copied into a file so you didn’t have to re-type them. That’s not real programming, is it? But bash also has variables, data structures, functions, and even process forking. That’s enough to get a fair amount done.

Network servers have the opposite problem. They’re big bits of infrastructure that Other People write. And infrastructure-grade servers - like the Apache web server - are complicated. Apache has to implement the full HTTP protocol, not just the small subset of it that most people use. It’s got all this logic for authenticating users, negotiating content types, redirecting to pages that have moved, and so on. On top of all that, it’s got developer-decades worth of edge case handling, performance optimizations, and feature creep.

But the core of what it does is straightforward: It listens on a socket, you connect to it and send it a message in a particular format, it does some processing on that, and it sends you a message in response. That’s fundamentally what all network servers do. The ones we’re familiar with - web, mail, and chat servers - have rich and complex message protocols, but a quick skim through the /etc/services file turns up sedimentary layers of oddly specific services, like ntp and biff. They do small, specific, useful things.

So the what’s the simplest server I can write that does something even minimally useful? And can I write it in bash?

I figured netcat is a good place to start. Last week, we used it to send messages back and forth between two people. All we want to do now is replace one of those people with a very small shell script. netcat -l port starts up a server that listens on a port and dumps anything it gets to standard output (stdout). It also sends anything it gets on standard input (stdin) back to the client. We just need to redirect netcat’s stdout to a program, and then redirect that program’s output to netcat’s stdin. Doing either of those alone would be trivial; doing both is tricky.

Figuring that out took a fair amount of digging through the bash man page, and experimenting to get the syntax right, but in the end it was a trivial amount of code. Let’s take it as read for now that we’ve got a program called wtf_server, which reads from stdin and writes to stdout. What we’re going to do is use bash’s built-in coproc command, which will start it up as a background process, and set up new file handles for its stdin and stdout.

coproc WTF { wtf_server; }

The WTF tells coproc to create an array named WTF and save the file handles in it. ${WTF[0]} will be wtf_server’s stdout, ${WTF[1]} will be its stdin. So now we can start up the netcat server with its stdout and stdin jumper-cabled to wtf_server as desired.

port=2345
nc -l $port <&${WTF[0]} >&${WTF[1]}

Really, that’s the hard bit. Now we just need a program to read stdin and write stdout. In fact, we don’t even need a real program; our wtf_server is actually just a bash function. In its simplest incarnation, it just echoes back what was sent to it:

function wtf_server () {
    while true ; do
        read msg
        echo "You said: '$msg'"
    done
}

With the coproc and netcat server running, you can switch to another terminal, open a client connection with netcat, and have an exchange like this:

$ nc localhost 2345
hello world!
You said: 'hello world!'

Ok, so that’s the proof of concept. We’re definitely falling short of the “minimally useful” criteria, but we can replace our echo with any bash commands we want. The only constraint is what it’s safe to do - this is still a toy service, anonymous and going over an unencrypted connection. Don’t run the input as shell commands, fer chrissakes. Within those limits, there’s still plenty of useful things we can do: reporting on system info or serving up static content. Here’s a sketch with a few ideas:

function wtf_server () {
    while true ; do
        read msg
        case $msg in
            i | index )
                ls $docs ;;
            get\ * )
                f=${msg#get }
                cat $docs/$f ;;
            t | time )
                date ;;
            u | uptime )
                uptime ;;
            * )
                echo "Commands: t, time; u, uptime; i, index; get <file>"
                echo "    ctrl-c to exit"
        esac
        echo -n "> "
    done
}

This gives us a limited interactive shell. Each case statement handles a different request format. We can get the machine’s current time and uptime stats. It also has a docs directory; we can list the files in it and cat them out individually. A session looks like this:

$ nc localhost 2345

Commands: t, time; u, uptime; i, index; get <file>
    ctrl-c to exit
> t
Sat May  4 11:52:52 EDT 2013
> u
 11:52:53 up 42 days,  5:48, 26 users,  load average: 0.67, 0.37, 0.35
> i
about.txt
status.txt
> get status.txt
Up late on a Friday, hacking bash scripts.
> ^C
$

(After connecting, I just hit return to send a blank line, and the server responded with the help text and the ‘>’ prompt. Every server response ends with a prompt.)

That’s it. No real protocol, certainly nothing formal like HTTP, just a set of ad-hoc request handlers, made up as we went along. The beauty of this is that it doesn’t depend on anything else. It’s not running behind Apache or anything. There’s no development environment to set up, no gems to install; just one standard unix utility - netcat - and bash handles all the rest.

Here’s the full wtf_server.sh script that starts this up.

#!/bin/bash

# Weird little TCP server
# Tells time and uptime; can list and dump files in an "docs" subdir

# Takes a port parameter, just so you know which one you're running on.
test -n "$1" || { echo "$0 <port>"; exit 1; }
port=$1
dir=`dirname $0`
docs=$dir/docs

function wtf_server () {
    while true ; do
        read msg
        case $msg in
            i | index )
                ls $docs ;;
            get\ * )
                f=${msg#get }
                cat $docs/$f ;;
            t | time )
                date ;;
            u | uptime )
                uptime ;;
            * )
                echo "Commands: t, time; u, uptime; i, index; get <file>"
                echo "    ctrl-c to exit"
        esac
        echo -n "> "
    done
}

# Start wtf_server as a background coprocess named WTF
# Its stdin filehandle is ${WTF[1]}, and its stdout is ${WTF[0]}
coproc WTF { wtf_server; }

# Start a netcat server, with its stdin redirected from WTF's stdout,
# and its stdout redirected to WTF's stdin
nc -l $port -k <&${WTF[0]} >&${WTF[1]}