Monitoring network traffic is a problem solved a long time ago, one might think. So why write the first entry in my shiny new Blog about monitoring network traffic? Well the answer is simple: because monitoring network traffic is a quite complex topic when multiple components, such as SSL, proxies etc. are involved.
Sure, almost every Linux user knows tcpdump, the intuitive solution for any basic monitoring need. Windows users might think of Wireshark or Proxomitron, for example. And for the Java people, there is at least Apache TCPMon which satifies basic monitoring needs.
All of these tools, however, could not satisfy my requirements: In my latest project, I had to monitor SOAP messages sent from a client on my machine and their replies which were sent from a remote server. And this is where the trouble begins: the server only accepts SSL connections AND to connect to the server, I have to go through a SOCKS proxy.
As long as only one of these restrictions is in place, most tools can still be used. But connecting to a server via SSL through a SOCKS proxy, while at the same time monitoring the (unencrypted) traffic transmitted to the server is something which is obviously not possible at the same time.
SOCAT -- the swiss army knife of all networking needs
The solution came in the form of socat which is described on its Web site as a "multipurpose relay". It is a command line utility which "establishes two bidirectional byte streams and transfers data between them". Sounds simple, but it is powerfull as it supports a wide range of data sources, data sinks, and a variety of options which can be applied to the streams.
For example, I could listen to the local port 5000 and forward all traffic to the server my.destination.host.net on port 443 via a SOCKS proxy at my.proxy.host.net:4033 by starting
socat TCP4-LISTEN:5000,reuseaddr,fork PROXY:my.proxy.host.net:my.destination.host.net:443,proxyport=4033
Or, in order to monitor packages between my machine and the server which only accepts SSL connections, I would invoke
socat -v TCP4-LISTEN:5000,fork OPENSSL:my.destination.host.net:443,verify=0
Naturally, to use the monitor, I have to reconfigure my client to access http://localhost:5000 instead of https://my.destination.host.net:443.
But sadly, even socat does not solve my problem out of the box. Both SOCKS proxies and OpenSSL connections are regarded as data sinks, so they cannot be combined directly. But as a solution, we can chain our socat relays:
Open two separate terminals. Then, in the first one, invoke
socat -v TCP4-LISTEN:5000,fork OPENSSL:localhost:5001,verify=0
This listens at port 5000, dumps all traffic to the console and wraps the data in an OpenSSL connection. In the second terminal, invoke
socat TCP4-LISTEN:5001,reuseaddr,fork PROXY:my.proxy.host.net:my.destination.host.net:443,proxyport=4033
This listens on port 5001 (the destination port of our encrypted data) and forward the data to our destination server via the SOCKS proxy. Voilà, here's our network monitor.
Conclusion
I love socat. Its uses are really endless. I have shown above, how multiple socat instances can be queued to match a rather challenging monitoring scenario. But the actual usage scenarios of socat are almost endless. For example, as it can write to and read from files instead of the network, it could even be used to mock servers in unit tests. The man page gives a whole lot of additional examples, including a simple SMTP server, an HTTP echo server, or a tool to merge data arriving on several different TCP connections into one.
Do I wish for something? Actually, I do. I'd really love a Java port of a socat-like tool with the same flexibility and maybe even with a GUI frontend to conveniently set up the tool in the way I need. So, if anyone has some spare time, please go ahead :-)