Tracing HTTP Requests with tcpflow (opens in new tab)

(ananthakumaran.in)

109 pointsananthakumaran3y ago16 comments

16 comments

16 comments · 8 top-level

nhoughto3y ago· 4 in thread

This looks handy, better ergonomics than strace to just hook into http requests of a running process.

I wonder how it works with TLS if it’s working at the socket level..

xtreak293y ago

There seems to be an issue open for this https://github.com/simsong/tcpflow/issues/58

youngtaff3y ago

I’m guessing it doesn’t work with TLS as it’d need the keys to decrypt the traffic

LinuxBender3y ago

I recall seeing a thread somewhere saying tcpflow would not add this capability and they point people to ssldump [1][2] and even that has some limitations.

[1] - https://github.com/adulau/ssldump

[2] - https://linux.die.net/man/1/ssldump

fmajid3y ago

With TLS 1.3 and Perfect Forward Safety, even knowing the secret key is no longer enough. Otherwise, ssldump is a tool that would dump sessions if you knew the server private key and were somehow able to force negotiating a non-PFS TLS 1.2 cipher suite.

imiric3y ago· 1 in thread

For a tool that understands TCP and HTTP conversations, the filter syntax is awfully similar to tcpdump's, and having to use `grep -A 15` to filter specific requests seems clunky.

It's good knowing this tool exists, but I think I'll stick to tcpdump and Wireshark. In Wireshark it's trivial to use the `http.request` filter to do this, and following the TCP conversation with decoded bodies, or specifying a TLS cert, is equally simple.

denysvitali3y ago

For a CLI Wireshark, use tshark

thegabez3y ago· 1 in thread

This assumes you have shell and those tools installed via your Dockerfile. Best practice is to have a multi stage build and just have the working binary, keeping the image as light as possible.

So when do you make a fat image with debug tools and when do you keep it skinny?

mozman3y ago

Or you’re running on a VM and can install these tools for debugging and then blow it away when you’re done

ystad3y ago· 1 in thread

A thought. I would usually start with my application logs. Reverting to a tcp tool for a http app problem seems cobtrived

0x4454423y ago

Yeah the app looked like it was delegating to another service and needed a bearer token for that. The app logs should have showed this straight away. Furthermore, I’ve been burned enough to know that if I have a service that’s dependent on other services then I setup tasks in my service to ping the health of those services periodically and log warnings/errors if they’re unreachable. This allows me to divert blame as quickly as possible if alerted in the middle of the night.

_ZeD_3y ago· 1 in thread

in this space I have successfully used mitmproxy, charles and, with the best experience, fiddler

mitmproxy.org www.charlesproxy.com https://www.telerik.com/fiddler/fiddler-classic

rwestergren3y ago

+1 for Fiddler, though it doesn't sound like the author was in a position to forward traffic to an HTTP proxy at the time in prod

jimmyed3y ago

There is also netpeek[1] which has better filtering capabilities and UX overall. It supports ngrep like bpf filters which we found useful.

1: https://github.com/darshanime/netpeek

29athrowaway3y ago

There's also termshark, a TUI for tshark inspired by Wireshark.

You also have mitmproxy which can be useful if TLS is involved.

aftbit3y ago

Just because I know them better, I would have used `netstat -putln` and `tcpdump -i lo -n -A port 8000` to do the same thing. I'll take a look at tcpflow and ss though - it's always nice to know more tools.

j / k navigate · click thread line to collapse