This is done by measuring the minimum TCP RTT (client.socket.tcpi_min_rtt) seen and the smoothed TCP RTT (client.socket.tcpi_rtt). I am getting this data by using Fastly Custom VCL, they get this data from the Linux kernel (struct tcp_info -> tcpi_min_rtt and tcpi_rtt). I am using Fastly for the Demo since they have PoPs all around the world and they expose TCP socket data to me.
The score is calculated by doing tcpi_min_rtt/tcpi_rtt. It's simple but it's what worked best for this with the data Fastly gives me. Based on my testing, 1-0.7 is normal, 0.7-0.3 is normal if the connection is somewhat unstable (WiFi, mobile data, satellite...), 0.3-0.1 is low and may be a proxy, anything lower than 0.1 is flagged as TCP proxy by the current code.
Also, the readme has slightly incorrect logic I think:
> According to Special Relativity, information cannot travel faster than the speed of light. Therefore, if the round trip time (RTT) is 4ms, it's physically impossible for them to be farther than 2 light milliseconds away, which is approximately 600 kilometers.
It calls out the 33% for fiber but ignores that there’s not a straightline path between two points on the network and there could be wireless, cable, and DSL links somewhere on that hop.
Also, the controlled variable here is latency, not distance. Thus you can always increase latency through buffering and therefor you could be made to appear further than you are. And that buffering need not even be intentional - your perceived distance estimate will vary based upon queuing delays in intermediary depending on time of day (itself a fingerprint if you incorporate time-aware measurements, but a source of error if you don’t).
Fingerprinting is hard and I dislike the framing that it’s absolutely impossible to mask or that there’s not false positive and false negative error rates with the fingerprint.
The point I was trying to make is that if the RTT is low enough you can know the connection is being made from close, it's an upper bound, and making some assumptions you can get it lower, so it's not a way of knowing the exact distance but rather the max distance the connection can be made from. If someone is in Spain but they can't be more than 400km from Australia, something went terribly wrong somewhere hehe
In hindsight I think the issue with my explanation is that I was trying to explain the differences when fingerprinting two different protocols, but ended up going for a TCP-only approach since Fastly wouldn't expose to me the data I needed for the TLS and HTTP RTT. But in theory fingerprinting with protocol RTT difference where one protocol is proxied and the other is impossible to bypass, but this is only the theory.
I think I will edit the README in the future since I don't like how it turned out too much. Thanks for the feedback!
By the way, it detects Tor, I tested it ;D
Alice wants you to think she's in New York when she's really in Taipei, so she gets a VM in New York and runs a browser in it via RDP. How are you detecting this?
Countermeasure: pick some min-RTT >= the actual client RTT (you can do this as a TCP proxy by measuring client ping). Measure server RTT and artificially delay responses to be >= min-RTT. This will require an added delay during the handshake and ACKs, but no added delay for the response payloads.
Counter-countermeasure: the above may lead to TCP message types that don't make sense given a traditional TCP client state machine (e.g., delayed ACK would bundle ACK and PUSH but the system shows separate/simultaneous ACK and PUSH packets. Counter-counter-countermeasure is left to the reader.
It's a complex but fun world we live in hehe
The difference in min TCP RTT and min RTT to respond to a websocket payload is a dead giveaway that there's a middlebox terminating TCP somewhere along the path. You can bypass this by sourcing your request within 30ms of wherever TCP is being terminated, anything under that threshold could be caused by regular noise and isn't a reliable fingerprint. Due to how many gateway's there are between you and a residential proxy exit node this makes fingerprinting them extremely easy.
I expect it won't be long until someone deploys the first proxy service that handles the initial CONNECT payload in the kernel before offloading packet forwarding to an eBPF script that will proxy packets between hosts at layer 3, making this fingerprinting technique obsolete. The cat and mouse game continues.
https://github.com/sshuttle/sshuttle basically works like this. I've used it for many years. I don't think it'll be possible to detect using this technique.
I suppose it's possible botnets ("residential proxies") may get detected this way if they're using SOCKS to forward requests?
Still, this looks like an interesting signal to add to a system like Anubis to increase the difficulty for suspicious traffic sources.
This does very reliably detect TOR traffic, though you can just download a list of exit nodes if that's what you want.
The problem is that the proxies which are targets of identification - think proxies for large scale web scraping which use CONNECT tunnels - dont get to "see" the request.
<html><body><h1>You don't seem to be using a TCP Proxy!</h1><p>(If you are using a VPN or any other kind of proxy that is not a TCP Proxy, this will not detect it)</p></body></html>
This tunnel operates at layer 3, where the client sends TCP segments to the proxy, the server unpacks the segments and then repacks them into new segments to send to the end target. These new TCP segments will contain the timestamp of when they were created.
The HTTP request sent through those segments is unmodified, meaning it will contain the original timestamp from the client machine.
The newer timestamp on the TCP segments means there is a mismatch between the TCP RTT and the HTTP RTT.
If you did the timings by comparing to other protocols, like TLS or HTTP you could do this with a single server, but that's a bit more complex than doing it on the same protocol since you have to account for more stuff, but it could be done, at the end of the day, my idea with Aroma was mostly to prove that it's possible, thanks for the feedback btw!
When deployed on a popular server, one bit of "IP intelligence" this detector itself can gather is keep database of lowest-seen RTT per given source IP, maybe with some filtering - to cut out "faster-than-light" datapoints, gracefully update when actual network topology changes, etc.
That would establish a baseline, and from there, additional end-to-end RTT should become much more visible.
I imagine any big CDN implementing something like this could keep a database of all of this, combined with the old kind of IP intelligence and collecting not only RTT on other protocols like TLS, HTTP, IP (aka ping, and traceroutes too), TCP fingerprint, TLS fingerprint, HTTP fingerprint...
And with algorithms that combine and compare all these data points, I think very accurate models of the proxy could be made. And for things like credit card fraud this could be quite useful.
Is there some class of bad actors who extensively use TCP proxies and not only _don't_ use VPNs, but would incur large costs in switching to them?
Theres no way to include a timestamp in a UDP datagram so all timestamps received would be from the client machine.
So far I've only seen Bright Data (among the large players) offer UDP proxying over QUIC/HTTP3, but that's pretty limiting since less than half of sites have HTTP/3 enabled to begin with.
We (PingProxies) might be the only company to offer H3 to the proxy/QUIC to the target using the CONNECT-UDP method publicly. Although, it is in beta/unstable until I merge my changes into Rust's H3 library.
If you wanna play around with it, email me and I'll get you some credit. I think theres potential for stealth since outdated proxy clients/servers mean automated actors never use H3.
The proxy industry is full of another 100 companies saying they offer H3/QUIC, when they mean UDP proxying using SOCKS. I suppose the knowledge gap and what customers care about (protocol to end target) is very different to what I care about (being right/protocol to the proxy server).
That's what I thought too, but it's working for me. (I've sent a lot of tickets, maybe they've put our account as something special without telling me, but doubt it.)
> If you wanna play around with it, email me and I'll get you some credit.
Done, emailed! :) Thanks!
> The proxy industry is full of another 100 companies saying they offer H3/QUIC, when they mean UDP proxying using SOCKS.
Out of the large players I've tested, none actually seem to even support SOCKS5's UDP ASSOCIATE. (I have not tested PingProxies yet.)
> I suppose the knowledge gap and what customers care about (protocol to end target) is very different to what I care about (being right/protocol to the proxy server).
I think there's a knowledge gap between the people making the sales landing pages, and the folks who actually run/maintain the proxy servers. There's some large vendors that advertise UDP support (for residential and/or mobile proxies) that I have yet to actually see working.
Also available as audiobook, and a documentary ("The KGB, The computer and Me"). https://www.youtube.com/watch?v=Xe5AE-qYan8
If you are then it means the score is sometimes a bit lower and sometimes a bit higher than 0.1, which is the threshold for getting blocked.
If you want to know the exact score, you can check https://aroma.global.ssl.fastly.net/score
It's set at a low threshold since I want to avoid blocking regular users at all costs, I think the detection can be improved a lot by using more data and not a single division to calculate the score, in this case it's a somewhat simple PoC.
Thanks for taking the time to test it, I really appreciate it!
It's a super cool tool, I've been wondering about an open source tool doing this since reading about the technique in one of Nikolai Tschacher's blog posts years ago (https://incolumitas.com/pages/about/).
There's a few ways to work around this, but I think it's one of the best signals available to detect low-effort/common proxy providers.
I want to clarify that the approaches are a bit different, they use IP intelligence too and this approach doesn't use any kind of websockets, which is a really good idea, and I have to admit I didn't think of that, but sadly it's not really possible to do it with Fastly.
Another big difference is that this could work with any TCP application, not only HTTP, and if you do it with HTTP/S you can know if it's a proxy or not on a request basis and totally passively, without adding any delay or changing the code of the app.
But yeah, it's a really cool demo, thanks again!