Requestdiff – Send two HTTP requests and visualize any differences (opens in new tab)

(requestdiff.com)

142 pointsvictordg10y ago33 comments

33 comments

32 comments · 18 top-level

jalfresi10y ago· 5 in thread

For command-line junkies:

diff <(curl -sS -L https://httpbin.org/get) <(curl -sS -L https://httpbin.org/get?show_env=1)

jalfresi10y ago

Actually this is closer still to the output. Requires colordiff:

diff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1) | colordiff

Aissen10y ago

You can replace colordiff with vim's less tool (/usr/share/vim/vim74/macros/less.sh ) or vimpager (https://github.com/rkitover/vimpager).

icdiff is also nice to have a side-by-side colored diff (https://github.com/jeffkaufman/icdiff). And of course, you could use meld, vimdiff, kompare etc. with the same bash anonymous pipes indirection.

dgulino10y ago

you can leave off the pipe: colordiff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1)

tacone10y ago

The website does show differences in the HTTP headers though.

Tepix10y ago

As does this solution.

zwischenzug10y ago· 3 in thread

This would be useful behind a firewall. Is the source available?

victordgOP10y ago

Not yet, but I'm planning to open source it later today. I will reply here with the Github link. Glad that you found it useful!

weruiojkl2410y ago

Awesome, waiting for the Github repo!

1 more reply

zwischenzug10y ago

Great, thank you.

therein10y ago· 2 in thread

Good work but frankly this would be useful if it was:

(1) Terminal based

(2) Supported other types of HTTP requests

(3) Supported request body

(4) Allowed editing request headers

(5) Wasn't so easily exploitable to be used as a proxy or a DDoS relay (server-side bummer):

http://requestdiff.com/proxy?url1=https://www.amazon.com&url...

JackuB10y ago

You might want to take a look at Gavel https://github.com/apiaryio/gavel (and its JS implementation Gavel.js https://github.com/apiaryio/gavel.js)

(disclosure: I work for Apiary.io)

odabaxok10y ago

(6) Support encodings other than utf-8.

blemasle10y ago· 2 in thread

Requests are made server side, which make this unusable for local/dev environnement purpose. Too bad !

victordgOP10y ago

Right. I was aiming to do it client-side only but I found out that web browsers restrict the headers you have access to through cross-origin XHR requests [1][2].

So it was a trade-off and I preferred showing all the headers at the expense of having to proxy the calls through a server.

[1] http://www.w3.org/TR/XMLHttpRequest/#the-getallresponseheade...

[2] http://www.w3.org/TR/cors/#cross-origin-request

I will make it available on Github later, so that you at least have the option of running it yourself.

JackuB10y ago

For dev or CI you might want to take a look at Gavel https://github.com/apiaryio/gavel (or even Dredd for that matter https://github.com/apiaryio/dredd)

(disclosure: I work for Apiary.io)

diggan10y ago· 1 in thread

This is very cool! Looking forward to it being open source (also, things from Barcelona rocks ;) )

For people wanting to have a CLI tool instead, John Graham-Cumming's httpdiff might be worth looking at. https://github.com/jgrahamc/httpdiff

victordgOP10y ago

Thanks a lot :)

Httpdiff is a really good tool and way more powerful than mine.

I've used it multiple times and wanted to try making something that focused more on the request body (which in Httpdiff are saved to temporary files) and that was also a bit more visual than a CLI.

Btw, here is the code! https://github.com/vdel26/requestdiff

smarx10y ago· 1 in thread

Seems like a useful tool; thanks for sharing it!

Two questions:

1) What's the diff logic? At first glance, it looks like JSON is reformatted (maybe canonicalized in some way) and then a line-by-line diff is applied. Is there more to it? Since the tool seems JSON-aware, I was surprised to see an added trailing comma up as a difference.

2) Do you have plans to expand the kind of HTTP requests users can make? It would be nice to use different verbs, headers, and request bodies. Runscope has a similar tool[0] built in that I believe (haven't tried it yet) allows a bit more flexibility, but it would be nice to have a standalone tool available.

[0] http://blog.runscope.com/posts/comparing-http-requests-using...

victordgOP10y ago

Great questions:

1) the diff is really simple, line by line and text based. I'm just using this really awesome library to do it: https://github.com/kpdecker/jsdiff The JSON is just being pretty printed to be able to do a text diff in a meaningful way.

2) Yes, I do, and thanks for the suggestions! But I just wanted to go with the minimal version first and see if people found it useful.

I didn't know about Runscope's diffing feature. Btw, I'm using a HTTPBin for the sample requests, which is a Runscope project as far as I know :)

gregingelmo10y ago

Use Vim, it's the best diff tool by a long shot. Syntax highlighting, side-by-side diffs, and code folding (when large contiguous sections are equal)

vimdiff <(curl -s "https://httpbin.org/get") <(curl -s "https://httpbin.org/get?show_env=1")

It takes some getting used to but once you go vimdiff you'll never go back.

no_gravity10y ago

Comparing URLs comes up from time to time here on Hacker News and at one point inspired me to write a tool that batch compares pages visually:

http://www.productchart.com/blog/2015-07-19-urldiff

It renders a set of pages in a headless browser, compares them visually and alerts you if something changed.

Just a few lines of bash as you can see. But it turned out to be pretty useful. UrlDiff is a regular part of our regression testing at Product Chart now.

kozhevnikov10y ago

Reminds me of Diffy, a tool Twitter is using to algorithmically diff test their HTTP-based services.

https://blog.twitter.com/2015/diffy-testing-services-without...

pixelbeat10y ago

Personally I use these from the command line:

http://www.pixelbeat.org/scripts/urldiff http://www.pixelbeat.org/scripts/idiff

See also mergely which supports diffing URLs: http://pixelbeat/programming/diffs/#mergely

andybak10y ago

Wonderful.

Just the other day I needed something similar and was disappointed that I could find it.

I wanted to discuss something with a remote colleague and to illustrate it I wanted a visual diff of two files. I was hoping there was a nice little web app offering this but I was forced to screenshare (I could have terminal-shared but it was more hassle).

I was hoping for something like Etherpad but with a live visual diff.

dannybtran10y ago

Not working on Safari Version 8.0.6 (10600.6.3) Mac OS X 10.10.3

From JS Console:

[Error] TypeError: undefined is not a function (evaluating 'Array.from(e)') _toConsumableArray2 (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) s (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) f (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) onload (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1)

vortico10y ago

It's pretty easy to write a recursive diff function that compares JSON strings, in order to avoid the JSON -> diff by line hack that you're doing. But it's a clever hack that easily translates to the command-line.

skastel10y ago

If you're looking for something similar, but with more features that natively understand HTTP, check out www.runscope.com. Full disclosure, I'm an employee and proud of what we've built!

krakensden10y ago

Another solution: https://github.com/xthexder/httptee

A daemon that sends to two backends, and diffs the results.

amelius10y ago

I'd like to have a bookmarklet that lets me compare the states of the DOM trees of a website taken at two different times.

gdc10y ago

Doesn't seem to work at all in IE11 / RTM Win10.

SimeVidas10y ago

Use cases?

j / k navigate · click thread line to collapse

33 comments

32 comments · 18 top-level

jalfresi10y ago· 5 in thread

For command-line junkies:

diff <(curl -sS -L https://httpbin.org/get) <(curl -sS -L https://httpbin.org/get?show_env=1)

jalfresi10y ago

Actually this is closer still to the output. Requires colordiff:

diff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1) | colordiff

Aissen10y ago

You can replace colordiff with vim's less tool (/usr/share/vim/vim74/macros/less.sh ) or vimpager (https://github.com/rkitover/vimpager).

dgulino10y ago

you can leave off the pipe: colordiff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1)

tacone10y ago

The website does show differences in the HTTP headers though.

Tepix10y ago

As does this solution.

zwischenzug10y ago· 3 in thread

This would be useful behind a firewall. Is the source available?

victordgOP10y ago

Not yet, but I'm planning to open source it later today. I will reply here with the Github link. Glad that you found it useful!

weruiojkl2410y ago

Awesome, waiting for the Github repo!

1 more reply

zwischenzug10y ago

Great, thank you.

therein10y ago· 2 in thread

Good work but frankly this would be useful if it was:

(1) Terminal based

(2) Supported other types of HTTP requests

(3) Supported request body

(4) Allowed editing request headers

(5) Wasn't so easily exploitable to be used as a proxy or a DDoS relay (server-side bummer):

http://requestdiff.com/proxy?url1=https://www.amazon.com&url...

JackuB10y ago

You might want to take a look at Gavel https://github.com/apiaryio/gavel (and its JS implementation Gavel.js https://github.com/apiaryio/gavel.js)

(disclosure: I work for Apiary.io)

odabaxok10y ago

(6) Support encodings other than utf-8.

blemasle10y ago· 2 in thread

Requests are made server side, which make this unusable for local/dev environnement purpose. Too bad !

victordgOP10y ago

Right. I was aiming to do it client-side only but I found out that web browsers restrict the headers you have access to through cross-origin XHR requests [1][2].

So it was a trade-off and I preferred showing all the headers at the expense of having to proxy the calls through a server.

[1] http://www.w3.org/TR/XMLHttpRequest/#the-getallresponseheade...

[2] http://www.w3.org/TR/cors/#cross-origin-request

I will make it available on Github later, so that you at least have the option of running it yourself.

JackuB10y ago

For dev or CI you might want to take a look at Gavel https://github.com/apiaryio/gavel (or even Dredd for that matter https://github.com/apiaryio/dredd)

(disclosure: I work for Apiary.io)

diggan10y ago· 1 in thread

This is very cool! Looking forward to it being open source (also, things from Barcelona rocks ;) )

For people wanting to have a CLI tool instead, John Graham-Cumming's httpdiff might be worth looking at. https://github.com/jgrahamc/httpdiff

victordgOP10y ago

Thanks a lot :)

Httpdiff is a really good tool and way more powerful than mine.

I've used it multiple times and wanted to try making something that focused more on the request body (which in Httpdiff are saved to temporary files) and that was also a bit more visual than a CLI.

Btw, here is the code! https://github.com/vdel26/requestdiff

smarx10y ago· 1 in thread

Seems like a useful tool; thanks for sharing it!

Two questions:

[0] http://blog.runscope.com/posts/comparing-http-requests-using...

victordgOP10y ago

Great questions:

2) Yes, I do, and thanks for the suggestions! But I just wanted to go with the minimal version first and see if people found it useful.

I didn't know about Runscope's diffing feature. Btw, I'm using a HTTPBin for the sample requests, which is a Runscope project as far as I know :)

gregingelmo10y ago

Use Vim, it's the best diff tool by a long shot. Syntax highlighting, side-by-side diffs, and code folding (when large contiguous sections are equal)

vimdiff <(curl -s "https://httpbin.org/get") <(curl -s "https://httpbin.org/get?show_env=1")

It takes some getting used to but once you go vimdiff you'll never go back.

no_gravity10y ago

Comparing URLs comes up from time to time here on Hacker News and at one point inspired me to write a tool that batch compares pages visually:

http://www.productchart.com/blog/2015-07-19-urldiff

It renders a set of pages in a headless browser, compares them visually and alerts you if something changed.

Just a few lines of bash as you can see. But it turned out to be pretty useful. UrlDiff is a regular part of our regression testing at Product Chart now.

kozhevnikov10y ago

Reminds me of Diffy, a tool Twitter is using to algorithmically diff test their HTTP-based services.

https://blog.twitter.com/2015/diffy-testing-services-without...

pixelbeat10y ago

Personally I use these from the command line:

http://www.pixelbeat.org/scripts/urldiff http://www.pixelbeat.org/scripts/idiff

See also mergely which supports diffing URLs: http://pixelbeat/programming/diffs/#mergely

andybak10y ago

Wonderful.

Just the other day I needed something similar and was disappointed that I could find it.

I was hoping for something like Etherpad but with a live visual diff.

dannybtran10y ago

Not working on Safari Version 8.0.6 (10600.6.3) Mac OS X 10.10.3

From JS Console:

vortico10y ago

skastel10y ago

If you're looking for something similar, but with more features that natively understand HTTP, check out www.runscope.com. Full disclosure, I'm an employee and proud of what we've built!

krakensden10y ago

Another solution: https://github.com/xthexder/httptee

A daemon that sends to two backends, and diffs the results.

amelius10y ago

I'd like to have a bookmarklet that lets me compare the states of the DOM trees of a website taken at two different times.

gdc10y ago

Doesn't seem to work at all in IE11 / RTM Win10.

SimeVidas10y ago

Use cases?

j / k navigate · click thread line to collapse