To a Man with `Jq`, Everything Looks Like JSON (opens in new tab)

(ishan.page)

48 pointsishandotpage2y ago12 comments

12 comments

heh, wait until one discovers that gojq <https://github.com/itchyny/gojq#difference-to-jq> accepts yaml input; so, if you happen to have a bunch of _almost_ structured data then some light sprinkling can turn it into yaml and then you're back in the loving embrace of the jq transformation/mutation language :heart:

   $ cat some-vendor-nonsense.txt
   product id: 5
   price: 3.14

   product id: 6
   price: 6.66

   $ sed "s/^/  /; s/^  product/- product/" < some-vendor-nonsense.txt \
     | gojq --yaml-input .

wwader2y ago

Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq :) It's using a fork of gojq btw!

mdaniel2y ago

Oh, I didn't know that; how cool! Have you asked itchyny about their interest in upstreaming the changes <https://github.com/itchyny/gojq/compare/main...wader:gojq:fq>? they don't look completely out of alignment with gojq and it would make reasoning about "gojq version what?" in fq easier and the inverse merge of theirs versus your fork seems to have a substantial changelog entry https://github.com/wader/gojq/compare/fq...itchyny:gojq:main

wwader2y ago

Yeap i've talked to itchyny quite a lot about various changes https://github.com/itchyny/gojq/issues/153 and also upstreamed quite a lot https://github.com/itchyny/gojq/issues?q=author%3Awader like custom iterators (to allow eval, own iterators and "empty" functions), query marshalling (query rewrite tricks) and a bunch of small things and bug fixes. But the largest change to add a JQValue interface is quite complex, other changes like extended literals is also a bit tricky.

Hmm weird list of changes for https://github.com/wader/gojq/compare/fq...itchyny:gojq:main but i guess it is because i haven't kept my main branch in sync. The fq branch should be based on latest gojq/main as of now. I usually try to rebase as quick as possible.

Let me know if you have any other questions or want to help out! maybe email etc as i usually don't check HN comments replies that often :)

lathiat2y ago

Oh, that is great.. I was using dasel to convert yaml to json first ha ha.

madduci2y ago

Or just use "yq", if you need to parse yaml

mdaniel2y ago

You somehow thought that my problem was parsing yaml, when I thought I made it abundantly clear that it was jq's power that was my interest; yq has its own syntax and is not the jqlang that I use

rnemeth912y ago

If you'd like to use jq in any JetBrains IDE (e.g. IntelliJ), here's a plugin for it: https://plugins.jetbrains.com/plugin/23360-jqexpress

QuercusMax2y ago

At some point in the last 25 years I learned "perl -pi -e" and "for x in [whatever]; do [something] $x; done" and I type those without thinking when I'm working in the shell. I haven't written any perl in probably 12 years, but I still use it for file manipulation.

mpalmer2y ago

jq has string interpolation too, so it can even get a little bit nicer:

    "<option value=\"/examples/\(.)/\">\(.)</option>"

mdaniel2y ago

Yeah, but muscle memory bites me all the time and I put the backslash on the closing paren, too, because I'm so used to the regex usage of that syntax which needs them to match

I also want to draw the reader's attention to the magic of |@uri <https://github.com/jqlang/jq/blob/jq-1.7/docs/content/manual...> for a bunch of cases, but doubly so in TFA's case where they're plugging strings into a URI context. Simple string concat often works great for "hello world", but the world is not always just hello, so one quick use of the filter and jq's got your back

  echo "the world's scary" | jq -Rr '"<option value=\"/examples/\(.|@uri)\">\(.)</value>"'

sam0x172y ago

> Back in the Bootstrap 3 days, there used to be a customizer.

There is still a quite good bootstrap customizer at bootstrap.build btw ;)

j / k navigate · click thread line to collapse

12 comments

mdaniel2y ago

   $ cat some-vendor-nonsense.txt
   product id: 5
   price: 3.14

   product id: 6
   price: 6.66

   $ sed "s/^/  /; s/^  product/- product/" < some-vendor-nonsense.txt \
     | gojq --yaml-input .

wwader2y ago

Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq :) It's using a fork of gojq btw!

mdaniel2y ago

wwader2y ago

Let me know if you have any other questions or want to help out! maybe email etc as i usually don't check HN comments replies that often :)

lathiat2y ago

Oh, that is great.. I was using dasel to convert yaml to json first ha ha.

madduci2y ago

Or just use "yq", if you need to parse yaml

mdaniel2y ago

You somehow thought that my problem was parsing yaml, when I thought I made it abundantly clear that it was jq's power that was my interest; yq has its own syntax and is not the jqlang that I use

rnemeth912y ago

If you'd like to use jq in any JetBrains IDE (e.g. IntelliJ), here's a plugin for it: https://plugins.jetbrains.com/plugin/23360-jqexpress

QuercusMax2y ago

mpalmer2y ago

jq has string interpolation too, so it can even get a little bit nicer:

    "<option value=\"/examples/\(.)/\">\(.)</option>"

mdaniel2y ago

Yeah, but muscle memory bites me all the time and I put the backslash on the closing paren, too, because I'm so used to the regex usage of that syntax which needs them to match

  echo "the world's scary" | jq -Rr '"<option value=\"/examples/\(.|@uri)\">\(.)</value>"'

sam0x172y ago

> Back in the Bootstrap 3 days, there used to be a customizer.

There is still a quite good bootstrap customizer at bootstrap.build btw ;)

j / k navigate · click thread line to collapse