Is there a (Unix) shell script to format JSON in human-readable form?
Basically, I want it to transform the following:
{ "foo": "lorem", "bar": "ipsum" }
... into something like this:
{
"foo": "lorem",
"bar": "ipsum"
}
Is there a (Unix) shell script to format JSON in human-readable form?
Basically, I want it to transform the following:
{ "foo": "lorem", "bar": "ipsum" }
... into something like this:
{
"foo": "lorem",
"bar": "ipsum"
}
With Python 2.6+ you can do:
echo '{"foo": "lorem", "bar": "ipsum"}' | python -m json.tool
or, if the JSON is in a file, you can do:
python -m json.tool my_json.json
if the JSON is from an internet source such as an API, you can use
curl http://my_url/ | python -m json.tool
For convenience in all of these cases you can make an alias:
alias prettyjson='python -m json.tool'
For even more convenience with a bit more typing to get it ready:
prettyjson_s() {
echo "$1" | python -m json.tool
}
prettyjson_f() {
python -m json.tool "$1"
}
prettyjson_w() {
curl "$1" | python -m json.tool
}
for all the above cases. You can put this in .bashrc and it will be available every time in shell. Invoke it like prettyjson_s '{"foo": "lorem", "bar": "ipsum"}'.
Note that as @pnd pointed out in the comments below, in Python 3.5+ the JSON object is no longer sorted by default. To sort, add the --sort-keys flag to the end. I.e. ... | python -m json.tool --sort-keys.
If you use npm and Node.js, you can do npm install -g json and then pipe the command through json. Do json -h to get all the options. It can also pull out specific fields and colorize the output with -i.
curl -s http://search.twitter.com/search.json?q=node.js | json
Thanks to J.F. Sebastian's very helpful pointers, here's a slightly enhanced script I've come up with:
#!/usr/bin/python
"""
Convert JSON data to human-readable form.
Usage:
prettyJSON.py inputFile [outputFile]
"""
import sys
import simplejson as json
def main(args):
try:
if args[1] == '-':
inputFile = sys.stdin
else:
inputFile = open(args[1])
input = json.load(inputFile)
inputFile.close()
except IndexError:
usage()
return False
if len(args) < 3:
print json.dumps(input, sort_keys = False, indent = 4)
else:
outputFile = open(args[2], "w")
json.dump(input, outputFile, sort_keys = False, indent = 4)
outputFile.close()
return True
def usage():
print __doc__
if __name__ == "__main__":
sys.exit(not main(sys.argv))
On *nix, reading from stdin and writing to stdout works better:
#!/usr/bin/env python
"""
Convert JSON data to human-readable form.
(Reads from stdin and writes to stdout)
"""
import sys
try:
import simplejson as json
except:
import json
print json.dumps(json.loads(sys.stdin.read()), indent=4)
sys.exit(0)
Put this in a file (I named mine "prettyJSON" after AnC's answer) in your PATH and chmod +x it, and you're good to go.
The JSON Ruby Gem is bundled with a shell script to prettify JSON:
sudo gem install json
echo '{ "foo": "bar" }' | prettify_json.rb
Script download: gist.github.com/3738968
$ echo '{ "foo": "lorem", "bar": "ipsum" }' \
> | python -c'import fileinput, json;
> print(json.dumps(json.loads("".join(fileinput.input())),
> sort_keys=True, indent=4))'
{
"bar": "ipsum",
"foo": "lorem"
}
NOTE: It is not the way to do it.
The same in Perl:
$ cat json.txt \
> | perl -0007 -MJSON -nE'say to_json(from_json($_, {allow_nonref=>1}),
> {pretty=>1})'
{
"bar" : "ipsum",
"foo" : "lorem"
}
Note 2: If you run
echo '{ "Düsseldorf": "lorem", "bar": "ipsum" }' \
| python -c'import fileinput, json;
print(json.dumps(json.loads("".join(fileinput.input())),
sort_keys=True, indent=4))'
the nicely readable word becomes \u encoded
{
"D\u00fcsseldorf": "lorem",
"bar": "ipsum"
}
If the remainder of your pipeline will gracefully handle unicode and you'd like your JSON to also be human-friendly, simply use ensure_ascii=False
echo '{ "Düsseldorf": "lorem", "bar": "ipsum" }' \
| python -c'import fileinput, json;
print json.dumps(json.loads("".join(fileinput.input())),
sort_keys=True, indent=4, ensure_ascii=False)'
and you'll get:
{
"Düsseldorf": "lorem",
"bar": "ipsum"
}
You can use this simple command to achieve the result:
echo "{ \"foo\": \"lorem\", \"bar\": \"ipsum\" }"|python -m json.tool
Check out Jazor. It's a simple command line JSON parser written in Ruby.
gem install jazor
jazor --help
Simply pipe the output to jq ..
Example:
twurl -H ads-api.twitter.com '.......' | jq .
You can simply use standard tools like jq or json_pp.
echo '{ "foo": "lorem", "bar": "ipsum" }' | json_pp
or
echo '{ "foo": "lorem", "bar": "ipsum" }' | jq
will both prettify output like the following (jq even more colorful):
{
"foo": "lorem",
"bar": "ipsum"
}
The huge advantage of jq is that it can do A LOT more if you'd like to parse and process the json.
you only need to use jq If jq is not installed then you need to installed jq first.
sudo apt-get update
sudo apt-get install jq
After installed jq then only need to use jq
echo '{ "foo": "lorem", "bar": "ipsum" }' | jq
Output looks like
{
"foo": "lorem",
"bar": "ipsum"
}
I recommend using the json_xs command line utility which is included in the JSON::XS perl module. JSON::XS is a Perl module for serializing/deserializing JSON, on a Debian or Ubuntu machine you can install it like this:
sudo apt-get install libjson-xs-perl
It is obviously also available on CPAN.
To use it to format JSON obtained from a URL you can use curl or wget like this:
$ curl -s http://page.that.serves.json.com/json/ | json_xs
or this:
$ wget -q -O - http://page.that.serves.json.com/json/ | json_xs
and to format JSON contained in a file you can do this:
$ json_xs < file-full-of.json
To reformat as YAML, which some people consider to be more humanly-readable than JSON:
$ json_xs -t yaml < file-full-of.json
bat is a cat clone with syntax highlighting:
Example:
echo '{"bignum":1e1000}' | bat -p -l json
-p will output without headers, and -l will explicitly specify the language.
It has colouring and formatting for JSON and does not have the problems noted in this comment: How can I pretty-print JSON in a shell script?
When you have node installed on your system the following works.
echo '{"test":1,"test2":2}' | npx json
{
"test": 1,
"test2": 2
}
Use Ruby in one line:
echo '{"test":1,"test2":2}' | ruby -e "require 'json'; puts JSON.pretty_generate(JSON.parse(STDIN.read))"
And you can set an alias for this:
alias to_j="ruby -e \"require 'json';puts JSON.pretty_generate(JSON.parse(STDIN.read))\""
Then you can use it more conveniently
echo '{"test":1,"test2":2}' | to_j
{
"test": 1,
"test2": 2
}
And if you want display JSON with color, your can install awesome_print,
gem install awesome_print
then
alias to_j="ruby -e \"require 'json';require 'awesome_print';ap JSON.parse(STDIN.read)\""
Try it!
echo '{"test":1,"test2":2, "arr":["aa","bb","cc"] }' | to_j
A one-line solution using Node.js will look like this:
$ node -e "console.log( JSON.stringify( JSON.parse(require('fs').readFileSync(0) ), 0, 1 ))"
For example:
$ cat test.json | node -e "console.log( JSON.stringify( JSON.parse(require('fs').readFileSync(0) ), 0, 1 ))"
TL;DR: for performances, use jj -p < my.json.
I took some solutions here and benchmarked them with the next dummy script:
function bench {
time (
for i in {1..1000}; do
echo '{ "foo" : { "bar": { "dolorem" : "ipsum", "quia" : { "dolor" : "sit"} } } }' \
| $@ > /dev/null
done
)
}
Here's the result on my mac (32 GB, Apple M1 Max, YMMV):
bench python -m json.tool
# 8.39s user 12.31s system 42% cpu 48.536 total
bench jq
# 13.12s user 1.28s system 87% cpu 16.535 total
bench bat -p -l json # NOTE: only syntax colorisation.
# 1.87s user 1.47s system 66% cpu 5.024 total
bench jj -p
# 1.94s user 2.44s system 57% cpu 7.591 total
bench xidel -s - -e '$json' --printed-json-format=pretty
# 4.32s user 1.89s system 76% cpu 8.101 total
Thanks @peak and your answer for this discovery of jj!
There is TidyJSON.
It's C#, so maybe you can get it to compile with Mono, and working on *nix. No guarantees though, sorry.
J.F. Sebastian's solutions didn't work for me in Ubuntu 8.04.
Here is a modified Perl version that works with the older 1.X JSON library:
perl -0007 -MJSON -ne 'print objToJson(jsonToObj($_, {allow_nonref=>1}), {pretty=>1}), "\n";'
I've came up with this solution: https://calbertts.medium.com/unix-pipelines-with-curl-requests-and-serverless-functions-e21117ae4c65
# this in your bash profile
jsonprettify() {
curl -Ss -X POST -H "Content-Type: text/plain" --data-binary @- https://jsonprettify.vercel.app/api/server?indent=$@
}
echo '{"prop": true, "key": [1,2]}' | jsonprettify 4
# {
# "prop": true,
# "key": [
# 1,
# 2
# ]
# }
There's no need to install anything, if you have an internet connection and cURL installed, you can use this function.
Are you in another host where you can't install anything, this would be a perfect solution to that issue.
If you don't mind using a third-party tool, you can simply curl to jsonprettyprint.org. This is for the case where you can't install packages on the machine.
curl -XPOST https://jsonprettyprint.org/api -d '{"user" : 1}'
My JSON files were not parsed by any of these methods.
My problem was similar to the post Is Google data source JSON not valid?.
The answer to that post helped me find a solution.
It is considered to be invalid JSON without the string keys.
{id:'name',label:'Name',type:'string'}
must be:
{"id": "name", "label": "Name", "type": "string"}
This link gives a nice comprehensive comparison of some of the different JSON parsers: http://deron.meranda.us/python/comparing_json_modules/basic
Which led me to http://deron.meranda.us/python/demjson/. I think this one parser is much more fault tolerant than many others.
You can use xidel.
Xidel is a command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern templates. It can also create new or transformed XML/HTML/JSON documents.
Xidel pretty-prints by default:
$ xidel -se '$json' <<< '{"foo":"lorem","bar":"ipsum"}'
{
"foo": "lorem",
"bar": "ipsum"
}
or:
$ echo '{"foo":"lorem","bar":"ipsum"}' | xidel -se '$json'
{
"foo": "lorem",
"bar": "ipsum"
}
You can use Prettier to do this.
npx prettier <JSON file> should print a prettified version of the JSON in the given file, while npx prettier --write <JSON file> will overwrite the given JSON file with prettified JSON.
If you want to visualize json log at console you can use munia-pretty-json
npm install -g munia-pretty-json
Your json data (app-log.json)
{"time":"2021-06-09T02:50:22Z","level":"info","message":"Log for pretty JSON","module":"init","hostip":"192.168.0.138","pid":123}
{"time":"2021-06-09T03:27:43Z","level":"warn","message":"Here is warning message","module":"send-message","hostip":"192.168.0.138","pid":123}
Run the command:
munia-pretty-json app-log.json
Here is readable output on console:
You can format the output with the template. The default template is '{time} {level -c} {message}'
Using template:
munia-pretty-json -t '{module -c} - {level} - {message}' app-log.json
Output:
Agree about jq. You can add the following function to your $HOME/.bashrc:
jqless () {
args=$1
shift
jq --color-output . $args "$@" | less --raw-control-chars
}
This allows an arbitrary number of input JSON files.