Is there a query language for JSON?

Viewed 188758

Is there a (roughly) SQL or XQuery-like language for querying JSON?

I'm thinking of very small datasets that map nicely to JSON where it would be nice to easily answer queries such as "what are all the values of X where Y > 3" or to do the usual SUM / COUNT type operations.

As completely made-up example, something like this:

[{"x": 2, "y": 0}}, {"x": 3, "y": 1}, {"x": 4, "y": 1}]

SUM(X) WHERE Y > 0     (would equate to 7)
LIST(X) WHERE Y > 0    (would equate to [3,4])

I'm thinking this would work both client-side and server-side with results being converted to the appropriate language-specific data structure (or perhaps kept as JSON)

A quick Googling suggests that people have thought about it and implemented a few things (JAQL), but it doesn't seem like a standard usage or set of libraries has emerged yet. While each function is fairly trivial to implement on its own, if someone has already done it right I don't want to re-invent the wheel.

Any suggestions?

Edit: This may indeed be a bad idea or JSON may be too generic a format for what I'm thinking.. The reason for wanting a query language instead of just doing the summing/etc functions directly as needed is that I hope to build the queries dynamically based on user-input. Kinda like the argument that "we don't need SQL, we can just write the functions we need". Eventually that either gets out of hand or you end up writing your own version of SQL as you push it further and further. (Okay, I know that is a bit of a silly argument, but you get the idea..)

25 Answers

Sure, how about:

They all seem to be a bit work in progress, but work to some degree. They are also similar to XPath and XQuery conceptually; even though XML and JSON have different conceptual models (hierarchic vs object/struct).

EDIT Sep-2015: Actually there is now JSON Pointer standard that allows very simple and efficient traversal of JSON content. It is not only formally specified, but also supported by many JSON libraries. So I would call it actual real useful standard, although due to its limited expressiveness it may or may not be considered Query Language per se.

I'd recommend my project I'm working on called jLinq. I'm looking for feedback so I'd be interested in hearing what you think.

If lets you write queries similar to how you would in LINQ...

var results = jLinq.from(records.users)

    //you can join records
    .join(records.locations, "location", "locationId", "id")

    //write queries on the data
    .startsWith("firstname", "j")
    .or("k") //automatically remembers field and command names

    //even query joined items
    .equals("location.state", "TX")

    //and even do custom selections
    .select(function(rec) {
        return {
            fullname : rec.firstname + " " + rec.lastname,
            city : rec.location.city,
            ageInTenYears : (rec.age + 10)
        };
    });

It's fully extensible too!

The documentation is still in progress, but you can still try it online.

If you are using .NET then Json.NET supports LINQ queries over the top of JSON. This post has some examples. It supports filtering, mapping, grouping, etc.

if you want to use pure javascript try this:

var object = { result: { data: { point1: "x", value: 2 }, foo: { bar: 7 } } },
    path = 'result.data.value',
    getValue = (o, p) => p.split('.').reduce((r, k) => r[k], o);

console.log(getValue(object, path));

Check out https://github.com/niclasko/Cypher.js (note: I'm the author)

It's a zero-dependency Javascript implementation of the Cypher graph database query language along with a graph database. It runs in the browser (tested with Firefox, Chrome, IE).

With relevance to the question. It can be used to query JSON endpoints:

load json from "http://url/endpoint" as l return l limit 10

Here's an example of querying a complex JSON document and performing analysis on it:

Cypher.js JSON query example

You could use linq.js.

This allows to use aggregations and selectings from a data set of objects, as other structures data.

var data = [{ x: 2, y: 0 }, { x: 3, y: 1 }, { x: 4, y: 1 }];

// SUM(X) WHERE Y > 0     -> 7
console.log(Enumerable.From(data).Where("$.y > 0").Sum("$.x"));

// LIST(X) WHERE Y > 0    -> [3, 4]
console.log(Enumerable.From(data).Where("$.y > 0").Select("$.x").ToArray());
<script src="https://cdnjs.cloudflare.com/ajax/libs/linq.js/2.2.0.2/linq.js"></script>

PythonQL offers an embedded syntax that IMHO is an improvement on SQL, principally because group, window, where, let, etc. can be freely intermixed.

$ cat x.py
#coding: pythonql
data = [{"x": 2, "y": 0}, {"x": 3, "y": 1}, {"x": 4, "y": 1}]
q = [x match {'x': as x, 'y': as y} in data where y > 0]
print(sum(q))
print(list(q))

q = [x match {'x': as x, 'y': as y} as d in data where d['y'] > 0]
print(sum(q))

This code shows two different answers to your question, depending on your need to handle the entire structure or just the value. Execution gives you the expected result.

$ python x.py
7
[3, 4]
7

I'll second the notion of just using your own javascript, but for something a bit more sophisticated you might look at dojo data. Haven't used it but it looks like it gives you roughly the kind of query interface you're looking for.

The current Jaql implementation targets large data processing using a Hadoop cluster, so it might be more than you need. However, it runs easily without a Hadoop cluster (but still requires the Hadoop code and its dependencies to get compiled, which are mostly included). A small implementation of Jaql that could be embedded in Javascript and the a browser would be a great addition to the project.

Your examples above are easily written in jaql:

$data = [{"x": 2, "y": 0}, {"x": 3, "y": 1}, {"x": 4, "y": 1}];

$data -> filter $.y > 0 -> transform $.x -> sum(); // 7

$data -> filter $.y > 0 -> transform $.x; // [3,4]

Of course, there's much more too. For example:

// Compute multiple aggregates and change nesting structure:
$data -> group by $y = $.y into { $y, s:sum($[*].x), n:count($), xs:$[*].x}; 
    // [{ "y": 0, "s": 2, "n": 1, "xs": [2]   },
    //  { "y": 1, "s": 7, "n": 2, "xs": [3,4] }]

// Join multiple data sets:
$more = [{ "y": 0, "z": 5 }, { "y": 1, "z": 6 }];
join $data, $more where $data.y == $more.y into {$data, $more};
    // [{ "data": { "x": 2, "y": 0 }, "more": { "y": 0, "z": 5 }},
    //  { "data": { "x": 3, "y": 1 }, "more": { "y": 1, "z": 6 }},
    //  { "data": { "x": 4, "y": 1 }, "more": { "y": 1, "z": 6 }}]

Jaql can be downloaded/discussed at http://code.google.com/p/jaql/

If you are on python, There is Mini open source version of MongoDB, MontyDB https://github.com/davidlatwe/montydb which has over 500+ stars on github, and it is supported by JetBrains.

from montydb.utils import MontyList
response = [
    {'namespace': 'dash_mantine_components',
     'props': {'checked': True,
               'id': {'index': 0, 'type': 'checkbox'},
               'label': 'My first to do'},
     'type': 'Checkbox'},
    {'namespace': 'dash_mantine_components',
     'props': {'checked': True,
               'id': {'index': 1, 'type': 'checkbox'},
               'label': 'My Another to do'},
     'type': 'Input'}, 
    {'namespace': 'dash_mantine_components',
     'props': {'checked': False,
               'id': {'index': 2, 'type': 'checkbox'},
               'label': 'My next level to do'},
     'type': 'Div'}, 
]

cli = MontyList(response)
cli.find({'props.checked': True},{'type':1})

Output:

MontyList([{'type': 'Checkbox'}, {'type': 'Input'}])

I like this over other jsonquery options, as this library gives the same feel of MongoDB.

I am the developer of a new query language for JSON, called ~Q (pronounced "unquery"). It was designed to address many of the shortcomings of other JSON query languages, and balance ease of use and expressive power. As the main developer, I am biased, of course, but you can check it out and decide for yourself:

https://github.com/xcite-db/Unquery

As for the OP's examples, the queries in ~Q would be:

{
    "example1:[]" : "$sum(x)?y>0",
    "example2:[]" : ["x?y>0"]
}
Related