jackson-jq

jq for Jackson JSON Processor

Usage

First, you need Java 8 or later.

If you use Maven, add the following snippet to the <dependencies> section of your POM. For instructions for other build tools (Gradle, etc.), visit jackson-jq on search.maven.org.

<dependency>
	<groupId>net.thisptr</groupId>
	<artifactId>jackson-jq</artifactId>
	<version>1.0.0-preview.20191208</version>
</dependency>

See jackson-jq/src/test/java/examples/Usage.java for the API usage.

Using a jackson-jq command line tool

To test a query quickly, we provide jackson-jq CLI.

$ curl -LO https://repo1.maven.org/maven2/net/thisptr/jackson-jq-cli/1.0.0-preview.20191208/jackson-jq-cli-1.0.0-preview.20191208.jar

$ java -jar jackson-jq-cli-1.0.0-preview.20191208.jar --help
usage: jackson-jq [OPTIONS...] QUERY
 -c,--compact      compact instead of pretty-printed output
 -h,--help         print this message
    --jq <arg>     specify jq version
 -n,--null-input   use `null` as the single input value
 -r,--raw          output raw strings, not JSON texts

$ java -jar jackson-jq-cli-1.0.0-preview.20191208.jar '.foo'
{"foo": 42}
42

To test a query with a specific jq version,

$ java -jar jackson-jq-cli-1.0.0-preview.20191208.jar --jq 1.5 'join("-")'
["1", 2]
jq: error: string ("-") and number (2) cannot be added

$ java -jar jackson-jq-cli-1.0.0-preview.20191208.jar --jq 1.6 'join("-")' # jq-1.6 can join any values, not only strings
["1", 2]
"1-2"

Homebrew (or Linuxbrew) users can alternatively run brew tap eiiches/jackson-jq && brew install jackson-jq to install the CLI. jackson-jq will be available on your $PATH.

Branches and versioning

There are currently two development branches.

• develop/1.x: This branch (you are viewing), which is currently under development for the future 1.0 release. The API is not stable yet. You can find preview releases (not stable, not recommended for production) at Releases page (tags: 1.0.0-preview.yyyyMMdd).
• develop/0.x: The development branch for 0.x versions. Features that need breaking API changes will no longer be added. Go to Releases and find the latest 0.x.y version.

PRs can be sent to any of the develop/* branches. The patch will be ported to the other branch(es) if necessary.

We use Semantic Versioning 2.0.0 for Java API versioning, 1.0.0 onwards. A jq behavior fix (even if it may possibly affect users) will not be considered a major change if the fix is to make the bahavior compatible with ./jq; these kind of incompatible changes are documented in the release note.

If you get different results between ./jq and jackson-jq, please file an issue. That is a bug on jackson-jq side.

Implementation status and current limitations

jackson-jq aims to be a compatible jq implementation. However, not all the features are available; some features are not relevant as being a java library and some features are just yet to be implemented.

This table illustrates which features (picked from jq-1.5 manual) are supported and which are not in jackson-jq. We try to keep this list accurate and up to date. If you find something is missing or wrong, please file an issue.

Language Features / Functions	jackson-jq
Basic filters	○
• .	○
• .foo, .foo.bar	○
• .foo?	○
• .[<string>], .[2], .[10:15]	○
• .[]	○
• .[]?	○
• ,	○
• ǀ	○
Types and Values	○
• Array construction - []	○
• Objects - {}	○
Builtin operators and functions	○
• Addition - +	○
• Subtraction - -	○
• Multiplication, division, modulo - *, /, and %	○*5
• length	○
• keys, keys_unsorted	○
• has(key)	○
• in	○
• path(path_expression)	○*7
• del(path_expression)	○
• to_entries, from_entries, with_entries	○
• select(boolean_expression)	○
• arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars	○
• empty	○
• error(message)	○
• $__loc__	×
• map(x), map_values(x)	○
• paths, paths(node_filter), leaf_paths	○
• add	○
• any, any(condition), any(generator; condition)	○
• all, all(condition), all(generator; condition)	○
• flatten, flatten(depth)	○
• range(upto), range(from;upto) range(from;upto;by)	○
• floor	○
• sqrt	○
• tonumber	○
• tostring	○
• type	○
• infinite, nan, isinfinite, isnan, isfinite, isnormal	○
• sort, sort_by(path_expression)	○
• group_by(path_expression)	○
• min, max, min_by(path_exp), max_by(path_exp)	○
• unique, unique_by(path_exp)	○
• reverse	○
• contains(element)	○
• indices(s)	○
• index(s), rindex(s)	○
• inside	○
• startswith(str)	○
• endswith(str)	○
• combinations, combinations(n)	○
• ltrimstr(str)	○
• rtrimstr(str)	○
• explode	○
• implode	○
• split	○
• join(str)	○
• ascii_downcase, ascii_upcase	○
• while(cond; update)	○
• until(cond; next)	○
• recurse(f), recurse, recurse(f; condition), recurse_down	○
• ..	○
• env	○*6
• transpose	○
• bsearch(x)	×
• String interpolation - \(foo)	○
• Convert to/from JSON	○
• Format strings and escaping	○
• Dates	×
Conditionals and Comparisons	○
• ==, !=	○
• if-then-else	○
• >, >=, <=, <	○
• and/or/not	○
• Alternative operator - //	○
• try-catch	○*1
• Breaking out of control structures	○*2
• ? operator	○
Regular expressions (PCRE)	○
• test(val), test(regex; flags)	○
• match(val), match(regex; flags)	○
• capture(val), capture(regex; flags)	○
• scan(regex), scan(regex; flags)	○
• split(regex; flags)	○
• splits(regex), splits(regex; flags)	○
• sub(regex; tostring) sub(regex; string; flags)	○
• gsub(regex; string), gsub(regex; string; flags)	○
Advanced features	○
• Variables	○
• Destructuring Alternative Operator: ?//	✕ (#44)
• Defining Functions	○*3
• Reduce	○
• limit(n; exp)	○
• first(expr), last(expr), nth(n; expr)	○
• first, last, nth(n)	○
• foreach	○
• Recursion	○
• Generators and iterators	○
Math	○
I/O	N/A
• input	N/A
• inputs	N/A
• debug	N/A
• input_filename	N/A
• input_line_number	N/A
Streaming	N/A
• truncate_stream(stream_expression)	N/A
• fromstream(stream_expression)	N/A
• tostream	N/A
Assignment	○
• =	○
• ǀ=	○*8
• +=, -=, *=, /=, %=, //=	○
• Complex assignments	○
Modules	×
• import RelativePathString as NAME [<metadata>];	×
• include RelativePathString [<metadata>];	×
• import RelativePathString as $NAME [<metadata>];	×
• module <metadata>;	×
• modulemeta	×

*1) Error messages differ between jq and jackson-jq and they also tend to change between versions.

*2) Catching a break (try (break $out) catch .) always produces a {__jq: 0} in jackson-jq, while jq produces {__jq: n} where n is the index of the label the break statement tries to jump to. E.g.

• label $a | label $b | try (break $b) catch . evaluates to {"__jq":0} not {"__jq":1}.

*3) When the function with the same name is defined more than once in the same-level scope, jackson-jq uses the last one. E.g.

• def f: 1; def g: f; def f: 2; g evaluates to 2 in jackson-jq, while jq evaluates it to 1.

*5) While jq-1.5 '0 / .' <<< 0 throws a zero-division error as one might expect, jq-1.5 silently evaluates a compile-time constant 0 / 0 division as null.

• jackson-jq just throws an error whenever divisor is zero.

*6) env/0 function is not available by default and must be added manually to the scope.

• SCOPE.addFunction("env", 0, new EnvFunction())

*7) path(.foo as $a | $a) always throws an error as $variables in jackson-jq does not carry path information like jq-1.5 accidentally? did. The behavior is fixed in jq-1.6 whose documentation explicitly states them as "not a valid or useful path expression". So, I dicided not to implement it even in jq-1.5 compatible mode.

*8) .foo |= empty always throws an error in jackson-jq instead of producing an unexpected result. jq-1.5 and jq-1.6 respectively produces a different and incorrect result for [1,2,3] | ((.[] | select(. > 1)) |= empty). jq#897 says "empty in the RHS is undefined". You can still use _modify/2 directly if you really want to emulate the exact jq-1.5 or jq-1.6 behavior.

Using jackson-jq-extra module

jackson-jq-extra module provides extra functions that you might find useful. These functions do not exist in jq.

<dependency>
	<groupId>net.thisptr</groupId>
	<artifactId>jackson-jq-extra</artifactId>
	<version>1.0.0-preview.20191208</version>
</dependency>

Examples

uuid4/0

• jackson-jq -n 'uuid4' #=> "a69cf146-f40e-42e1-ae88-12590bdae947"

random/0

• jackson-jq -n 'random' #=> 0.43292159535427466

timestamp/0, strptime/{1, 2}, strftime/{1, 2}

• jackson-jq -n 'timestamp' #=> 1477162056362
• jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX")' #=> "2016-10-23 03:52:22.372+09:00"
• jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX"; "UTC")' #=> "2016-10-22 18:52:22.372Z"
• jackson-jq -n '"2016-10-23 03:52:22.372+09:00" | strptime("yyyy-MM-dd HH:mm:ss.SSSXXX")' #=> 1477162342372
• jackson-jq -n '"2016-10-22 18:52:22.372" | strptime("yyyy-MM-dd HH:mm:ss.SSS"; "UTC")' #=> 1477162342372

uriparse/0

• jackson-jq -n '"http://user@www.example.com:8080/index.html?foo=1&bar=%20#hash" | uriparse' #=>

  {
    "scheme" : "http",
    "user_info" : "user",
    "raw_user_info" : "user",
    "host" : "www.example.com",
    "port" : 8080,
    "authority" : "user@www.example.com:8080",
    "raw_authority" : "user@www.example.com:8080",
    "path" : "/index.html",
    "raw_path" : "/index.html",
    "query" : "foo=1&bar= ",
    "raw_query" : "foo=1&bar=%20",
    "query_obj" : {
      "bar" : " ",
      "foo" : "1"
    },
    "fragment" : "hash",
    "raw_fragment" : "hash"
  }

uridecode/0

• jackson-jq -n '"%66%6f%6f" | uridecode' #=> "foo"

hostname/0

• jackson-jq -n 'hostname' #=> "jenkins-slave01"

Contributing

• If you are planning to send a PR and the change is not small, please open an issue and discuss it with the authors first.
• Other than bug reports or patches, documentation improvements (including small grammatical or wording corrections) would be greatly appreciated.

License

This software is licensed under Apache Software License, Version 2.0, with some exceptions:

• jackson-jq/src/test/resources contains test cases from stedolan/jq.
• jackson-jq/src/main/resources/net/thisptr/jackson/jq/jq.json contains function definitions extracted from stedolan/jq.

See COPYING for details.