How to manage module dependencies in jq

user_373710 0
Viewed 465

Since version 1.5 the jq data processing language has a library module system. A module consists of optional metadata and a set of functions. For instance

module { name: "util", version: "1.0.0" };
def digitsum: tostring|split("")|map(tonumber)|add;

stored as file util.jq can be used like this:

$ echo '789' | jq -L. 'include "util"; digitsum'
24

Modules can use other modules and the dependencies are tracked by the modulemeta directive but how to express and check for a minimum version of a module number? For instance:

module {
  name: "math",
  version: "0.1.0",
};

include "util"; # TODO: require at least version 1.0.0!

def digitroot:
  (.|digitsum) as $sum |
  if $sum<10 then $sum else $sum|digitroot end;
jq dependency-management module
1 Answers

The support for modules in jq is currently (June 2019) still very minimal, though on github there is a module management system for jq: https://github.com/joelpurra/jqnpm

Without using such an external module management system, what can be done in jq itself? Extending the given example, the following illustrates one approach to supporting version requirements. Notice the additional key named dependencies in the metadata of the math module. (Currently, this key cannot be named deps as jq overwrites it.)

Files

dependencies.jq

# Recursively check specified version constraints 

module { name: "dependencies", version: "0.0.2" };

# parents of a module as defined by its .deps
def parents:
  . as $in
  | if type == "array" then map(parents) | add
    else modulemeta | .deps | map(.relpath)
    end ;

# ancestors of a single module or an array of modules.
# The array of "ancestors" of a module includes itself.
def ancestors:
  # input and $visited should be arrays of strings
  def ancestors($visited):
    . as $in
    | ($in - $visited) as $new
    | if $new == [] then $visited
      else $new | parents | ancestors($visited + $new | unique)
      end;
  if type == "array" then . else [.] end
  | ancestors([]) ;

def versionsort:
  def parse:
    sub("(?<a>(alpha|beta|gamma))"; "\(.a).")
    | [splits("[-.]")]
    | map(tonumber? // .) ;
  sort_by(parse);

# Input: a module name
# Emit empty if the constraints for the given module are satisfied, otherwise raise an error
def dependencies($version):
  def le($y):  (. == $y) or ([.,$y] | . == versionsort);

  modulemeta
  | .version as $mv
  | if (($mv == null) or ($version | le($mv))) then empty
    else ("module \(.name) dependencies version \($version) vs \($mv)" | error)
    end ;

# Input: a module name or array of module names
# Check the module-version dependencies in .dependencies, proceeding up the chain as defined by .deps
def dependencies:
  def check:
    modulemeta
    | select(has("dependencies"))
    | all( .dependencies | to_entries[];
           .key as $m | .value as $v | ($m | dependencies($v) ))
    | empty;
  ancestors[] | check;

util.jq

module { name: "util", version: "1.0.0" };

def digitsum: tostring|split("")|map(tonumber)|add;

math.jq

module { 
   name: "math",
   version: "0.1.0",
   dependencies: {"util": "1.0.0"} };

include "util" ;

def digitroot:
  digitsum as $sum
  | if $sum<10 then $sum
    else $sum|digitroot
    end;

Invocation

jq -n -L . '
  include "dependencies";
  include "math";

  "math" | dependencies,
  (123|digitroot) '
user_9973580
Related