(lispkit json)

Library (lispkit json) defines an API for representing, querying, and manipulating generic JSON values. The framework includes:

  • A representation for mutable and immutable JSON values as defined by RFC 8259

  • Functionality for creating and manipulating JSON values including support for reading and writing JSON data

  • An implementation of JSON Pointer as defined by RFC 6901 for locating values within a JSON document

  • An implementation of JSON Path as defined by RFC 9535 for querying JSON data

  • An implementation of JSON Patch as defined by RFC 6902 for mutating JSON data

  • An implementation of JSON Merge Patch as defined by RFC 7396 for merging JSON data with JSON patches

JSON values

Library (lispkit json) provides an abstract data type json encapsulating potentially very large JSON values. A JSON value has one of the following six types:

  • null: the "empty" value

  • boolean: representing #t and #f

  • number: either an integer (fixnum) or a floating-point number (flonum)

  • string: a sequence of unicode characters

  • array: a fixed length sequence of JSON values

  • object: a collection of name/value pairs

Library (lispkit json) implements a rich API for creating, accessing, and transforming JSON values. There are both mutable and immutable JSON values.

Symbol representing the json type. The type-for procedure of library (lispkit type) returns this symbol for all JSON values.

Returns #t if obj is a mutable or immutable JSON value; #f otherwise. If argument strict? is provided and set to true, then #t is returned only if obj is an immutable JSON value. For checking if a given object is a mutable JSON value, procedure mutable-json? can be used.

Returns #t if obj corresponds to the JSON null value; #f otherwise.

Returns #t if obj is a boolean JSON value; #f otherwise.

Returns #t if obj is a numeric JSON value, i.e. an integer or floating-point number. Otherwise, #f is returned.

Returns #t if obj is a JSON string value; #f otherwise.

Returns #t if obj is a JSON array; #f otherwise.

Returns #t if obj is a JSON value; #f otherwise.

Procedure json maps Scheme data structures to immutable JSON values. If no argument is provided, then the JSON null value is returned. If one argument x is provided, then x is mapped to a JSON value following the rules stated below. If more than one arguments x ... is provided, then a JSON array is returned whose elements have been created by mapping the corresponding argument.

The following mapping rules are being used for regular Scheme values x:

  • The symbol null is mapped to the JSON null value

  • #f and #t are mapped to the corresponding JSON boolean values

  • Fixnum values are mapped to corresponding JSON integer values

  • Flonum values are mapped to corresponding JSON floating-point values

  • Symbols (other than null) are mapped to JSON string values

  • Vectors and growable vectors are mapped to JSON arrays

  • Association lists are mapped to JSON objects; association lists need to have the form (("key1" . value1)("key2" . value2) ...) or ((key1 . value1)(key2 . value2) ...).

  • Instances of record types are mapped to JSON objects where each record field and value correspond to a key and mapped JSON value

  • Hashtables are mapped to a corresponding JSON object if all keys can be mapped to JSON object keys (symbols and strings) and values can be mapped according to these rules

  • JSON values map to itself

  • Mutable JSON values map to a corresponding immutable JSON value

  • For all other Scheme values, an error is signaled

The inverse mapping is implemented by procedure json->value.

Returns a new immutable JSON array of length len. If JSON value default is provided, then it is used as the default element value of the new array. Otherwise, all elements of the new JSON array are set to the JSON null value.

Returns #t if all JSON values json ... are structurally equivalent; otherwise #f is returned.

Returns #t if JSON value a is a refinement of JSON value b; otherwise, #f is returned. a is a refinement of b if

  1. Both a and b are JSON values of the same type,

  2. If a and b are arrays, they have the same length n and ai is a refinement of bi holds for every i ∈ [0; n[,

  3. If a and b are objects, for every member m of b with value bm, there is a member m of a with value am such that am is a refinement of bm,

  4. For all other types, a and b are the same.

This relationship intuitively models that whenever it is possible to read a value at a given location from b, it is also possible to read a value at the same location from a and the value that is read for a is a refinement of the value read from b. The following example showcases this relationship:

(define a
  (string->json "{ \"a\": [1, { \"b\": 2 }], \"c\": { \"d\": [{}] }}"))
(define b
  (string->json "{ \"a\": [1, { \"b\": 2, \"e\": 4 }], \"c\": { \"d\": [{\"f\": 5}] }}"))
(json-refinement? a b)  ⇒  #f
(json-refinement? b a)  ⇒  #t

Returns a JSON value for the data structure represented in string str.

Decodes the given bytevector bvec between start and end and returns a JSON value for the encoded value. If is an error if bvec between start and end does not represent a JSON value encoded in a UTF8-encoded string. If end is not provided, it is assumed to be the length of bvec. If start is not provided, it is assumed to be 0.

Loads a text file at path, parses its content as JSON and returns it as a JSON value.

If json represents a JSON object, then json-members returns a list of all members of this object. Each member is represented as a symbol. For all other JSON values, json-members returns an empty list.

Returns all the direct children of json as a list. For null, boolean, numeric and string values, an empty list is returned. For JSON arrays, the array itself is returned. For JSON objects, the values of the object (without their corresponding keys) are returned in an undefined order.

For JSON objects, json-children-count returns the number of members. For JSON arrays, json-children-count returns the length of the array. For all other JSON types, zero is returned.

Applies JSON references ref ... sequentially to json and returns the result of the last application. (json-ref x r1 ... rn) is equivalent to (json-ref ... (json-ref (json-ref x r1) r2) ... rn).

Within json, replaces the value at JSON reference ref1 with v1 followed by replacing the value at JSON reference ref2 with v2 etc. This procedure does not mutate json returning a new JSON value with the changes applied.

Applies update list updates to json. An update list is a list of pairs consisting of a JSON reference and a JSON value. The values replace the content at the given reference. They are applied in order. This procedure does not mutate json returning a new JSON value with the changes applied.

Converts a JSON value into corresponding Scheme values. This procedure implements the inverse mapping of procedure json, i.e. null values are mapped to the symbol null, boolean values are mapped to #t and #f, numbers are mapped to fixnum and flonum values, strings are mapped to regular Scheme strings, arrays are mapped to immutable vectors, and objects are mapped to association lists with symbols representing the member names.

Returns a string representation of json. The output is pretty-printed if pretty? is provided and set to true. If sort? is provided and set to true, the members of an object are printed in sorted order allowing for a deterministic output. If slash? is provided and set to true, slashes get escaped in strings, allowing outputted JSON to be safely embedded within HTML/XML.

Returns a bytevector of a UTF8-encoded string representation of json. The output options pretty?, sort?, and slash? correspond to the options of procedure json->string.

Applies procedure f to every element of JSON array arr. f is a procedure accepting one JSON value as argument.

Applies procedure f to every member of JSON object obj. f is a procedure accepting two arguments: a symbol representing the member name and a JSON value representing the value of the object member.

Mutable JSON values

Symbol representing the json type. The type-for procedure of library (lispkit type) returns this symbol for all JSON values.

Returns #t if obj is a mutable JSON value; #f otherwise.

Procedure mutable-json maps Scheme data structures expr to mutable JSON values using the same mapping rules as json. This procedure can also be used to turn an immutable JSON value into a mutable JSON value if expr is an immutable JSON value already. The value only gets copied if there are other references to expr. If expr is already a mutable JSON value, then that value is returned by mutable-json unless argument force? is provided and set to true. In this case, even if expr is a mutable JSON value already, a copy is created and returned.

Within json, sets the value at JSON reference ref to value, mutating json in place. If json is not a mutable JSON value, then an error is signaled.

This procedure appends the JSON values x ... to the JSON array at the JSON reference ref of mutable JSON value json. If ref does not refer to an array, no change is made to json and no error is signaled.

This procedure inserts the JSON values x ... in the JSON array at the JSON reference ref of mutable JSON value json. If ref does not refer to an array, no change is made to json and no error is signaled.

Removes the values at the JSON references ref ... from mutable JSON value json. If a JSON reference does not refer to location where a value can be removed, no change is made and no error is signaled.

JSON references

Supported formalisms

Library (lispkit json) supports multiple abstractions for referring to values within a JSON document. These abstractions are called JSON references. The most established formalism for referring to a location within a JSON document is JSON Pointer. Here is the complete list of supported JSON references:

  • Strings containing a valid JSON Pointer reference as defined by RFC 6901; e.g. "/store/book/0/title" is a valid JSON reference.

  • Strings containing a valid JSON Location reference. JSON location syntax is based on how values are uniquely identified in JSON Path.

  • A fixnum value i refers to the i-th element of a JSON array if i >= 0. If i < 0, then this refers to the n+i-th element assuming n is the length of the array.

  • A symbol s refers to member s of a JSON object.

  • The empty list () refers to the root of a JSON document.

  • A list of symbols and integers refers to a sequence of member and array index selections; e.g. (store book 0 title).

JSON locations

It is recommended to use JSON locations were possible since their semantics is independent of the JSON value they are applied to (unlike JSON Pointer references which have an ambiguous interpretation for their numeric segments).

A JSON location is a path to an element in a JSON structure. Each element of the path is called a segment. The JSON location syntax supports two different forms to express such sequences of segments. Each sequence starts with $ indicating the "root" of a JSON document. The most common form for expressing the segment sequence is using the dot notation:

$.store.book[0].title

While accessing an array index is always done using bracket notation, it is possible to also express the access of members of an object using bracket notation as well:

$['store']['book'][0]['title']

Is is also possible to mix the dot and bracket notation. Dots are only used before property names and never together with brackets:

$['store'].book[-1].title

The previous example also shows the usage of negative indices, which are interpreted as offsets from the end of arrays with -1 referring to the last element.

API

Returns #t if string str contains a valid JSON location specification; #f otherwise.

Returns #t if string str contains a valid JSON pointer specification; #f otherwise.

Returns #t if obj is a valid JSON reference; i.e. it is either:

  • A string containing a valid JSON Pointer reference,

  • A string containing a valid JSON Location reference,

  • A fixnum value i referring to the i-th element of a JSON array if i >= 0, or to the n+i-th element if i < 0 and n being the length of the array,

  • A symbol s referring to member s of a JSON object,

  • A list of symbols and integers referring to a sequence of member and array index selections.

Returns #t if obj is a valid JSON reference that refers to the root of a JSON document; #f otherwise.

Returns a string with a JSON location representation of the JSON reference ref.

Returns a string with a JSON pointer representation of the JSON reference ref.

(json-pointer "$.store.book[0].title")
⇒  "/store/book/0/title"

Returns a list of symbols and integers representing the sequence of segments for the given JSON reference ref.

(json-reference-segments "$.store.book[0].title")
⇒  ("store" "book" 0 "title")
(json-reference-segments "/store/book/0/title")
⇒  ("store" "book" 0 "title")
(json-reference-segments '(a -1 c 2))
⇒  ("a" -1 "c" 2)

JSON Path

The full JSON Path standard as defined by RFC 9535 is supported by library (lispkit json). JSON Path queries are simply represented as strings. They can be applied to JSON values with procedures json-query, json-query-results, and json-query-locations. To illustrate the usage of JSON Path queries, the following JSON value is being defined:

(define jval (string->json (string-append
  "{ \"store\": {\n"
  "   \"book\": [\n"
  "     { \"category\": \"reference\",\n"
  "       \"author\": \"Nigel Rees\","
  "       \"title\": \"Sayings of the Century\",\n"
  "       \"price\": 8.95 },\n"
  "     { \"category\": \"fiction\",\n"
  "       \"author\": \"Evelyn Waugh\",\n"
  "       \"title\": \"Sword of Honour\",\n"
  "       \"price\": 12.99 },\n"
  "     { \"category\": \"fiction\",\n"
  "       \"author\": \"Herman Melville\",\n"
  "       \"title\": \"Moby Dick\",\n"
  "       \"isbn\": \"0-553-21311-3\",\n"
  "       \"price\": 8.99 },\n"
  "     { \"category\": \"fiction\",\n"
  "       \"author\": \"J. R. R. Tolkien\",\n"
  "       \"title\": \"The Lord of the Rings\",\n"
  "       \"isbn\": \"0-395-19395-8\",\n"
  "       \"price\": 22.99 }\n"
  "   ],\n"
  "     \"bicycle\": {\n"
  "     \"color\": \"red\",\n"
  "     \"price\": 399\n"
  "   }\n"
  " }\n"
  "}")))

Now a JSON Path query $.store.book[?@.price < 10].title can be applied to jval via procedure json-query to extract pairs of matching JSON values and the corresponding JSON references:

(json-query jval "$.store.book[?@.price < 10].title")
⇒  ((#<json "Moby Dick"> "store" "book" 2 "title")
    (#<json "Sayings of the Century"> "store" "book" 0 "title"))

Returns #t if string str constitutes a valid JSON query string; #f otherwise. If argument strict? is provided and set to false, the syntax and semantics of the JSON Path string is slighly relaxed. For instance, non-singular queries are supported in query filters.

Returns #t if string str constitutes a valid singular JSON query string; #f otherwise. A JSON query is singular if it refers to exactly one location in any JSON document. If argument strict? is provided and set to false, the syntax and semantics of the JSON Path string is slighly relaxed. For instance, non-singular queries are supported in query filters.

Applies JSON Path query to json returning the values matching the query together with the corresponding locations of the matching values. json-query returns a list of pairs of JSON values and JSON locations.

Applies JSON Path query to json returning the values matching the query in a list.

Applies JSON Path query to json returning the locations of matching values in a list.

JSON Patch

JSON Patch defines a JSON document structure for expressing a sequence of operations to apply to a JSON document. Each operation mutates parts of the JSON document. The supported operations specified by RFC 6902 are represented by the following 6 shapes of lists:

  • (add ref json): Add json to the JSON value the JSON pointer ref is referring to

  • (remove ref): Remove the JSON value at the location the JSON pointer ref is referring to

  • (replace ref json): Replace the value at the location the JSON pointer ref is referring to with json

  • (move from to): Move the value at the location at which the JSON pointer to is referring to with the value at from. This is equivalent to first removing the value at from and then adding it to path.

  • (copy from to): Copy the value at the location at which the JSON pointer to is referring to with the value at from. This is equivalent to first looking up the value at from and then adding it to path.

  • (test ref json): Compares value at ref with json and fails if the two are different.

From lists of operations it is possible to construct JSON patch objects, which are mutable containers for sequences of operations. They are created with procedure json-patch:

(define jp (json-patch
  `((test "/a/b/c" ,(json "foo"))
    (remove "/a/b/c")
    (add "/a/b/c" ,(json "foo" "bar"))
    (replace "/a/b/c" ,(json 42))
    (move "/a/b/d" "/a/b/c")
    (copy "/a/b/e" "/a/b/d"))))

A more conventional approach would be to define the JSON patch operations in JSON directly and using again procedure json-patch:

(define jp2 (json-patch (string->json (string-append
  "["
  " { \"op\": \"test\", \"path\": \"/a/b/c\", \"value\": \"foo\" },"
  " { \"op\": \"remove\", \"path\": \"/a/b/c\" },"
  " { \"op\": \"add\", \"path\": \"/a/b/c\", \"value\": [ \"foo\", \"bar\" ] },"
  " { \"op\": \"replace\", \"path\": \"/a/b/c\", \"value\": 42 },"
  " { \"op\": \"move\", \"from\": \"/a/b/c\", \"path\": \"/a/b/d\" },"
  " { \"op\": \"copy\", \"from\": \"/a/b/d\", \"path\": \"/a/b/e\" }"
  "]"
))))
(json-patch=? jp jp2)  ⇒  #t

JSON patch objects can be applied to mutable JSON values with procedure json-apply!.

Symbol representing the json-patch type. The type-for procedure of library (lispkit type) returns this symbol for all JSON patch objects.

Returns #t if obj is a JSON patch object; #f otherwise.

Returns a new JSON patch object. JSON patch objects are mutable containers for lists of operations. If no argument is provided to json-patch, then an empty JSON patch object is returned which can be extended via procedure json-patch-append!. If expr is provided, it is either an already existing JSON patch object and a copy is returned, or a JSON value representing the JSON patch operation list according to RFC 6902, or it is a list of operations. Each operation has one of the following 6 shapes:

  • (add ref json): Add json to the JSON value at ref

  • (remove ref): Remove the JSON value at ref

  • (replace ref json): Replace the value at ref with json

  • (move from to): Move the value at from to to

  • (copy from to): Copy the value at from to to

  • (test ref json): Compares value at ref with json and fails if the two are different.

JSON pointer is used for specifying references to values within a larger JSON document.

Removes all operations from the JSON patch object patch.

Appends operations oper to the JSON patch object patch. Each operation has one of the following 6 shapes: (add ref json), (remove ref), (replace ref json), (move from to), (copy from to), and (test ref json).

Compares the JSON patch objects patch ... and return #t if all patch objects are equivalent; otherwise return #f.

Returns the list of operations for JSON patch object patch.

Returns a JSON value representing the operations of the JSON patch object patch as defined by RFC 7396.

Applies a JSON patch object patch to mutable JSON value json using application rules as defined by RFC 7396.

Merging JSON values

Merges the JSON value json with the JSON value other such that the result res of the merge is the "smallest" JSON value that is a refinement of both json and other; i.e. both (json-refinement? res json) and (json-refinement? res other) hold. If such a merged value does not exist, then json-merge will return #f.

This approach is also called a symmetrical merge. Intuitively, it combines two JSON values by adding all non-existing values to the merged value and merging overlapping values or failing whenever a symmetrical merge is not possible.

Merges the JSON value json with the JSON value other using merge semantics which let other override values of json whenever merging as implemented by json-merge would fail otherwise. As opposed to procedure json-merge, combining arrays does not require the arrays to be of the same length. The resulting array has always the length of the longest of the two arrays and individual elements are combined using json-override whenever two elements are available.

Merges the JSON value json with the JSON value patch using the rules defined by RFC 7396. The merged JSON value is returned.

The patch value describes changes to be made to json using a syntax that closely mimics the document being modified. Recipients of a merge patch value determine the exact set of changes being requested by comparing the content of the provided patch against the current value json. If the provided patch value contains members that do not appear within json, those members are added. If json does contain the member, the value is replaced. Null values in patch are given special meaning to indicate the removal of existing values in the target.

Last updated