JSON Kotlin core library

The input of JSON data generally consists of two main phases – parsing the input text and converting the human-readable form into an easily navigated internal representation, and then mapping that internal form into pre-existing data types. Output may similarly use an intermediate form, but it is on the input side that the converted form is most useful – it allows, for example, all of the properties of an object to be analysed before the determination of the appropriate representation for the object.

There are also many types of JSON processing functions that do not require mapping to a target class – they simply require an internal representation of the JSON data.

The kjson-core library provides the basic functionality required to represent JSON values in Kotlin, including:

• parsing functions to convert JSON text to a structure of JSON values
• classes to hold the internal forms of the values
• output functions to create valid JSON representations from the internal form

The library is an evolution of the jsonutil Java library; it makes better use of Kotlin-specific functionality like controlled nullability.

All JSON values are represented by Kotlin objects of type JSONValue? – that is, they are all instances of classes that implement the JSONValue interface, or in the case of the JSON “null” value they are null.

The JSONValue interface specifies four functions:

• appendTo() – this appends the JSON text form of the object to a specified Appendable, e.g. a Writer (when outputting JSON, it is more efficient to append to a single Appendable, as opposed to creating strings for each element)
• toJSON() – this outputs the value in syntactically-correct JSON (a default implementation makes use of the above appendTo() function)
• outputTo() – this outputs the JSON text form of the object using an IntConsumer (similar to appendTo(), but allowing a greater choice of output mechanism)
• coOutputTo() (suspend function) – non-blocking version of outputTo(), suitable for use in a coroutine-based environment

JSONValue is a sealed interface and the implementing classes are limited to:

• JSONString – a string value
• JSONInt – a number value that fits in a 32-bit signed integer
• JSONLong – a number value that fits in a 64-bit signed integer
• JSONDecimal – any number value, including non-integer (uses BigDecimal internally)
• JSONBoolean – a boolean value
• JSONArray – an array
• JSONObject – an object

The implementing classes are all immutable.

Following a common Kotlin pattern, there are creation functions named JSONValue which create the appropriate implementing class depending on the parameter type:

JSONPrimitive is a sealed interface (and a sub-interface of JSONValue) implemented by the classes for primitive values, i.e. JSONInt, JSONLong, JSONDecimal, JSONString and JSONBoolean. It is a parameterised interface, where the parameter is the type of the value. The interface specifies a single value (named value), of the parameter type. The value is never null.

In addition to implementing JSONPrimitive, the number value classes JSONInt, JSONLong and JSONDecimal all derive from the sealed class JSONNumber, which itself derives from the system class Number. This means that these classes may be used without conversion anywhere a Number is called for.

The Number class provides a set of toInt(), toLong() etc. functions, to which JSONNumber adds the following:

JSONNumber also provides the following boolean functions:

The JSONNumber classes also override equals() (and hashCode()) so that instances with the same value but different types will be regarded as equal. JSONInt(27), JSONLong(27) and JSONDecimal(27) will all be considered equal, and will all return the same hash code.

Following a common Kotlin pattern, there are creation functions named JSONNumber which create the appropriate derived class depending on the parameter type:

The JSONInt class holds JSON number values that fit in a 32-bit signed integer. The class derives from JSONNumber, providing implementations for all the abstract functions of that class, and it also implements JSONPrimitive with the parameter type Int.

The Int value may be accessed by the property value.

The JSONLong class holds JSON number values that will fit in a 64-bit signed long integer. The class derives from JSONNumber, providing implementations for all the abstract functions of that class, and it also implements JSONPrimitive with the parameter type Long.

The Long value may be accessed by the property value.

The JSONDecimal class holds any JSON number values, including non-integer values. The class derives from JSONNumber, providing implementations for all the abstract functions of that class, and it also implements JSONPrimitive with the parameter type BigDecimal.

The BigDecimal value may be accessed by the property value.

The JSONString class holds a JSON string value. The class implements JSONPrimitive with the parameter type String.

The parser converts JSON escape sequences on input, and the appendJSON() and toJSON() functions convert non-ASCII characters to escape sequences on output.

The String value may be accessed by the property value (which will never be null).

A JSONString may be constructed dynamically using the build {} function, which takes as a lambda parameter an extension function on StringBuilder; anything appended to the StringBuilder will become part of the JSONStrong. For example:

    val jsonString = JSONString.build {
        append("Number = ")
        append(number)
    }

JSONString also implements the CharSequence interface, which allows access to all the functionality of that interface without having to extract the value property. The subSequence() function will return a new JSONString.

JSONBoolean is an enum class with two members – TRUE and FALSE. The class implements JSONPrimitive with the parameter type Boolean.

The Boolean value may be accessed by the property value.

JSONStructure is a sealed interface (another sub-interface of JSONValue) implemented by the classes for structured types, that is, arrays and objects. It specifies a single value size (Int) which gives the number of entries in the array or object, and the functions isEmpty() and isNotEmpty() which (unsurprisingly) return true or false respectively if the structure is empty.

JSONStructure is a parameterised interface, where the parameter K is the type of the value to locate entries in the structure, i.e. Int for JSONArray or String for JSONObject.

It also provides convenience functions to both get a member of the structure (using a key of the parameter type K) and convert it to the required type:

These have the advantage over, for example, json["property"].asString, that the JSONTypeException thrown when the type is incorrect includes the key or index used to select the item.

The JSONArray class implements the List<JSONValue?> interface, and all the functions of that interface are available to navigate the array (indexing via array[n], contains(obj), iterator() etc.). The subList() function will return a new JSONArray.

The class also implements the JSONStructure interface with a parameter type of Int.

The constructor for JSONArray is not publicly accessible, but an of() function is available in the companion object, and a build function and the Builder nested class allow arrays to be constructed dynamically.

Following a common Kotlin pattern, there is also a creation function named JSONArray taking a vararg array of JSONValue?.

JSONArray implements the equals() and hashCode() functions as specified for the Java Collections classes, so that an instance of JSONArray may be compared safely with an instance of any class correctly implementing List<JSONValue?>.

The JSONArray class also provides optimised iteration functions to iterate over the items of an array. Because the class implements the List interface, there are iteration functions available from the standard library, but the additional functions are optimised for the specific implementation details of the JSONArray class.

    jsonArray.forEachItem {
        println("Item = $it")
    }
    jsonArray.forEachItemIndexed { index, item ->
        println("Item #$index = $item")
    }

The JSONObject class implements the Map<String, JSONValue?> interface, and all the functions of that interface are available to navigate the object (retrieval via structure["name"], containsKey("name"), entries etc.). The class also implements the JSONStructure interface with a parameter type of String.

The original order of the input is maintained on parsing or on the programmatic creation of a JSONObject, and to take advantage of this sequential ordering of properties, the JSONObject class also implements the List<Property> interface, where Property is a nested class representing the Map.Entry objects used by JSONObject (see below). This means that the JSONObject class provides both:

    jsonObject["name"]  // get the property named "name" as a JSONValue?

and:

    jsonObject[3] // get the fourth property (index 3) as a JSONObject.Property

The constructor for JSONObject is not publicly accessible, but an of() function is available in the companion object, and a build function and the Builder nested class allow objects to be constructed dynamically.

Following a common Kotlin pattern, there are also creation functions named JSONObject taking a vararg array of Pair<String, JSONValue?> or JSONObject.Property.

JSONObject implements the equals() and hashCode() functions as specified for the Java Collections classes, so that an instance of JSONObject may be compared safely with an instance of any class correctly implementing Map<String, JSONValue?>.

The JSONObject class also provides optimised iteration functions to iterate over the entries, the keys (property names) or the values of an object. Because the class implements the Map interface, there are iteration functions available from the standard library, but the additional functions are optimised for the specific implementation details of the JSONObject class.

    jsonObject.forEachEntry { name, value ->
        println("Property $name = $value")
    }
    jsonObject.forEachProperty { property ->
        println("Property ${property.name} = ${property.value}")
    }
    jsonObject.forEachKey {
        println("Property name = $it")
    }
    jsonObject.forEachValue {
        println("Property value = $it")
    }

The JSONObject.Property nested class implements the Map.Entry<String, JSONValue?> interface, and is used to hold the key-value pairs of the Map behind JSONObject,

It has two properties:

The JSONObject.Property object is immutable.

There is an infix function refersTo taking a String and a JSONValue? which creates a JSONObject.Property:

    val property = "propertyName" refersTo JSONString("Property value")

Error conditions will usually result in a JSONException being thrown. This is a derived class of RuntimeException, and the message property will contain a text description of the error.

The exception also includes a property key (of type Any?) which is used to provide information on the location of the error, for example a JSONPointer or a property name. When the key is provided, it will be appended to the message, as “, at {key}”.

Starting from version 8.1 of this library, JSONException has been extracted to a separate library – kjson-exception – so that it may be included in other projects independent from this library

A common error case arises when a JSONValue is found to be of the wrong type, for example, when a JSONArray is supplied as a parameter to a function that expects a JSONObject, or when a property of an object is a JSONString when a JSONInt was expected. The JSONTypeException provides a way of reporting such errors in a consistent manner, with error messages including the human-readable node name, the expected type, the actual value and an optional key (as described above).

The JSONTypeException constructor takes the following parameters:

For example, the following exception:

    throw JSONTypeException("Surname", "string", surname, "/person/surname")

will result in this message (if an array was supplied in place of a string):

Surname not correct type (string), was [ ... ], at /person/surname

The actual value will be displayed using the displayValue() function, and the “at” clause will be omitted if the key is null or key.toString() returns an empty string.

For a more convenient way of using this exception type, see Error Reporting below.

The JSON object contains a number of functions to assist with parsing and object creation.

The simplest way to parse JSON text is:

        val json = JSON.parse(text)

The result will be of type JSONValue? – it will be null if the text consists of just the string “null” (with possible leading and trailing whitespace).

If only non-null JSON values are expected:

        val json = JSON.parseNonNull(text)

The result of this function will be of type JSONValue (no question mark) and an exception will be thrown if the JSON was “null”.

If the JSON is expected to be an object (and it is an error if it is not):

        val json = JSON.parseObject(text)

In this case the result will be of type JSONObject, and an exception will be thrown if it is not an object.

Similarly, if the JSON is expected to be an array:

        val json = JSON.parseArray(text)

The result type will be JSONArray, and again, an exception will be thrown if the input is not of the correct type.

The JSON object also provides a number of shortcut functions to create JSONValues:

To simplify error reporting, the JSON object provides a displayValue() extension function on JSONValue? to create an abbreviated form of the value suitable for error messages. Arrays are displayed as [ ... ], objects are displayed as { ... }, and long strings are shortened with “...” in the middle.

For example:

     JSONString("the quick brown fox jumps over the lazy dog").displayValue()

will display:

"the quic ... lazy dog"

The maximum number of characters to display in a string defaults to 21, but may be specified as a parameter, e.g. displayValue(17) (odd numbers are best, because they result in the same number of characters before and after the elision).

There is often a requirement to log JSON inputs for later error diagnosis, with the restriction that logs must not contain sensitive information. The elidedValue() extension function on JSONValue? allows JSON values to be converted to the text form with certain nominated elements excluded.

For example:

    val json = JSON.parse("""{"name":"Adam","accountNo":"12345678"}""")
    json.elidedValue(exclude = setOf("accountNo"))

will display:

{"name":"Adam","accountNo":"****"}

All elements with the specified name will be elided, wherever they occur in the object tree.

The elements to be elided may be specified as a Collection of element names to be excluded as shown above, or (less usefully) as a Collection of element names to be included (using the include parameter). The substitute string (default “****”) may also be specified using the substitute parameter.

To simplify the creation of a JSONTypeException, the JSON object includes the typeError() extension function on JSONValue?. It takes the following parameters:

Its use is best illustrated by example:

    if (node !is JSONString)
        node.typeError("String", "/person/surname", "Surname")

This will produce an exception like the one shown in the description of JSONTypeException.

The conversion may be combined with the error reporting using the asXxxxOrError() functions:

Note that the functions representing a simple value return the actual value type, not the JSONValue subtype. The asArrayOrError() and asObjectOrError() functions return JSONArray and JSONObject respectively, but these can of course be used as the underlying implementation types (List and Map).

The functions all take the same parameters as the typeError() function (which they all call if the type is not correct), but in the case of these functions, the target parameter also has a default value, a string representing the target type.

Using these functions, the above example (for the use of typeError) may be written:

    node.asStringOrError(key = "/person/surname", nodeName = "Surname")

To simplify casting a JSONValue to the expected type, the JSON object provides extension values on JSONValue?:

The JSONTypeException will use the default value "Node" for the nodeName, and the class name of the target type as the default for target. The default value for key is null.

As with the asXxxxOrError() functions, the extension values representing a simple value return the actual value type, not the JSONValue subtype (i.e. asInt returns Int, not JSONInt), but the asArrayOrError() and asObjectOrError() functions return JSONArray and JSONObject respectively.

A further way of casting a JSONValue to the expected type is provided by the asXxxxOr functions:

The functions all take a single parameter – a lambda with the JSONValue? as the receiver which will be invoked if the JSONValue? is not the correct type. This may be used to provide a default value, silently ignoring the type error, but more commonly it will be used to throw an exception. For example:

    node.asStringOr { typeError(target = "string", key = "/person/surname", nodeName = "Surname") }

The advantage of using these functions as compared to asXxxxOrError(), is that these functions are inline, and the code to set up the parameters and call a separate function will not be executed if the node is of the correct type.

Again, as with the asXxxxOrError() functions, the extension functions returning a simple value return the actual value type, not the JSONValue subtype, and the asArrayOr() and asObjectOr() functions return JSONArray and JSONObject respectively.

The JSON Lines specification allows multiple JSON values to be specified in a single stream of data, separated by newline (\u000a) characters. For example, events may be logged to a file as a sequence of objects on separate lines; the alternative would be to output a JSON array, but this would require a “]” terminator, complicating the shutdown of the process (particularly abnormal shutdown).

{"time":"2023-06-24T12:24:10.321+10:00","eventType":"ACCOUNT_OPEN","accountNumber": "123456789"}
{"time":"2023-06-24T12:24:10.321+10:00","eventType":"DEPOSIT","accountNumber": "123456789","amount":"1000.00"}

The individual items are usually objects (or sometimes arrays) formatted similarly, but that is not a requirement – the items may be of any JSON type.

The JSON Lines format is particularly suitable for streaming data, so the kjson-stream library is more likely to be useful for JSON Lines input than the functions in this library that parse a complete file, but the functions here are provided for completeness.

The kjson-core library includes functions to parse JSON Lines format:

    val lines = JSON.parseLines(multiLineString)

The result will always be a JSONArray; an empty string will result in a zero-length array.

While the parseLines() function (and its corresponding function in the Parser class) will correctly parse a stream of data in JSON Lines format, the newline separator is in fact not required. The function will accept JSON objects and/or arrays concatenated without any delimiters, but because whitespace is allowed between tokens of JSON data, the newline (if present) will be ignored.

In most cases, JSON Lines data will be output as individual objects using appendTo() or toJSON(). If an entire JSONArray is required to be output in JSON Lines format, there are four additional functions for this purpose:

• appendJSONLinesTo() – this appends the JSON Lines form of the JSONArray to a specified Appendable, e.g. a Writer
• toJSONLinesTo() – this converts the JSONArray to a String in JSON Lines format
• outputJSONLinesTo() – this outputs the JSON Lines form of the JSONArray using an IntConsumer (similar to appendJSONLinesTo(), but allowing a greater choice of output mechanism)
• coOutputJSONLinesTo() (suspend function) – non-blocking version of outputJSONLinesTo(), suitable for use in a coroutine-based environment

The functions all add a single newline after each item in the JSONArray for human readability reasons, even though (as noted above) this is not strictly necessary.

The parser will by default apply strict validation to the JSON input, and in some cases this may be unhelpful. There is occasionally a need to parse JSON that is not correctly formatted according to the specification, and to accommodate this requirement, the parser may be supplied with a ParseOptions object containing option settings for parser leniency. For example:

    val options = ParseOptions(
        objectKeyDuplicate = JSONObject.DuplicateKeyOption.ERROR,
        objectKeyUnquoted = false,
        objectTrailingComma = false,
        arrayTrailingComma = false,
    )
    val jsonValue = Parser.parse(jsonString, options)

Note that in order to use this functionality, the Parser must be called directly; the helper functions in the JSON object do not include this capability.

The JSON specification states that a given key SHOULD appear only once in an object, but some software may output objects with the same key repeated multiple times. Under normal circumstances, the parser will throw an exception when it encounters a second occurrence of the same key, but if such data is required to be accepted, the objectKeyDuplicate options setting may be used to specify the desired behaviour.

The field is an enum (JSONObject.DuplicateKeyOption), and the possible values are:

• ERROR: treat the duplicate key as an error (this is the default)
• TAKE_FIRST: take the value of the first occurrence and ignore duplicates
• TAKE_LAST: take only the last occurrence and ignore any preceding occurrences
• CHECK_IDENTICAL: ignore duplicates only if they are identical to the original value, otherwise report an error

Unlike JavaScript, on which it is based, JSON requires that object keys be enclosed in quotes. Sometimes, particularly when parsing human-edited JSON, it can be a helpful to allow keys to be conventional computer language identifiers, and this can be selected by the objectKeyUnquoted option.

Setting this flag to true will cause the parser to allow object keys to be specified without quotes. When using this option, the keys must follow this pattern:

• the first character must be ASCII alphabetic (upper or lower case) or underscore
• subsequent characters must be ASCII alphabetic (upper or lower case) or numeric or underscore

When outputting the members of an object, it can be simpler to add a comma after each member, regardless of whether it is the last one. To allow trailing commas in objects, the option objectTrailingComma can be set to true.

Similarly, when outputting the items of an array, it can be simpler to add a comma after each item. To allow trailing commas in arrays, the option arrayTrailingComma can be set to true.

This class diagram may help to explain the main classes and interfaces and the inheritance and interface implementation relationships between them.

The diagram was produced by Dia; the diagram file is at diagram.dia.

The latest version of the library is 9.1, and it may be obtained from the Maven Central repository.

    <dependency>
      <groupId>io.kjson</groupId>
      <artifactId>kjson-core</artifactId>
      <version>9.1</version>
    </dependency>

    implementation "io.kjson:kjson-core:9.1"

    implementation("io.kjson:kjson-core:9.1")

Peter Wall

2024-08-17