Helpers to serialize and deserialize binary data.

nu-serde-bin is a Nushell package that ships a single module: nu-serde-bin.

This module then exports two commands:

• deserialize: takes in some binary data and a schema and outputs a Nushell value that is the deserialized equivalent of the input.
• serialize: takes in some Nushell value and a schema and outputs binary data. It is the inverse of deserialize.

The recommended way is to use Nupm.

nu-serde-bin currently supports three kinds of binary data and their associated Nushell values:

• integers:
  • the format is int:n where $n$ is the number of bytes of the integer
    • $n$ could be any strictly positive integer
  • the associated Nushell values are int
• lists / vectors of integers:
  • the format is vec:n where $n$ is the number of bytes of each element in the vector
    • all elements in the vector should take the same amount of bytes $n$
    • $n$ could be any strictly positive integer
  • the length of the vector is encoded with $8$ bytes, before the first item of the vector
  • the associated Nushell values are list<binary>
• records / key-value structures:
  • the format is { k1: v1, k2, v2 } where
    • the $k_i$ are string names that you can define however you like
    • the $v_i$ are any of the valid nu-serde-bin formats
  • this format is recursive, i.e. a $v_i$ can itself be a record
  • the associated Nushell values are record<>

0x[01 00] | deserialize "not-a:format"

0x[01 10] | deserialize "int:4"

assert equal (123456 | serialize "int:1") 0x[40] # instead of `0x[40 e2 01 00]`

assert equal (0x[01 10] | deserialize "int:2") 4097

assert equal (123456 | serialize "int:4") 0x[40 e2 01 00]

assert equal (123456 | serialize "int:8") 0x[40 e2 01 00  00 00 00 00]

let actual = 0x[03 00 00 00  00 00 00 00  00 01 02] | deserialize "vec:1"
#               \______________________/  \______/
#                        length             items
let expected = [0x[00], 0x[01], 0x[02]]

assert equal $actual $expected

let actual = [0x[01 00], 0x[02, 00], 0x[03, 00], 0x[04, 00]] | serialize "vec:2"
let expected = 0x[04 00 00 00  00 00 00 00  01 00 02 00  03 00 04 00]

assert equal $actual $expected

const SCHEMA = {
    a: "int:2",
    v: "vec:1",
    b: { a: "int:1", b: "int:3", c: "int:2" },
}

let bin = 0x[
    01 01 05 00  00 00 00 00  00 00 ff ff  ff ff ff 01
    02 03 04 05  06
]
let value = {
    a: 257,
    v: [0x[ff], 0x[ff], 0x[ff], 0x[ff], 0x[ff]],
    b: { a: 1, b: 262914, c: 1541 },
}

assert equal ($bin | deserialize $SCHEMA) $value

assert equal ($value | serialize $SCHEMA) $bin

Below is a non-exhaustive list of Rust libraries that should be supported by nu-serde-bin:

• the bincode crate
• the serde::Serialize and serde::Deserialize traits from the Serde framework
• the ark_serialize::CanonicalSerialize and ark_serialize::CanonicalDeserialize traits from the Arkworks ecosystem

• polish serialization error messages
• add documentation