-
Notifications
You must be signed in to change notification settings - Fork 38
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Having a more formal spec is helpful for third-party authors of additional libraries. I also add tests to make sure our `go` implementation complies with the spec.
- Loading branch information
Showing
6 changed files
with
248 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# TypeID Specification (Version 0.1.0) | ||
|
||
## Overview | ||
TypeIDs are a type-safe extension of UUIDv7, they encode UUIDs in base32 and add a type suffix. | ||
|
||
Here's an example of a TypeID of type `user`: | ||
|
||
``` | ||
user_2x4y6z8a0b1c2d3e4f5g6h7j8k | ||
└──┘ └────────────────────────┘ | ||
type uuid suffix (base32) | ||
``` | ||
|
||
This document formalizes the specification for TypeIDs. | ||
|
||
## Specification | ||
|
||
A typeid consists of three parts: | ||
1. A **type prefix**: a string denoting the type of the ID. The prefix should be | ||
at most 63 characters in all lowercase ASCII [a-z]. | ||
1. A **separator**: an underscore '_' character. | ||
1. A **UUID suffix**: a 128-bit UUIDv7 encoded as a 26-character string in base32. | ||
|
||
### Type Prefix | ||
A type prefix is a string denoting the type of the ID. The prefix should be at most | ||
63 characters in all lowercase ASCII [a-z]. Valid prefixes should match the following | ||
regex: `[a-z]{0,63}`. | ||
|
||
The empty string is a valid prefix, it's there for very specific use cases in which | ||
applications need to encode a typeid but elide the type information. In general though, | ||
applications should use a prefix that is at least 3 characters long. | ||
|
||
> Note: [There's a proposal](https://github.com/jetpack-io/typeid/issues/7) to add `_` as | ||
> an allowed separator within type prefixes. | ||
### Separator | ||
The separator is a single underscore character `_`. If the prefix is empty, the separator | ||
is omitted. | ||
|
||
### UUID Suffix | ||
The UUID suffix encodes exactly 128-bits of data in 26 characters. It uses the base32 | ||
encoding described below. | ||
|
||
#### Base32 Encoding | ||
Bytes from the UUID are encoded from left to right. Two zeroed bits are pre-pended | ||
to the 128-bits of the UUID, resulting in 130-bits of data. The 130-bits are then | ||
split into 5-bit chunks, and each chunk is encoded as a single character in the | ||
base32 alphabet, resulting in a total of 26 characters. | ||
|
||
In practice this is most often done by using bit-shifting and a lookup table. See | ||
the [reference implementation encoding](https://github.com/jetpack-io/typeid-go/blob/main/base32/base32.go) | ||
for an example. | ||
|
||
Note that this is different from the standard base32 encoding which encodes in | ||
groups of 5 bytes (40 bits) and appends any padding at the end of the data. | ||
|
||
The encoding uses the following alphabet `0123456789abcdefghjkmnpqrstvwxyz` as | ||
specified by the following table: | ||
|
||
| Value | Symbol | Value | Symbol | Value | Symbol | Value | Symbol | | ||
|-------|--------|-------|--------|-------|--------|-------|--------| | ||
| 0 | 0 | 8 | 8 | 16 | g | 24 | r | | ||
| 1 | 1 | 9 | 9 | 17 | h | 25 | s | | ||
| 2 | 2 | 10 | a | 18 | j | 26 | t | | ||
| 3 | 3 | 11 | b | 19 | k | 27 | v | | ||
| 4 | 4 | 12 | c | 20 | m | 28 | w | | ||
| 5 | 5 | 13 | d | 21 | n | 29 | x | | ||
| 6 | 6 | 14 | e | 22 | p | 30 | y | | ||
| 7 | 7 | 15 | f | 23 | q | 31 | z | | ||
|
||
This is the same alphabet used by [Crockford's base32 encoding](https://www.crockford.com/base32.html), | ||
but in our case the alphabet encoding is strict: always in lowercase, no hyphens allowed, | ||
and we never decode multiple ambiguous characters to the same value. | ||
|
||
#### Compatibility with UUID | ||
When genarating a new TypeID, the generated UUID suffix MUST decode to a valid UUIDv7. | ||
|
||
Implementations MAY allow encoding/decoding of other UUID variants when the | ||
bits are provided by end users. This makes it possible for applications to encode | ||
other UUID variants like UUIDv1 or UUIDv4 at their discretion. | ||
|
||
## Versioning | ||
This spec uses semantic versioning: `MAJOR.MINOR.PATCH`. The version is incremented | ||
when the spec changes in a way that is not backwards compatible. | ||
|
||
Libraries that implement this spec should also use semantic versioning, and their | ||
MAJOR and MINOR versions should match the version of the spec they implement. | ||
The PATCH version is up to the discretion of the library author. | ||
|
||
## Validating Implementations | ||
To assist library authors in validating their implementations, we provide: | ||
+ A reference implementation in [Go](https://github.com/jetpack-io/typeid-go) | ||
with extensive testing. | ||
+ A [valid.toml](valid.toml) file containing a list of valid typeids along | ||
with their corresponding decoded UUIDs. | ||
+ An [invalid.toml](invalid.toml) file containing a list of strings that are | ||
invalid typeids and should fail to parse/decode. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# This file contains test data that should be treated as *invalid* TypeIDs by | ||
# conforming implementations. | ||
# | ||
# Each example contains an invalid TypeID string. Implementations are expected | ||
# to throw an error when attempting to parse/validate these strings. | ||
# | ||
# Last updated: 2023-06-29 | ||
|
||
- name: prefix-uppercase | ||
typeid: "PREFIX_00000000000000000000000000" | ||
description: "The prefix should be lowercase with no uppercase letters" | ||
|
||
- name: prefix-numeric | ||
typeid: "12345_00000000000000000000000000" | ||
description: "The prefix can't have numbers, it needs to be alphabetic" | ||
|
||
- name: prefix-period | ||
typeid: "pre.fix_00000000000000000000000000" | ||
description: "The prefix can't have symbols, it needs to be alphabetic" | ||
|
||
- name: prefix-underscore | ||
typeid: "pre_fix_00000000000000000000000000" | ||
description: "The prefix can't have symbols, it needs to be alphabetic" | ||
|
||
- name: prefix-non-ascii | ||
typeid: "préfix_00000000000000000000000000" | ||
description: "The prefix can only have ascii letters" | ||
|
||
- name: prefix-spaces | ||
typeid: " prefix_00000000000000000000000000" | ||
description: "The prefix can't have any spaces" | ||
|
||
- name: prefix-64-chars | ||
# 123456789 123456789 123456789 123456789 123456789 123456789 1234 | ||
typeid: "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijkl_00000000000000000000000000" | ||
description: "The prefix can't be 64 characters, it needs to be 63 characters or less" | ||
|
||
- name: separator-empty-prefix | ||
typeid: "_00000000000000000000000000" | ||
description: "If the prefix is empty, the separator should not be there" | ||
|
||
- name: separator-empty | ||
typeid: "_" | ||
description: "A separator by itself should not be treated as the empty string" | ||
|
||
- name: suffix-short | ||
typeid: "prefix_1234567890123456789012345" | ||
description: "The suffix can't be 25 characters, it needs to be exactly 26 characters" | ||
|
||
- name: suffix-long | ||
typeid: "prefix_123456789012345678901234567" | ||
description: "The suffix can't be 27 characters, it needs to be exactly 26 characters" | ||
|
||
- name: suffix-spaces | ||
# This example has the right length, so that the failure is caused by the space | ||
# and not the suffix length | ||
typeid: "prefix_1234567890123456789012345 " | ||
description: "The suffix can't have any spaces" | ||
|
||
- name: suffix-uppercase | ||
# This example is picked because it would be valid in lowercase | ||
typeid: "prefix_0123456789ABCDEFGHJKMNPQRS" | ||
description: "The suffix should be lowercase with no uppercase letters" | ||
|
||
- name: suffix-hyphens | ||
# This example has the right length, so that the failure is caused by the hyphens | ||
# and not the suffix length | ||
typeid: "prefix_123456789-123456789-123456" | ||
description: "The suffix should be lowercase with no uppercase letters" | ||
|
||
- name: suffix-wrong-alphabet | ||
typeid: "prefix_ooooooiiiiiiuuuuuuulllllll" | ||
description: "The suffix should only have letters from the spec's alphabet" | ||
|
||
- name: suffix-ambiguous-crockford | ||
# This example would be valid if we were using the crockford disambiguation rules | ||
typeid: "prefix_i23456789ol23456789oi23456" | ||
description: "The suffix should not have any ambiguous characters from the crockford encoding" | ||
|
||
- name: suffix-hyphens-crockford | ||
# This example would be valid if we were using the crockford hyphenation rules | ||
typeid: "prefix_123456789-0123456789-0123456" | ||
description: "The suffix can't ignore hyphens as in the crockford encoding" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# This file contains test data that should parse as valid TypeIDs by conforming | ||
# implementations. | ||
# | ||
# Each example contains: | ||
# - The TypeID in its canonical string representation. | ||
# - The prefix | ||
# - The decoded UUID as a hex string | ||
# | ||
# Implementations should verify that they can encode/decode the data | ||
# in both directions: | ||
# 1. If the TypeID is decoded, it should result in the given prefix and UUID. | ||
# 2. If the UUID is encoded as a TypeID with the given prefix, it should | ||
# result in the given TypeID. | ||
# | ||
# In addition to using these examples, it's recommended that implementations | ||
# generate a thousands of random ids during testing, and verify that after | ||
# decoding and re-encoding the id, the result is the same as the original. | ||
# | ||
# In other words, the following property should always hold: | ||
# random_typeid: encode(decode(random_typeid)) | ||
# | ||
# Finally, while implementations should be able to decode the values below, | ||
# note that not all of them are UUIDv7s. When *generating* new random typeids, | ||
# implementations should always use UUIDv7s. | ||
# | ||
# Last updated: 2023-06-29 | ||
|
||
- name: nil | ||
typeid: "00000000000000000000000000" | ||
prefix: "" | ||
uuid: "00000000-0000-0000-0000-000000000000" | ||
|
||
- name: one | ||
typeid: "00000000000000000000000001" | ||
prefix: "" | ||
uuid: "00000000-0000-0000-0000-000000000001" | ||
|
||
- name: ten | ||
typeid: "0000000000000000000000000a" | ||
prefix: "" | ||
uuid: "00000000-0000-0000-0000-00000000000a" | ||
|
||
- name: sixteen | ||
typeid: "0000000000000000000000000g" | ||
prefix: "" | ||
uuid: "00000000-0000-0000-0000-000000000010" | ||
|
||
- name: thirty-two | ||
typeid: "00000000000000000000000010" | ||
prefix: "" | ||
uuid: "00000000-0000-0000-0000-000000000020" | ||
|
||
- name: valid-alphabet | ||
typeid: "prefix_0123456789abcdefghjkmnpqrs" | ||
prefix: "prefix" | ||
uuid: "0110c853-1d09-52d8-d73e-1194e95b5f19" | ||
|
||
- name: valid-uuidv7 | ||
typeid: "prefix_01h455vb4pex5vsknk084sn02q" | ||
prefix: "prefix" | ||
uuid: "01890a5d-ac96-774b-bcce-b302099a8057" |