Do not shipping records over the inter-node wire

[wohali]

Originally by @chewbranca

We now depend on Erlang 17 minimum, and can easily move up to 19 or 20 if desired.

We could possibly look at this now using maps or versioned records.

/cc @rnewson @davisp

[iilyak]

I think we should use versioned structures everywhere not only over the inter-node wire.

Even if we use maps we still would want to include the version field there.

We use at least 3 different ways to implement versioned structures (I am sure we have more):

• couch_bt_engine_header.erl - get/set/upgrade in the module.
• couch_att - store/fetch/transform in the module
• fabric_util:upgrade_mrargs/1 - uses pattern match to distinguish between record and tuple

Useful features to consider

Inspired by migration pattern from other languages and frameworks.

• ability to upgrade to specified version from any known version
• ability to downgrade to specified version from any known version
• ability to detect the version of the structure and cast (upgrade or downgrade) the record to the version used on the node
• ability to construct the structure from key-value list
• ability to construct the structure from ejson
• ability to convert structure to ejson
• ability to convert structure to key-value list (useful in debug messages)
• ability to get current version of the structure
• callback module should implement defined upgrade/downgrade interface to do the conversion
• implementation of new structure should be free of huge boilerplate
• encapsulate all operations for a structure in a single module
• access to the current version of the structure should have low overhead
• we should be able to receive newer version of the structure and send the old

For the reference one simple implementation is here and here

Proposal

1. define new callback module (behavior)
2. implement some of the common functionality in couch_structure helper module
3. every new versioned structure would need to
   • implement prescribed interface in dedicated module
   • have migration functions (or clauses) to be able to upgrade and downgrade to any version from any given one
4. We would still need to split the structure update to two separate commits
   • implement compatibility clauses in structure module
   • bump the version of structure we would be sending out (or storing on disk)

Open questions

1. Define the callback module API
2. Should we consider a runtime toggle for the version of structure we sending out (storing on disk). This might help us to avoid creating two releases (two-stage deployment). It could be implemented efficiently if we decide to use code-generation to produce structure's callback module.
3. Define the scope

• only inter-node communications
• inter-node communications and persistent structures
• all structures