You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To simplify Tarantool builds' identification, the versioning policy for stable
releases is changing and will take effect for builds starting with 2.10.0
numbers as well as future major series ones (3.0.0+).
The new version numbering scheme replaces the legacy policy
and changes to semver-like MAJOR.MINOR.PATCH format for releases and to
MAJOR.MINOR.PATCH-<pre-release suffix> for pre-releases. Pre-releases are
marked with suffixes "-alphaN", "-betaN" and "-rcN" explicitly so that users
could avoid installing these versions on production systems. Backwards
compatibility is guaranteed between minor builds in the same major release
series and is appreciated but not guaranteed between different major numbers.
The topics below describe the new versioning policy in great detail.
This document is the exhaustive description of the new release policy. Its main
purpose is to collect feedback regarding the policy, so the audience is
tarantool developers, application developers, administrators, architects and
everyone who is interested in the discussion.
The document gives many details, quite heavy and supposed to be a base for a
set of different documents / webpages:
For customers:
A short release policy description.
Our formal guarantees.
A table of release series, lifetime and support state.
Public roadmap.
Download instructions.
For development:
A developer operations manual.
A release manager operations manual.
QA requirements for an upcoming release.
Goals
Elaborate better release naming and series naming policy to improve
transparency for users (we want to shift closer to semver).
Extend support time of a release series to better meet the industry
expectations. At least two years since a first release.
Extend time for designing, implementing and stabilizing major features.
Applicability
Existing 2.X.Y releases follow the old release policy, where 2.X was a
release series. The new policy is in effect since 2.10.0.
All future series must follow this document.
Terms
Pre-release
A frozen commit for early adopters, preview of a future release.
Release
A frozen commit that we advertize as production ready.
Release / pre-release version
An unique identifier of a pre-release or a release.
Release series
A product with linear evolution, pre-release and release points and certain
compatibility guarantees within the series and between series.
Backward compatibility (binary data layout)
A newer release (its runtime) is backward compatible in this sense with an
older one when the newer release is operational when working on top of data
(*.xlog, *.snap, *.vylog, *.run) from the older release. All
functionality that is part of the older release is working in this
configuration.
An attempt to use a new functionality either successful or give a meaningful
error until the database schema upgrade: it does not lead to a service outage
or data corruption. An instance is able to upgrade the data layout using the box.schema.upgrade() call to enable all features of the new release (when
all instances of the replicaset are run on the same tarantool version).
Backward compatibility (binary protocol)
All binary protocol requests that were operational in an older release are
remain operational on a newer release and does not change meaning. Responses
have the same format, but mappings may have fields that were not present in
the older release.
net.box client of the older release able to work with the newer release
(except, maybe, functionality introduced in the newer release). net.box
client of the newer release is fully operational with the server that is run
under the older release (except, maybe, functionality that the older release
does not implement).
Backward compatibility (replication protocol)
A instance that is run on a newer release may work as upstream (master) of an
instance with an older release or as downstream (replica) without database
schema upgrade.
The database schema upgrade (box.schema.upgrade()) must be performed when
all replicaset instances run on the same tarantool version. The upgrade does
not cause downtime (if the application does not lean on internal schema
representation).
Backward compatibility (Lua code)
A code that are run on an older release is run with the same effect on a
newer release. However only meaningful code counts: if any code just give an
error, but starts doing something useful, the change is considered
compatible.
A room for new functionality is still here: we can add new arguments to the
end and options (fields in a table argument), add more fields to a returning
table, add more returning values (multireturn).
Adding a new built-in module or a new global value is considered as the
compatible change.
Adding a new field to an existing metatable is okay if it is not listed in the
Lua 5.1 Reference Manual. Otherwise it should be proven that it may not broke
any meaningful code.
Examples of what is okay:
Add __pairs, __ipairs to a metatable of a userdata/cdata (because it is
not from Lua 5.1 and because userdata/cdata has no default behaviour for pairs() and ipairs() calls).
Add or extend __lt, __le metamethod (if the attempt to use <, <=
and so on leads to an error before the change -- had no defined behaviour).
Extend existing __eq metamethod implementation for a case, where it leads
to en error before the change.
Examples of NOT compatible changes:
Add __pairs, __ipairs to a metatable of a table (because it already has
its own behaviour before the change).
Add __eq metamethod if it was not defined (default behaviour is defined
for any pair of Lua objects).
Backward compatibility (SQL code)
The same principle: if any request is run with the same effect, the change is
compatible (don't counting requests that always lead to an error).
What is okay:
Add a new keyword.
Add a new type.
Add a new built-in function.
Add a new system table that starts from underscore.
Add a new collation.
Add an implicit or explicit cast rule for a set of operations {X} and a list
of types [Y] if [operation from {X}]([list of values of [Y] types]) is not
implemented before the change (raising an error).
Change order of tuples in SELECT's resultset if ORDER BY is not specified.
Technically those changes may break a working code in case of a name clash,
but the probability of this situation is considered as negligible. (We can
restrict this rule in a future.)
Examples of NOT compatible changes:
Change result of working implicit or explicit cast.
Change of a literal type.
Backward compatibility (C code)
A module or a C stored procedure that is run on an older release is
run with the same effect on a newer release.
It is completely okay to add a new function or a new structure to the public
C API. It must use one of tarantool prefixes (box_, fiber_, luaT_, luaM_ and so on) or introduce a new one.
A symbol from a library we use must not be exported directly (because the
library may be used in a module per se and the clash may lead to problems).
The exception is when we export the whole public API of the library (as for libcurl).
Don't introduce new functions or structures with the lua_ and luaL_
prefixes. Those prefixes are for the Lua runtime. Use luaT_ for tarantool
specific functions and luaM_ for general-purpose ones.
Compatibility rules
Pre-releases and releases of one release series are compatible in all
senses defined above (any release with any release).
Pre-releases and releases of consequent series are compatible by data
layout, binary protocol and replication protocol.
We don't give any guarantees regarding compatibility between
pre-releases/releases of non-consequent release series if the opposite
is not stated in the release notes.
We don't give any guarantees regarding compatibility between alpha/beta
versions and between alpha/beta and pre-release/release even within one
series.
A release series lifecycle
A release series goes over the following stages:
Stage
Versions to publish
Early development
Alpha, beta, release candidate
Support
Release candidate, release
End of life
N/A
The sections below describe various details about those stages.
Version naming
Everything spins around the three digit release naming like 3.1.2. Here the
release series is 3 (major version). 3.1 identifies a feature set, the
minor version is increased with new features. 3.1.2 and 3.1.3 may differ by
fixed bugs.
During early development we publish alpha, beta and release candidates, they
are marked this way: 3.0.0-alphaN, 3.0.0-betaN, 3.0.0-rcN (N is a
number).
During the support stage we also can publish a release candidate, it'll be
marked similarly: 3.2.0-rcN.
How the version is reported
A user can see the version in different contexts and, in fact, it is reported
differently in those contexts. Let's define those contexts.
Releases and pre-releases.
tarantool --version and friends.
A greeting in the network protocols.
CentOS, Fedora, openSUSE packages.
Debian and Ubuntu packages.
Source tarballs.
Homebrew formula (macOS).
The following subsections will describe how versions are reported in those
contexts.
Releases and pre-releases
It is what a maintainer set as a git tag and what a user sees on the releases
page.
The table below summarizes possible values.
Tarantool version/build
(Pre-)release
<= 2.8
X.Y.Z
>= 2.10 (release)
X.Y.Z
>= 2.10 (pre-release)
X.Y.Z-rcN
>= 2.10 (dev build)
N/A
Notes:
Where rcN is written, assumed alphaN, betaN as well.
tarantool --version
This version is reported in the following places:
The first line of tarantool --version output.
The _TARANTOOL global variable in Lua.
The box.info.version expression in Lua.
The require('tarantool').version expression in Lua.
The first stderr line at start in the interactive mode.
The PACKAGE_VERSION macro (string literal) in module.h.
It is git describe --always --long <commit id> result for tarantool built
from <commit id>.
The table below summarizes possible values.
Tarantool version/build
tarantool --version
<= 2.8
X.Y.Z-N-g<commit id>
>= 2.10 (release)
X.Y.Z-0-g<commit id>
>= 2.10 (pre-release)
X.Y.Z-rcN-0-g<commit id>
>= 2.10 (dev build)
X.Y.Z[-rcN]-N-g<commit id>
Notes:
Where rcN is written, assumed alphaN, betaN as well.
For developer builds the special entrypoint mark also possible here.
Tarantool Enterprise versioning is out of scope of this document.
Reasoning
It is technically possible to trim the version to X.Y.Z and X.Y.Z-rcN
(alphaN, betaN) for (pre-)releases, however we want to keep it as is.
The motivation is the following:
It is more reliable to see the <commit id> when a problem is reported by a
user to a developer. It is possible that a git tag is added locally. As a
side effect, it is just convenient to checkout the particular revision using
the git hash.
Many Lua modules, CI and administrator scripts, CLI tools (like cartridge-cli) leans on existing version format. Pre-releases have alphaN, betaN, rcN suffixes, the format is inevitably changed here.
However we can keep it unchanges for tarantool releases, so the old tools
will continue to work without changes on them.
It is also technically possible to add something like -dev suffix into the
version string on a developer build. It would help a user to easily distinguish
(pre-)release version from a developer build.
However we want to avoid the change for the same of better compatibility and
stability:
If we'll keep it, existing verson parsing code will not require changes to
support this -dev suffix.
Now the version format is stable and don't vary between build types. It
means that the version parsing code that takes care only to (pre-)releases
will not fail on a developer build.
It is possible to deliver user-friendly information regarding type of the build
(release, pre-release, developer build) in a less intrusive way. We should
elaborate those possibilities.
A greeting in the network protocols
The binary protocol and the console text protocol have greeting messages:
$ nc localhost 3301 # binary port
Tarantool 2.10.0 (Binary) 84560a97-1cba-4e24-8814-d49fa1f9d6a5
P1T2S9A6/3sONmqeT1OVvRZh8fOYBv1c+C8NMC2FGZg=
$ nc localhost 3311 # console text port
Tarantool 2.10.0 (Lua console)
type 'help' for interactive help
It contains X.Y.Z version. This RFC does not change it.
However it worth to note that X.Y.Z-rcN would be reported as X.Y.Z here.
The same for a developer build: X.Y.Z-N-gHHHHHHHHH will be reported as X.Y.Z. It is the compromise to keep the compatibility with connectors and old
tarantool versions.
RPM package versions (CentOS, Fedora, openSUSE)
The version of tarantool's RPM package (Version field in the RPM spec) is
basically the same as tarantool's version, but alphaN, betaN, rcN
suffixes are separated by the tilde symbol (~) to order X.Y.Z~rc1 before X.Y.Z.
Tarantool version/build
tarantool --version
RPM version
<= 2.8
X.Y.Z-N-g<commit id>
X.Y.Z.N
>= 2.10 (release)
X.Y.Z-0-g<commit id>
X.Y.Z
>= 2.10 (pre-release)
X.Y.Z-rcN-0-g<commit id>
X.Y.Z~rcN
>= 2.10 (dev build)
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[~rcN].N.dev
Notes:
Where rcN is written, assumed alphaN, betaN as well.
For developer builds the special entrypoint mark also possible here.
The table shows the Version RPM spec field values and omits the Release
field values. The latter contains a build number and a dist tag: for example, 1.el7. The resulting RPM file name will contain both Version and Release values.
Idea: add git hash for developer builds
Now we format the Version and the Release field in our own way, but it is
possible to move toward the standard way described in Fedora Guidelines.
The recommended way is the following: X.Y.Z[~rcN]^NgitHHHHHHHHH. However the caret operator (^) is not
supported in RHEL7.
Fedora Guidelines proposes to use the following schema to support RHEL7:
This section does not allow to use a number for sorting,
it only allows the date. It seems the real snapshot packages follow it.
We should investigate this topic further. Including the <commit id> into
filename of an RPM package with a developer build sounds as the meaningful
idea.
Deb package versions (Debian, Ubuntu)
Tarantool version/build
tarantool --version
Deb version
<= 2.8
X.Y.Z-N-g<commit id>
X.Y.Z.N.g<commit id>
>= 2.10 (release)
X.Y.Z-0-g<commit id>
X.Y.Z
>= 2.10 (pre-release)
X.Y.Z-rcN-0-g<commit id>
X.Y.Z~rcN
>= 2.10 (dev build)
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[~rcN].N.dev
Notes:
2.10.0 was released as 2.10.0.g0a5ce0b9c, but it looks as a mistake.
Where rcN is written, assumed alphaN, betaN as well.
For developer builds the special entrypoint mark also possible here.
Idea: add git hash for developer builds
It seems meaningful to keep the <commit id> for developer builds in the
Debian package filename. We should investigate what is the common practice
here.
Source tarball
Tarantool version/build
tarantool --version
Source tarball
<= 2.8
X.Y.Z-N-g<commit id>
X.Y.Z.N
>= 2.10 (release)
X.Y.Z-0-g<commit id>
X.Y.Z
>= 2.10 (pre-release)
X.Y.Z-rcN-0-g<commit id>
X.Y.Z-rcN
>= 2.10 (dev build)
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[-rcN].N.dev
Notes:
Where rcN is written, assumed alphaN, betaN as well.
For developer builds the special entrypoint mark also possible here.
Homebrew formula (macOS)
We publish only releases (not pre-releases), so the question how to represent
alpha/beta/rc marks to keep version comparisons correct is irrelevant for us.
A release series stages
Early development
The stage goes on until a first release. Alpha, beta versions and pre-releases
are published within this stage.
In fact, this stage falls into two phases: development of a new functionality
and its stabilization.
A premature functionality may be removed on the alpha/beta stage, but it will
not be removed after publication of a release candidate.
Support
The stage starts when a first release is published. The release series now is
object of only backward compatible changes.
During this stage we're fixing all known security problems and fixing all found
degradations since the previous series.
A series receives not only degradation fixes, but also other bugfixes, till the
end of life.
The decision whether to fix a particular problem in a particular release series
is based on the impact of the problem, risks around backward compatibility and
complexity of backporting a fix.
A release series may receive new features on this stage, but only in the
backward compatible manner. A release candidate may be published for a new
functionality before a release.
During the support period we're adding new versions of supported Linux distros
to our build infrastructure.
A support period may be extended.
See the 'Delivery channels' section for information where to find in which
release a particular bug or a feature request will be resolved.
End of life
A series reaches the end of life, when the last release in the series is
published. The series will not receive updates anymore.
In modules, connectors and tools we don't guarantee support of a release series
that reaches EOL.
In fact, EOL cannot actually be reached until vast majority of productions
(where we have commitments / SLA) will be updated to a newer series.
Version string meaning
Nightly build
Those versions are not supposed to be used by customers. A versions string
contains -dev postfix.
Alpha
An alpha version is for early adopters and developers of dependent components
(such as connectors and modules).
It is early stage of a release series, the functionality may be incomplete or
unstable.
Beta
A beta version is good to start developing of a new application.
We start publishing beta versions, when all functionality planned for the
release series becomes implemented.
At this point we can reevaluate readiness of a feature and decide to remove it
from the series (to finish it later or even to replace with something else).
A beta version may have a known bug in the new functionality or a known
degradation since a previous release series that affects a common use case
(unlike a release candidate).
Release candidate
A release candidate fits good to setup a staging server.
In fact, we have two kinds of a release candidate. First, during early
development, when the series goes to be mature enough. Second, on the support
stage, to collect feedback prior to an upcoming release.
The key difference between beta and release candidate is maturity of the new
functionality. The formal rules are:
No known bugs in typical usage scenarious for a new functionality.
No known degradations since a previous release series.
Release
A release is the version that is ready for production usage.
The requirements are the same as for a release candidate. Aside of this, we may
perform extra pre-release testing and adoption in our internal projects if
there are doubts regarding stability.
Delivery channels
Tarantool is delivered through APT and YUM repositories.
For each release series and each supported Linux distribution we offer two
repositories: one for releases and one for pre-releases (alpha, beta, release
candidate).
For macOS brew offers a latest release and tarantool/tap offers all
supported series.
We also provide source tarballs.
Release notes are stored on the GitHub releases page. A letter regarding a new
release is published in the 'Tarantool news' Telegram channel.
Upcoming changes
We have no a good channel to share information about upcoming changes, but
there are plans to provide a high level plan. Possibly using GitHub's projects.
Previously the information regarding problems to be fixed in upcoming releases
was delivered using GitHub milestones, but it poorly works with our parallel
feature sets/release series development.
A bug may appear in several release series, a feature may be planned for
several release series. In this case the issue was marked with a milestone for
lowest applicable release. It is not very convenient.
A repository structure
CentOS, Fedora, openSUSE
Repositories are defined in the /etc/yum.repos.d/*.repo file. A full file for
a tarantool's repository can be found on Tarantool's download page. The main
directive of the file is baseurl, it is where the repository root resides
from a user point of view.
The old URL is constructed as follows (line breaks are for readability, the
real file will not have them):
It works for tarantool 2.10+. The 'modules' repository is independent from
tarantool versions and can be enabled to use with any supported tarantool
version.
Debian, Ubuntu
Repositories are defined in the /etc/apt/sources.list.d/*.list file. A full
file for a tarantool's repository can be found on Tarantool's download page.
Each line describes a repository, for example deb https://example.org/foo/bar buster main.
The old URL is constructed as follows (line breaks are for readability, the
real file will not have them):
https://download.tarantool.org
/tarantool
/{live,release}
/{2.7,2.8,...}
/{debian,ubuntu} {buster,focal,...} main
Example of the /etc/apt/sources.list.d/tarantool_2_8.list file:
deb https://download.tarantool.org/tarantool/release/2.8/debian buster main
deb-src https://download.tarantool.org/tarantool/release/2.8/debian buster main
It works for 1.10 and {2.1-2.8} series.
See the 'CentOS, Fedora, openSUSE' section regarding changes in the repository layout.
The URL for the series 2 is constructed as follows (line breaks are for readability):
deb https://download.tarantool.org/tarantool/release/series-2/debian buster main
deb-src https://download.tarantool.org/tarantool/release/series-2/debian buster main
It works for tarantool 2.10+. See the 'CentOS, Fedora, openSUSE' section
regarding 'modules' repository.
Source tarballs
GitHub provides its own archives for releases, but they do not contain git
submodules and generated VERSION file, so they're not suitable to build
tarantool from sources.
We archive source tarballs correctly and make them available from our website.
The new release policy changes this layout: now we place all tarballs into one
directory. The main consumers of the source tarballs are automatic build
scripts in OS distributions (for example, Gentoo, Arch, FreeBSD). It is a bit
tricky to determine correct directory of the tarball automatically, when it
requires to parse tarantool version.
Pre-release and release source tarballs are there.
It works for tarantool-2.10+.
Development flow
A git branch tracks certain feature set and so contains a major and a
minor version: 3.0, 3.1 and so on. The latest branch is always named master.
Since the whole series is backward compatible, we stop backporting to
the X.Y branch when a first release of the X.(Y+1) branch is
published.
Push a bugfix
Push to master.
Cherry-pick to X.Y branches for all non-EOL release series (where the bug
is present).
A series have two active branches if it have a released feature set and a
new feature set in development. Otherwise it has one active branch.
Consider publishing pre-releases and releases.
Push a feature
If it is part of a new feature set or a new release series, proceed with the
corresponding section below.
Push to master.
Cherry-pick to X.Y branches according to agreements on which release series
the feature should appear.
Consider publishing pre-releases and releases.
Create a new feature set
Let's assume as have X.Y feature set (that lives in master) and want to
start the X.(Y+1) feature set.
Push current master to the new X.Y branch.
Add a dummy commit to master.
Add annotated tag X.(Y+1).0-entrypoint.
The dummy commit is necessary, because the new entrypoint tag MUST NOT be
reachable from the X.Y branch. It would break git describe and so would
make nightly builds numeration horrible.
Create a new release series
The same as creating the (X+1).0 feature set:
Push current master to the new X.Y branch.
Add a dummy commit to master.
Add annotated tag (X+1).0.0-entrypoint.
Publish alpha
When:
When the functionality planned for a release series is not fully implemented.
What:
Any meaningful set of changes may be published as a new alpha version. Say,
when an issue or set of issues are closed.
How:
Add annotated tag X.Y.0-alphaN. List changes in the tag message (since the
previous tag).
Publish beta
When:
When all functionality planned for a release series is implemented.
What:
Same as for alpha, any meaningful set of changes.
How:
Add annotated tag X.Y.0-betaN. List changes in the tag message (since the
previous tag).
Publish release candidate
When:
We don't know bugs in typical usage scenarious for a new functionality.
We don't know degradations since a previous release series.
What:
Again, any meaningful set of changes.
How:
Add annotated tag X.Y.Z-rcN. List changes in the tag message (since the
previous tag).
Publish release
Add annotated tag X.Y.Z. Other activities around publishing a release (before
and after setting a tag) are beyond the scope of this document.
Development flow in examples
For example, we develop series 3 and have version 3.1.1 released. The latest
feature set is 3.1 and it lives in the master branch:
Now we decided that 3.2 is stable enough to release 3.2.0 and stop
publishing 3.1.X releases:
(we don't push to 3.1 anymore: there is a 3.2.X release)
master: -*-*
^ ^
| |
| dummy commit
| tag: 3.2.0
tag: 3.2.0-rc2
The dummy commit is necessary for our build machinery: the same commit should
be identified either as a release candidate or as a release in an unambiguous
way.
Creating of a new release series is the same as creating a new feature set from
the development flow side of view. The difference is in the rules for pushing
to those branches.
While a release series is in early development stage:
...the whole series lives in one branch (master). We don't need to split it
to separate feature sets, because the series have no releases yet.
Backporting
How to determine, to which branches a patchset should be applied?
Say, we have two release series (2 and 3) and their latest releases are 2.12.0 and 3.2.0. Aside of this, we have 3.3 feature set in development,
but there were no 3.3.X releases yet:
First, a patchset always goes to master (if it is not very specific for, say,
series 2). Next steps depend of a nature of the patch.
Bugfix
A bugfix should be backported to all release series, where it is applicable. If
there are two branches within a series (as for series 3 in our example), push
to all:
If a patch is about the new functionality, the key question is to which release
series it is planned. Say, we want this functionality only in series 3:
If the functionality is planned for series 2 as well, we should create the 2.13 branch, push a dummy commit, set the entrypoint tag and cherry-pick the
feature here:
The status of the document becomes just 'early draft', without 'it is
too abstract and so on'. Added 'there are important details to be
(re)considered'.
Minor wording tweaks.
Added many 'to be done' marks across the document to define important details
or even reconsider some processes.
Invariants / Early development: explicitly mention the stabilization
phase.
Invariants / Support: added note that we add support of new versions of Linux
distributions during the support period.
Invariants / Support: added note re bugfixes that're not degradations.
Invariants / Support: added note re new features (possible, but
usually we don't).
Current release policy: drop transposed table (information
duplication).
Current release policy: explicitly mark the stabilization phases.
Proposed release policy: eliminated the 'not so much release series in
a fly' goal, because we'll anyway estimate cons and pros of starting a
new series each time we'll want to do so.
Proposed release policy: clarified that a development / stabilization
times is highly depends on a nature of a
new functionality.
Proposed release policy: don't assume any frozen schedule for forking
new release series. It'll always depend of amount of required work for
designing, implementing and stabilizing the new functionality.
Transition plan / Blockers: removed the section, because in fact it
lists our plans for series 2.
Pre-releases / Criterias: added the section. Removed 'doubts' from
'Proposed release namings' so.
Pre-releases / Delivery channels: added the section.
Transition plan: reworded with decision to continue 2 series under the
new release policy.
Proposed release naming: add question and thoughts re 3-alpha1 vs 3.0.0-alpha1.
Proposed release naming: add question when to bump a minor version.
v2 -> v3
Status of the document: removed the sentence re adding a feature into
a series.
Status of the document: add 'to be done' entry re structure of the document.
Current release policy: drop the section.
Current release naming (versioning): drop the section.
Proposed release naming (versioning): allow new features with
increasing a minor version, also fixed 'Support' stage description,
added a note re a feature set entrypoint to the 'Criterias/Entrypoint'
section.
Development flow: added the section.
Invariants: give compatibility guarantees for pre-releases.
Invariants: explicitly state that a feature may be removed on the
alpha/beta stage, but not after a release candidate.
Proposed release naming (versioning): hold 3.0.0-alpha1 tags
(discard 3-alpha1 and 3.0-alpha1), made the similar update in the
'Pre-releases' section.
Proposed release policy: added a parapraph regarding how often
releases will be published -- no precise schedule here.
Proposed release naming (versioning): removed the 'Can we offer a
release with a feature preview?' question, because it seems we usually
should not do this.
Transition plan: added near and further steps.
v3 -> v4
Clarified how we mark issues with milestones (lowest applicable series).
Clarified EOL stage a bit, highlighted that a support time may be extended.
Restructured the document: the main change is that now common description of
the policy is splitted from developer side operations (branches, tags and so
on).
Updated compatibility guarantees with more details.
v4 -> v5
Summary: added the section suggested by Alexey Koryakin.
v5 -> v6
Update wording here and there before publication.
Transition plan: drop obsolete items, add a bit more notes to the 'What can
going bad?' subsection.
v6 -> v7
Improved and clarified Lua and SQL compatibility guarantees.
Define a first release in the new series 2: 2.10.0.
v7 -> v8
Table of Contents: added the section.
Version naming: added new subsections:
How the version is reported
Releases and pre-releases
tarantool --version
A greeting in the network protocols
RPM package versions (CentOS, Fedora, openSUSE)
Deb package versions (Debian, Ubuntu)
Source tarball
Homebrew formula (macOS)
The sections have tables to illustrate the version format in each particular
case.
It also contains ideas how to name packages for developer builds, but it is
not investigated deeply yet.
This RFC version rejects the idea with -dev suffix for the tarantool --version output. See 'Reasoning' in the related section.
Delivery channels: changelogs for pre-releases are now published on GitHub,
updated the section accordingly.
Upcoming changes: reject milestones as information about upcoming changes --
unlikely they're in an actual state now.
A repository structure: added new subsections:
CentOS, Fedora, openSUSE
Debian, Ubuntu
Source tarballs
Each section describes how the base repository URL is constructed for old and
new (pre-)releases and illustares it with examples.
Transition plan: dropped the section, it is not actual anymore.
v8 -> v8.post1
Added HTML anchors to make the ToC working in GitHub Discussions.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Summary
To simplify Tarantool builds' identification, the versioning policy for stable
releases is changing and will take effect for builds starting with 2.10.0
numbers as well as future major series ones (3.0.0+).
The new version numbering scheme replaces the legacy policy
and changes to semver-like MAJOR.MINOR.PATCH format for releases and to
MAJOR.MINOR.PATCH-<pre-release suffix> for pre-releases. Pre-releases are
marked with suffixes "-alphaN", "-betaN" and "-rcN" explicitly so that users
could avoid installing these versions on production systems. Backwards
compatibility is guaranteed between minor builds in the same major release
series and is appreciated but not guaranteed between different major numbers.
The topics below describe the new versioning policy in great detail.
Table of Contents
Status of the document
Version: 8.post1.
Proposal.
Audience
This document is the exhaustive description of the new release policy. Its main
purpose is to collect feedback regarding the policy, so the audience is
tarantool developers, application developers, administrators, architects and
everyone who is interested in the discussion.
The document gives many details, quite heavy and supposed to be a base for a
set of different documents / webpages:
For customers:
For development:
Goals
transparency for users (we want to shift closer to semver).
expectations. At least two years since a first release.
Applicability
Existing
2.X.Y
releases follow the old release policy, where2.X
was arelease series. The new policy is in effect since 2.10.0.
All future series must follow this document.
Terms
Pre-release
A frozen commit for early adopters, preview of a future release.
Release
A frozen commit that we advertize as production ready.
Release / pre-release version
An unique identifier of a pre-release or a release.
Release series
A product with linear evolution, pre-release and release points and certain
compatibility guarantees within the series and between series.
Backward compatibility (binary data layout)
A newer release (its runtime) is backward compatible in this sense with an
older one when the newer release is operational when working on top of data
(
*.xlog
,*.snap
,*.vylog
,*.run
) from the older release. Allfunctionality that is part of the older release is working in this
configuration.
An attempt to use a new functionality either successful or give a meaningful
error until the database schema upgrade: it does not lead to a service outage
or data corruption. An instance is able to upgrade the data layout using the
box.schema.upgrade()
call to enable all features of the new release (whenall instances of the replicaset are run on the same tarantool version).
Backward compatibility (binary protocol)
All binary protocol requests that were operational in an older release are
remain operational on a newer release and does not change meaning. Responses
have the same format, but mappings may have fields that were not present in
the older release.
net.box
client of the older release able to work with the newer release(except, maybe, functionality introduced in the newer release).
net.box
client of the newer release is fully operational with the server that is run
under the older release (except, maybe, functionality that the older release
does not implement).
Backward compatibility (replication protocol)
A instance that is run on a newer release may work as upstream (master) of an
instance with an older release or as downstream (replica) without database
schema upgrade.
The database schema upgrade (
box.schema.upgrade()
) must be performed whenall replicaset instances run on the same tarantool version. The upgrade does
not cause downtime (if the application does not lean on internal schema
representation).
Backward compatibility (Lua code)
A code that are run on an older release is run with the same effect on a
newer release. However only meaningful code counts: if any code just give an
error, but starts doing something useful, the change is considered
compatible.
A room for new functionality is still here: we can add new arguments to the
end and options (fields in a table argument), add more fields to a returning
table, add more returning values (multireturn).
Adding a new built-in module or a new global value is considered as the
compatible change.
Adding a new field to an existing metatable is okay if it is not listed in the
Lua 5.1 Reference Manual. Otherwise it should be proven that it may not broke
any meaningful code.
Examples of what is okay:
__pairs
,__ipairs
to a metatable of a userdata/cdata (because it isnot from Lua 5.1 and because userdata/cdata has no default behaviour for
pairs()
andipairs()
calls).__lt
,__le
metamethod (if the attempt to use<
,<=
and so on leads to an error before the change -- had no defined behaviour).
__eq
metamethod implementation for a case, where it leadsto en error before the change.
Examples of NOT compatible changes:
__pairs
,__ipairs
to a metatable of a table (because it already hasits own behaviour before the change).
__eq
metamethod if it was not defined (default behaviour is definedfor any pair of Lua objects).
Backward compatibility (SQL code)
The same principle: if any request is run with the same effect, the change is
compatible (don't counting requests that always lead to an error).
What is okay:
of types [Y] if [operation from {X}]([list of values of [Y] types]) is not
implemented before the change (raising an error).
SELECT
's resultset ifORDER BY
is not specified.Technically those changes may break a working code in case of a name clash,
but the probability of this situation is considered as negligible. (We can
restrict this rule in a future.)
Examples of NOT compatible changes:
Backward compatibility (C code)
A module or a C stored procedure that is run on an older release is
run with the same effect on a newer release.
It is completely okay to add a new function or a new structure to the public
C API. It must use one of tarantool prefixes (
box_
,fiber_
,luaT_
,luaM_
and so on) or introduce a new one.A symbol from a library we use must not be exported directly (because the
library may be used in a module per se and the clash may lead to problems).
The exception is when we export the whole public API of the library (as for
libcurl
).Don't introduce new functions or structures with the
lua_
andluaL_
prefixes. Those prefixes are for the Lua runtime. Use
luaT_
for tarantoolspecific functions and
luaM_
for general-purpose ones.Compatibility rules
Pre-releases and releases of one release series are compatible in all
senses defined above (any release with any release).
Pre-releases and releases of consequent series are compatible by data
layout, binary protocol and replication protocol.
We don't give any guarantees regarding compatibility between
pre-releases/releases of non-consequent release series if the opposite
is not stated in the release notes.
We don't give any guarantees regarding compatibility between alpha/beta
versions and between alpha/beta and pre-release/release even within one
series.
A release series lifecycle
A release series goes over the following stages:
The sections below describe various details about those stages.
Version naming
Everything spins around the three digit release naming like
3.1.2
. Here therelease series is
3
(major version).3.1
identifies a feature set, theminor version is increased with new features.
3.1.2
and3.1.3
may differ byfixed bugs.
During early development we publish alpha, beta and release candidates, they
are marked this way:
3.0.0-alphaN
,3.0.0-betaN
,3.0.0-rcN
(N
is anumber).
During the support stage we also can publish a release candidate, it'll be
marked similarly:
3.2.0-rcN
.How the version is reported
A user can see the version in different contexts and, in fact, it is reported
differently in those contexts. Let's define those contexts.
tarantool --version
and friends.The following subsections will describe how versions are reported in those
contexts.
Releases and pre-releases
It is what a maintainer set as a git tag and what a user sees on the releases
page.
The table below summarizes possible values.
X.Y.Z
X.Y.Z
X.Y.Z-rcN
Notes:
rcN
is written, assumedalphaN
,betaN
as well.tarantool --version
This version is reported in the following places:
tarantool --version
output._TARANTOOL
global variable in Lua.box.info.version
expression in Lua.require('tarantool').version
expression in Lua.PACKAGE_VERSION
macro (string literal) inmodule.h
.It is
git describe --always --long <commit id>
result for tarantool builtfrom
<commit id>
.The table below summarizes possible values.
X.Y.Z-N-g<commit id>
X.Y.Z-0-g<commit id>
X.Y.Z-rcN-0-g<commit id>
X.Y.Z[-rcN]-N-g<commit id>
Notes:
rcN
is written, assumedalphaN
,betaN
as well.entrypoint
mark also possible here.Tarantool Enterprise versioning is out of scope of this document.
Reasoning
It is technically possible to trim the version to
X.Y.Z
andX.Y.Z-rcN
(
alphaN
,betaN
) for (pre-)releases, however we want to keep it as is.The motivation is the following:
<commit id>
when a problem is reported by auser to a developer. It is possible that a git tag is added locally. As a
side effect, it is just convenient to checkout the particular revision using
the git hash.
cartridge-cli
) leans on existing version format. Pre-releases havealphaN
,betaN
,rcN
suffixes, the format is inevitably changed here.However we can keep it unchanges for tarantool releases, so the old tools
will continue to work without changes on them.
It is also technically possible to add something like
-dev
suffix into theversion string on a developer build. It would help a user to easily distinguish
(pre-)release version from a developer build.
However we want to avoid the change for the same of better compatibility and
stability:
support this
-dev
suffix.means that the version parsing code that takes care only to (pre-)releases
will not fail on a developer build.
It is possible to deliver user-friendly information regarding type of the build
(release, pre-release, developer build) in a less intrusive way. We should
elaborate those possibilities.
A greeting in the network protocols
The binary protocol and the console text protocol have greeting messages:
It contains
X.Y.Z
version. This RFC does not change it.However it worth to note that
X.Y.Z-rcN
would be reported asX.Y.Z
here.The same for a developer build:
X.Y.Z-N-gHHHHHHHHH
will be reported asX.Y.Z
. It is the compromise to keep the compatibility with connectors and oldtarantool versions.
RPM package versions (CentOS, Fedora, openSUSE)
The version of tarantool's RPM package (
Version
field in the RPM spec) isbasically the same as tarantool's version, but
alphaN
,betaN
,rcN
suffixes are separated by the tilde symbol (
~
) to orderX.Y.Z~rc1
beforeX.Y.Z
.X.Y.Z-N-g<commit id>
X.Y.Z.N
X.Y.Z-0-g<commit id>
X.Y.Z
X.Y.Z-rcN-0-g<commit id>
X.Y.Z~rcN
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[~rcN].N.dev
Notes:
rcN
is written, assumedalphaN
,betaN
as well.entrypoint
mark also possible here.Version
RPM spec field values and omits theRelease
field values. The latter contains a build number and a dist tag: for example,
1.el7
. The resulting RPM file name will contain bothVersion
andRelease
values.Idea: add git hash for developer builds
Now we format the
Version
and theRelease
field in our own way, but it ispossible to move toward the standard way described in Fedora Guidelines.
The recommended way is the following:
X.Y.Z[~rcN]^NgitHHHHHHHHH
. However the caret operator (^
) is notsupported in RHEL7.
Fedora Guidelines proposes to use the following schema to support RHEL7:
This section does not allow to use a number for sorting,
it only allows the date. It seems the real snapshot packages follow it.
We should investigate this topic further. Including the
<commit id>
intofilename of an RPM package with a developer build sounds as the meaningful
idea.
Deb package versions (Debian, Ubuntu)
X.Y.Z-N-g<commit id>
X.Y.Z.N.g<commit id>
X.Y.Z-0-g<commit id>
X.Y.Z
X.Y.Z-rcN-0-g<commit id>
X.Y.Z~rcN
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[~rcN].N.dev
Notes:
2.10.0
was released as2.10.0.g0a5ce0b9c
, but it looks as a mistake.rcN
is written, assumedalphaN
,betaN
as well.entrypoint
mark also possible here.Idea: add git hash for developer builds
It seems meaningful to keep the
<commit id>
for developer builds in theDebian package filename. We should investigate what is the common practice
here.
Source tarball
X.Y.Z-N-g<commit id>
X.Y.Z.N
X.Y.Z-0-g<commit id>
X.Y.Z
X.Y.Z-rcN-0-g<commit id>
X.Y.Z-rcN
X.Y.Z[-rcN]-N-g<commit id>
X.Y.Z[-rcN].N.dev
Notes:
rcN
is written, assumedalphaN
,betaN
as well.entrypoint
mark also possible here.Homebrew formula (macOS)
We publish only releases (not pre-releases), so the question how to represent
alpha/beta/rc marks to keep version comparisons correct is irrelevant for us.
A release series stages
Early development
The stage goes on until a first release. Alpha, beta versions and pre-releases
are published within this stage.
In fact, this stage falls into two phases: development of a new functionality
and its stabilization.
A premature functionality may be removed on the alpha/beta stage, but it will
not be removed after publication of a release candidate.
Support
The stage starts when a first release is published. The release series now is
object of only backward compatible changes.
During this stage we're fixing all known security problems and fixing all found
degradations since the previous series.
A series receives not only degradation fixes, but also other bugfixes, till the
end of life.
The decision whether to fix a particular problem in a particular release series
is based on the impact of the problem, risks around backward compatibility and
complexity of backporting a fix.
A release series may receive new features on this stage, but only in the
backward compatible manner. A release candidate may be published for a new
functionality before a release.
During the support period we're adding new versions of supported Linux distros
to our build infrastructure.
A support period may be extended.
See the 'Delivery channels' section for information where to find in which
release a particular bug or a feature request will be resolved.
End of life
A series reaches the end of life, when the last release in the series is
published. The series will not receive updates anymore.
In modules, connectors and tools we don't guarantee support of a release series
that reaches EOL.
In fact, EOL cannot actually be reached until vast majority of productions
(where we have commitments / SLA) will be updated to a newer series.
Version string meaning
Nightly build
Those versions are not supposed to be used by customers. A versions string
contains
-dev
postfix.Alpha
An alpha version is for early adopters and developers of dependent components
(such as connectors and modules).
It is early stage of a release series, the functionality may be incomplete or
unstable.
Beta
A beta version is good to start developing of a new application.
We start publishing beta versions, when all functionality planned for the
release series becomes implemented.
At this point we can reevaluate readiness of a feature and decide to remove it
from the series (to finish it later or even to replace with something else).
A beta version may have a known bug in the new functionality or a known
degradation since a previous release series that affects a common use case
(unlike a release candidate).
Release candidate
A release candidate fits good to setup a staging server.
In fact, we have two kinds of a release candidate. First, during early
development, when the series goes to be mature enough. Second, on the support
stage, to collect feedback prior to an upcoming release.
The key difference between beta and release candidate is maturity of the new
functionality. The formal rules are:
Release
A release is the version that is ready for production usage.
The requirements are the same as for a release candidate. Aside of this, we may
perform extra pre-release testing and adoption in our internal projects if
there are doubts regarding stability.
Delivery channels
Tarantool is delivered through APT and YUM repositories.
For each release series and each supported Linux distribution we offer two
repositories: one for releases and one for pre-releases (alpha, beta, release
candidate).
For macOS
brew
offers a latest release andtarantool/tap
offers allsupported series.
We also provide source tarballs.
Release notes are stored on the GitHub releases page. A letter regarding a new
release is published in the 'Tarantool news' Telegram channel.
Upcoming changes
We have no a good channel to share information about upcoming changes, but
there are plans to provide a high level plan. Possibly using GitHub's projects.
Previously the information regarding problems to be fixed in upcoming releases
was delivered using GitHub milestones, but it poorly works with our parallel
feature sets/release series development.
A bug may appear in several release series, a feature may be planned for
several release series. In this case the issue was marked with a milestone for
lowest applicable release. It is not very convenient.
A repository structure
CentOS, Fedora, openSUSE
Repositories are defined in the
/etc/yum.repos.d/*.repo
file. A full file fora tarantool's repository can be found on Tarantool's download page. The main
directive of the file is
baseurl
, it is where the repository root residesfrom a user point of view.
The old URL is constructed as follows (line breaks are for readability, the
real file will not have them):
Example:
It works for 1.10, and {2.1-2.8} series.
The series 2 (tarantool 2.10+) and following release series have different layout:
MAJOR.MINOR
.The URL for the series 2 is constructed as follows (line breaks are for
readability):
Example:
It works for tarantool 2.10+. The 'modules' repository is independent from
tarantool versions and can be enabled to use with any supported tarantool
version.
Debian, Ubuntu
Repositories are defined in the
/etc/apt/sources.list.d/*.list
file. A fullfile for a tarantool's repository can be found on Tarantool's download page.
Each line describes a repository, for example
deb https://example.org/foo/bar buster main
.The old URL is constructed as follows (line breaks are for readability, the
real file will not have them):
Example of the
/etc/apt/sources.list.d/tarantool_2_8.list
file:It works for 1.10 and {2.1-2.8} series.
See the 'CentOS, Fedora, openSUSE' section regarding changes in the repository layout.
The URL for the series 2 is constructed as follows (line breaks are for readability):
Example:
It works for tarantool 2.10+. See the 'CentOS, Fedora, openSUSE' section
regarding 'modules' repository.
Source tarballs
GitHub provides its own archives for releases, but they do not contain git
submodules and generated
VERSION
file, so they're not suitable to buildtarantool from sources.
We archive source tarballs correctly and make them available from our website.
The old URL is constructed as follows:
Example:
Release and per-push tarballs -- all are there.
It works for 1.10 and {2.1-2.8} series.
The new release policy changes this layout: now we place all tarballs into one
directory. The main consumers of the source tarballs are automatic build
scripts in OS distributions (for example, Gentoo, Arch, FreeBSD). It is a bit
tricky to determine correct directory of the tarball automatically, when it
requires to parse tarantool version.
The URL for series 2 is constructed as follows:
Examples:
Pre-release and release source tarballs are there.
It works for tarantool-2.10+.
Development flow
A git branch tracks certain feature set and so contains a major and a
minor version:
3.0
,3.1
and so on. The latest branch is always namedmaster
.Since the whole series is backward compatible, we stop backporting to
the
X.Y
branch when a first release of theX.(Y+1)
branch ispublished.
Push a bugfix
master
.X.Y
branches for all non-EOL release series (where the bugis present).
new feature set in development. Otherwise it has one active branch.
Push a feature
corresponding section below.
master
.X.Y
branches according to agreements on which release seriesthe feature should appear.
Create a new feature set
Let's assume as have
X.Y
feature set (that lives inmaster
) and want tostart the
X.(Y+1)
feature set.master
to the newX.Y
branch.master
.X.(Y+1).0-entrypoint
.The dummy commit is necessary, because the new entrypoint tag MUST NOT be
reachable from the
X.Y
branch. It would breakgit describe
and so wouldmake nightly builds numeration horrible.
Create a new release series
The same as creating the
(X+1).0
feature set:master
to the newX.Y
branch.master
.(X+1).0.0-entrypoint
.Publish alpha
When:
When the functionality planned for a release series is not fully implemented.
What:
Any meaningful set of changes may be published as a new alpha version. Say,
when an issue or set of issues are closed.
How:
Add annotated tag
X.Y.0-alphaN
. List changes in the tag message (since theprevious tag).
Publish beta
When:
When all functionality planned for a release series is implemented.
What:
Same as for alpha, any meaningful set of changes.
How:
Add annotated tag
X.Y.0-betaN
. List changes in the tag message (since theprevious tag).
Publish release candidate
When:
What:
Again, any meaningful set of changes.
How:
Add annotated tag
X.Y.Z-rcN
. List changes in the tag message (since theprevious tag).
Publish release
Add annotated tag
X.Y.Z
. Other activities around publishing a release (beforeand after setting a tag) are beyond the scope of this document.
Development flow in examples
For example, we develop series 3 and have version
3.1.1
released. The latestfeature set is
3.1
and it lives in themaster
branch:We want to create a new feature set
3.2
.3.1
branch is forked andmaster
becomes a place for
3.2
:Now we continue development of features in
3.2
:And bugfixes for both
3.2
and3.1
:Now we decided that
3.2
is stable enough to release3.2.0
and stoppublishing
3.1.X
releases:The dummy commit is necessary for our build machinery: the same commit should
be identified either as a release candidate or as a release in an unambiguous
way.
Creating of a new release series is the same as creating a new feature set from
the development flow side of view. The difference is in the rules for pushing
to those branches.
While a release series is in early development stage:
...the whole series lives in one branch (
master
). We don't need to split itto separate feature sets, because the series have no releases yet.
Backporting
How to determine, to which branches a patchset should be applied?
Say, we have two release series (
2
and3
) and their latest releases are2.12.0
and3.2.0
. Aside of this, we have3.3
feature set in development,but there were no
3.3.X
releases yet:First, a patchset always goes to
master
(if it is not very specific for, say,series 2). Next steps depend of a nature of the patch.
Bugfix
A bugfix should be backported to all release series, where it is applicable. If
there are two branches within a series (as for series 3 in our example), push
to all:
New releases may be published after a bugfix:
Feature
If a patch is about the new functionality, the key question is to which release
series it is planned. Say, we want this functionality only in series 3:
If the functionality is planned for series 2 as well, we should create the
2.13
branch, push a dummy commit, set the entrypoint tag and cherry-pick thefeature here:
If we consider the feature as mature enough, we can release
2.13.0
:Authors
Persons who drives the changes, express, discuss and challenge
proposals.
Changelog
v1 -> v2
too abstract and so on'. Added 'there are important details to be
(re)considered'.
or even reconsider some processes.
phase.
distributions during the support period.
usually we don't).
duplication).
a fly' goal, because we'll anyway estimate cons and pros of starting a
new series each time we'll want to do so.
times is highly depends on a nature of a
new functionality.
new release series. It'll always depend of amount of required work for
designing, implementing and stabilizing the new functionality.
lists our plans for series 2.
'Proposed release namings' so.
new release policy.
3-alpha1
vs3.0.0-alpha1
.v2 -> v3
a series.
increasing a minor version, also fixed 'Support' stage description,
added a note re a feature set entrypoint to the 'Criterias/Entrypoint'
section.
alpha/beta stage, but not after a release candidate.
3.0.0-alpha1
tags(discard
3-alpha1
and3.0-alpha1
), made the similar update in the'Pre-releases' section.
releases will be published -- no precise schedule here.
release with a feature preview?' question, because it seems we usually
should not do this.
v3 -> v4
the policy is splitted from developer side operations (branches, tags and so
on).
v4 -> v5
v5 -> v6
going bad?' subsection.
v6 -> v7
v7 -> v8
Table of Contents: added the section.
Version naming: added new subsections:
The sections have tables to illustrate the version format in each particular
case.
It also contains ideas how to name packages for developer builds, but it is
not investigated deeply yet.
This RFC version rejects the idea with
-dev
suffix for thetarantool --version
output. See 'Reasoning' in the related section.Delivery channels: changelogs for pre-releases are now published on GitHub,
updated the section accordingly.
Upcoming changes: reject milestones as information about upcoming changes --
unlikely they're in an actual state now.
A repository structure: added new subsections:
Each section describes how the base repository URL is constructed for old and
new (pre-)releases and illustares it with examples.
Transition plan: dropped the section, it is not actual anymore.
v8 -> v8.post1
Beta Was this translation helpful? Give feedback.
All reactions